Methods and apparatus, including computer program products, are provided for watermarking neural networks. In some embodiments, there may be provided a method. The method may include determining, for a neural network, an activation layer output by a hidden layer of the neural network. The method may include selecting a watermarking process. The method may include applying the selected watermarking process to the activation layer output to generate a key. The method may include storing, for the neural network to enable detection of copying of the neural network, the selected watermarking process and the key. Related systems, methods, and articles of manufacture are also described.