Implementing a neural network can include receiving a macro instruction for implementing the neural network within a control unit of a neural network processor. The macro instruction can indicate a first data set, a second data set, a macro operation for the neural network, and a mode of operation for performing the macro operation. The macro operation can be automatically initiated using a processing unit of the neural network processor by applying the second data set to the first data set based on the mode of operation.