A method and device for training a neural network are disclosed. The method comprises: selecting, by a training device, a teacher network performing the same functions of a student network; and iteratively training the student network and obtaining a target network, through aligning distributions of features between a first middle layer and a second middle layer corresponding to the same training sample data, so as to transfer knowledge of features of a middle layer of the teacher network to the student network.