A method and apparatus for compressing a neural network are provided. A specific embodiment of the method includes: acquiring a to-be-compressed trained neural network; selecting at least one layer from layers of the neural network as a to-be-compressed layer; performing following processing steps sequentially on each of the to-be-compressed layers in descending order of the number of level of the to-be-compressed layer: determining a pruning ratio based on a total number of parameters included in the to-be-compressed layer, selecting a parameter for pruning from the parameters included in the to-be-compressed layer based on the pruning ratio and a parameter value threshold, and training the pruned neural network based on a preset training sample using a machine learning method; and determining the neural network obtained after performing the processing steps on the selected at least one to-be-compressed layer as a compressed neural network, and storing the compressed neural network.