A method and apparatus for accelerating a convolutional neural network. The method comprises: splitting, according to rows, a weight matrix of a convolutional layer into a plurality of weight segments, and respectively caching the plurality of weight segments to a plurality of calculation units in a calculation unit array (step 301); reading a plurality of input data streams respectively corresponding to the plurality of weight segments, and inputting the plurality of input data streams in parallel into the plurality of calculation units (step 302), wherein the input data streams are formed by means of splicing a plurality of rows of data in an input feature map of the convolutional layer; and within each calculation unit, performing a sliding window operation and a multiply-accumulate computation on the input data streams on the basis of the cached weight segments, so as to obtain an output feature map of the convolutional layer (step 303). By means of the method, data that enters each row of calculation units is continuously read according to the whole row of data, and does not need to be read in a cross-row or cross-column manner, such that a special design is not needed for the arrangement of a memory, convolutions of different sizes are supported, and the function of Im2col does not need to be realized, thereby reducing the complexity.