A convolution neural network (CNN)-based image processing method and apparatus are provided. The CNN-based image processing method includes identifying whether values of pixels of each of feature maps having a plurality of channels at a first layer are zero, and storing information regarding a result of identifying whether the values of the pixels are zero; writing image feature information of the feature maps at the first layer to an external memory; reading information regarding pixels having values which are not zero among the written image feature information from the external memory based on the information regarding the result of identifying whether the values of the pixels are zero; and performing a feature map operation at a second layer using the read image feature information of the feature maps.