A convolution operation method includes the following steps of: performing convolution operations for data inputted in channels, respectively, so as to output a plurality of convolution results; and alternately summing the convolution results of the channels in order so as to output a sum result. A convolution operation device executing the convolution operation method is also disclosed.