Blocks in two different frames are subjected to Hadamard transform by first and second Hadamard transform circuits, and a sum component and a difference component are calculated for the components of the two blocks corresponding to each other to perform three-dimensional Hadamard transform, so that detection is made as to whether or not these blocks include a motion. While the sum component and the difference component are encoded for the block determined as including a motion, a motion detection result, rather than the difference component, is encoded for the block determined as including no motion. As a result, a high compression ratio can be achieved in video encoding with a small-scale circuit.