A motion vector detection apparatus that obtains, for each of the blocks of a predetermined size into which one frame is divided, motion vector information in units of a plurality of partitions of differing rectangular shapes and differing sizes, up to the size of the blocks, has a partition connector means (23) that, when motion vectors for a plurality of partitions are obtained for one of the blocks, decides whether or not to connect a plurality of the partitions for which the motion vectors are obtained, and if the partitions are connected, includes motion vector information about the resulting connected partition in the motion vector information output from the motion vector detection apparatus. The motion vector encoding rate can be reduced.