A motion detection apparatus includes a CPU. The CPU detects partial motion vectors of an object scene corresponding to motion detection blocks based on raw image data. The CPU also reduces a weighted amount allocated to a motion detection block which captures a partial object scene having a contrast below a reference out of the motion detection blocks. Furthermore, the CPU performs a weighted average process referring to the weighted amount on the partial motion vectors so as to detect a motion of the entire object scene. The CPU obtains a minimum correlation value of a luminance difference between frames of the raw image data corresponding to each of the motion detection blocks, and executes a process for increasing the reference according to an increase of the obtained minimum correlation value corresponding to the motion detection block which captures the partial object scene having a contrast below the reference.