A video encoding device includes a local decode generation unit for generating a reference image based on a result of encoding of a divided image, a compression unit for compressing the reference image to generate a compressed data, a reference image storage determination unit for determining whether to store the compressed data in a memory, and an inter-prediction unit for performing motion vector search for inter-coding based on a reference image stored in the memory. The reference image storage determination unit sets an allowable data amount used for storing the reference image for each determined area of the moving image data, and determines whether or not to store the compressed data obtained by compressing the reference image in the memory based on the allowable data amount. Inter-prediction unit sets the reference image corresponding to the compressed data stored in the memory as the search range of motion vector search.