An image recognition apparatus includes an image recognition unit, an evaluation value calculation unit, and a motion extraction unit. The image recognition unit uses motion vectors that are generated in the course of coding image data into MPEG format data or in the course of decoding the MPEG coded data by the evaluation value calculation unit and the motion extraction unit as well as two dimensional DCT coefficients and encode information such as picture types and block types for generating the evaluation values that represent feature of the image. The apparatus further includes an update unit for recognizing the object in the image based on the determination rules for a unit of macro block. The apparatus can thus accurately detect the motion of the object based on the evaluation values derived from DCT coefficients even when generation of the motion vectors is difficult.