A method of extracting information parts, and a recorded medium recording the method, for retrieving image sequence data are disclosed. According to an embodiment of the present invention, image frames included in an image sequence are converted to frames, in a quantity of n, including only the object that is separated from a background, and the corresponding shape descriptors, in a quantity of n, are extracted. The shape descriptors, in a quantity of n, are aligned according to a temporal order, to generate a shape sequence, which is frequency-converted along the time axis to obtain conversion coefficients having frequency information. Using coefficients in a low frequency area among the obtained conversion coefficients, a shape sequence descriptor is extracted. With this invention, video data expressing the motion of an object can be captured as an image sequence, and the similarity between video data can be distinguished by extracting information parts from each image sequence.