There is provided a moving image processing apparatus, including a similarity determination unit configured to determine a degree of similarity between a subsequent cut and first and second cut groups based on feature amounts generated from extraction images of the first cut group included in a moving image and feature amounts generated from extraction images of the second cut group included in the moving image, a cut grouping unit configured to group the subsequent cut into a similar cut group similar to the subsequent cut, which is one of the first and second cut groups, when the subsequent cut is similar to the first or second cut group, and group the subsequent cut into a third cut group when the subsequent cut is not similar to either of the first and second cut groups, a feature amount generation unit configured to compare extraction images extracted from the third cut group with the extraction images extracted from the first and second cut groups when the subsequent cut is not similar to either of the first and second cut groups, and generate feature amounts of the third cut group, and an image extraction unit configured to preferentially extract an image with a later time code of the moving image from images included in each cut group, thereby obtaining extraction images of each cut group.