There is provided a contents processing apparatus including: a contents data acquisition unit that acquires data of contents; a section detection unit that detects from the acquired data of the contents a cross-fade section that includes a plurality of frames constituting a portion where a final picture of a predetermined scene is gradually changed and replaced with a picture of the next scene; a scene specifying unit that specifies a retrospective scene, where a plurality of scenes are changed in a predetermined interval, based on the detected cross-fade section; a similar scene searching unit that searches for scenes similar to the plurality of the scenes included in the specified retrospective scene from data of different contents acquired by the contents data acquisition unit; and a digest generation unit that generates a digest of the different contents by combining the searched scenes.