A video processing apparatus includes a storage unit, an input unit, and a determining unit. The storage unit stores a plurality of scenes included in a scene group and characteristic amounts which are extracted from a series of plural frames included in the respective scenes in the scene group, in association with each other, defining one scene out of the plurality of scenes included in the scene group as a typical scene and the other scenes as a derived scene, the scene group being a group of a plurality of scenes derived from a common scene. The input unit inputs a characteristic amount which is extracted from each of the frames of a new scene. The determining unit determines whether the new scene represented by the inputted characteristic amount is the derived scene of the scene group or not by verifying the inputted characteristic amount with the characteristic amount of the typical scene of the scene group stored in the storage unit.