An image summarization device includes a first image summarization section that performs a first image summarization process based on a similarity between a plurality of images to acquire a first summary image sequence, a second image summarization section that performs a second image summarization process based on a target object/scene recognition process on each image among the plurality of images to acquire a second summary image sequence, and an integration processing section that performs an integration process on the first summary image sequence and the second summary image sequence, or performs an integration process on the first image summarization process and the second image summarization process to acquire an output summary image sequence.