A history management apparatus includes: a storage unit that stores history information of image processing which contains set-membership information containing pieces of document identification information of an input document and an output document, and an image of at least one of the input document and the output document as history information; and a notification unit that, when history information in which a surveillance target element is contained in an image of an input document or an image of an output document is detected from the storage unit, notifies a notified party of such as the detected history information. When the surveillance target element is not contained in both of the images in the history information, and it is identified that at least one of ancestral documents of the input document has the surveillance target element, the notification unit notifies the notified party of the history information.