According to one embodiment, a road surface management system inputs image data of road surface and capturing information of each of the image data and registers them. The image data are collected by repeatedly capturing a same route at predetermined distance. The capturing information includes information of a location and time at time of capturing. The system selects old and new images at a same spot from the registered image data, performs an association process, extracts an original image which is an old image at an arbitrary spot and a target image which is a new image associated with the original image unit based on the result of the association process, and outputs the original image and the target image.