According to one embodiment, a structure evaluation system includes an impact imparting unit, a sensor, and a structure evaluation device. The impact imparting unit applies impacts to a structure. The impact imparting unit applies the impacts at a frequency equal to or less than a frequency determined in accordance with an intensity at which the impacts are imparted. The sensor detects elastic waves. The structure evaluation device evaluates a deterioration state of the structure on the basis of the detected elastic waves.