A punching device includes a detecting unit, a punching unit, a moving unit, a storage unit, and a controlling unit. The detecting unit detects a lateral edge position of a recording medium to obtain edge position data. When the edge position data does not indicate an error value, the edge position data is stored in the storage unit as reference data. The controlling unit determines, when the edge position data indicates an error value, a movement amount of the punching unit based on reference data previously obtained and stored in the storage unit. The moving unit moves the punching unit by the movement amount in a direction perpendicular to the conveying direction of the recording medium.