A sheet processing apparatus includes a conveying motor for conveying a sheet along a conveying path, a skew detecting unit configured to detect a quantity of skew of the conveyed sheet, a hole punching section arranged downstream from the skew detecting unit, an attitude control unit configured to carry out skew correction by changing tilt angle of the hole punching section in accordance with the quantity of skew, a detecting unit configured to detect the forward edge and the rear edge of the sheet conveyed into the hole punching section, and a control unit configured to control the conveying motor to control the conveying speed of the sheet. During a period from when the detecting unit detects the forward edge of the sheet until the detecting unit detects the rear edge, the sheet is decelerated from a first conveying speed to a second conveying speed. After the detecting unit detects the rear edge of the sheet, the conveying of the sheet is stopped and punching processing to the sheet is executed when the conveying is stopped.