A sheet processing apparatus which is cable of moving a sheet without applying extra load to the movement of the sheet in the case of laterally moving the sheet for sorting after completion of punching of holes in the sheet. A sheet is conveyed toward an abutment member. The sheet comes into abutment with the abutment member, whereby skew of the sheet is corrected. A punching unit punches holes in the sheet in abutment with the abutment member. A shift unit shifts the punched sheet that has been punched by the puncher in a direction intersecting with a conveying direction of the conveying unit. A controller causes the shift unit to start shifting the sheet in response to separate between the punched sheet and the abutment member.