A sheet processing apparatus includes a puncher, a first sensor positioned upstream of the puncher in the conveyance direction, a second sensor positioned upstream of the first detection position in the conveyance direction, a drive source configured to drive the puncher, a controller configured to control the drive source. The controller executes a control mode including a first process of controlling a rotation speed of the puncher on a basis of a detection result of the second sensor and a second process of controlling the rotation speed of the puncher on a basis of a detection result of the first sensor. In the control mode, the controller does not stop rotation of the puncher in a period between the punching process on the preceding sheet and a punching process on the succeeding sheet.