A sheet processing apparatus includes a control section configured to control a rotating speed of a longitudinal alignment roller, which is set on a recording medium stacking surface of a processing tray and configured to align a recording medium in a longitudinal direction, such that acceleration and deceleration of the rotating speed is performed at least twice during one longitudinal alignment operation and a paddle including a striking paddle configured to strike the recording medium, a dropping paddle configured to drop the recording medium, a draw-in paddle configured to draw the recording medium in a direction of a stopper, a dropping-paddle supporting member configured to support the dropping paddle from the back in a rotating direction of the paddle, and a draw-in-paddle supporting member configured to support the draw-in paddle from the back in the rotating direction.