A paper sheet processing apparatus comprises a taking-in unit 10 configured to take therein paper sheets, a transport unit 70 configured to transport the paper sheets, a recognition unit 20 configured to recognize the paper sheets, a plurality of stacking units 60a to 60d configured to stack the paper sheets, the paper sheets stacked in the stacking units 60a-60d being able to be removed from the stacking units 60a-60d, and a detection sensor 25a to 25d configured to detect whether the paper sheets have been removed from the stacking units 60a-60d. When a control unit 50 judges that the paper sheets have been removed from the stacking units 60a to 60d based on information from the detection sensor 25a to 25d, the control unit 50 outputs removal information on the removed paper sheets or the stacking units 60a to 60d from which the paper sheets have been removed.