A paper sheet processing apparatus that performs a recognizing process of a paper sheet includes: a paper sheet recognition sensor (11) that reads an image of a paper sheet; a number-attribute information memory (131) that stores number attribute information including number recognition conditions for recognizing a number given to a paper sheet; and an image processing unit (134) and a character recognizing unit (135) that recognize a number of a paper sheet from an image read by the paper sheet recognition sensor (11), based on number recognition conditions stored in the number-attribute information memory (131).