The present invention is directed to a pattern recognition system in which new reference data to be added is efficiently learned. In the pattern recognition system, there is performed the calculation of distances equivalent to similarities between input data of a pattern search target and a plurality of reference data, and based on input data of a fixed number of times corresponding to the reference data set as a recognized winner, a gravity center thereof is calculated to optimize the reference data. Furthermore, a threshold value is changed to enlarge/reduce recognition areas, whereby erroneous recognition is prevented and a recognition rate is improved.