In realizing a user recognition system and a method thereof, a user feature vector is extracted from an input facial image, and at least one cluster is generated and a user feature template is enrolled. The cluster includes the feature vector as a member of it. When a user facial image is inputted, a user feature vector is extracted from the image, and a similarity between the feature vector and the user feature template is calculated. When the similarity is greater than a predetermined threshold value, a user of the user feature template is recognized as the user of the input image.