A system and method for performing multi-image training for pattern recognition and registration is provided. A machine vision system first obtains N training images of the scene. Each of the N images is used as a baseline image and the N−1 images are registered to the baseline. Features that represent a set of corresponding image features are added to the model. The feature to be added to the model may comprise an average of the features from each of the images in which the feature appears. The process continues until every feature that meets a threshold requirement is accounted for. The model that results from the present invention represents those stable features that are found in at least the threshold number of the N training images. The model may then be used to train an alignment/inspection tool with the set of features.