A method of object recognition trains a convolutional neural network (CNN) with a set of training images, then classifies an image of an object using the trained CNN. A first layer of the CNN is trained by generating a set of first convolutional filters from eigenvectors produced from linear principal component analysis of patches of the training images. The training of each of multiple hidden layers CNN includes generating a set of convolutional filters from a selected subset of eigenvectors produced from linear principal component analysis of patches of an affinity matrix constructed using a set of prior convolutional filters from a prior layer of the CNN, where the affinity matrix represents correlations of feature vectors associated with the prior layer. The last layer of the CNN is trained with a regular classifier by error back-propagation using the training images and labels associated with the training images.