A system and processing methods for refining a convolutional neural network (CNN) to capture characterizing features of different classes are disclosed. In some embodiments, the system is programmed to start with the filters in one of the last few convolutional layers of the initial CNN, which often correspond to more class-specific features, rank them to hone in on more relevant filters, and update the initial CNN by turning off the less relevant filters in that one convolutional layer. The result is often a more generalized CNN that is rid of certain filters that do not help characterize the classes.