Disclosed are a method and system for receiving a plurality of potential features to be added to a model having existing features. For each of the potential features, an approximate model is learned by holding values of the existing features in the model constant. The approximate model includes the model having existing features and at least the potential feature. A performance metric is computed for evaluating performance of the approximate model. The performance metric is used to rank the potential feature based on a predetermined criterion.