A non-transitory computer readable storage medium includes instructions to maintain an original training set of labeled documents, where the labeled documents correspond to a variety of topics. A new labeled document corresponding to a new topic is received. The original training set of labeled documents is modulated such that the new labeled document is over-represented with respect to the original training set. This results in a modulated training set. A classifier is trained with the modulated training set to form a trained classifier.