Non-negative matrix factorization, NMF, is combined with identification of a maximum margin classifier by minimizing a cost function that contains a generative component and the discriminative component. The relative weighting between the generative component and the discriminative component are adjusting during subsequent iterations such that initially, when confidence is low, the generative model is favored. But as the iterations proceed, confidence increases and the weight of the discriminative component is steadily increased until it is of equal weight as the generative model. Preferably, the cost function to be minimized is: min F , G ≥ 0 ⁢  X - FG  2 + γ ⁡ (  w  2 + C ⁢ ∑ i = 1 n ⁢ L ⁡ ( y i , w · g i + b ) ) .