Methods for one-class learning using support vector machines from a plurality of data batches are provided. A first support vector machine is learned from the plurality of data batches by a processor. A new data batch is received by the processor and is classified by the first support vector machine. If a non-zero loss classification occurs a new support vector machine is trained using the first support vector machine and the new data batch only. Data batches can be discarded if they are represented by the current support vector machine or after being used for training an updated support vector machine. Weighing factors applied to update the first support vector machine depend upon a parameter which is optimized iteratively. Support vectors do not need to be recalculated. A classifier is learned in a number of stages equal to the number of data batches processed on-line.