Systems and methods are described for testing one or more machine learning algorithms in parallel with an existing machine learning algorithm implemented within a data processing pipeline. Each machine learning algorithm can train a machine learning model that receives a live stream of raw machine data. The output of the machine learning model trained by the existing machine learning algorithm may be written to an external storage system, but the output of the machine learning model(s) trained by the test machine learning algorithm(s) may not be written to an external storage system. After some time, performance of the test machine learning algorithm(s) and the existing machine learning algorithm is evaluated. If the test machine learning algorithm performs better than the existing machine learning algorithm, then the machine learning algorithms can be swapped without any downtime and without needed to re-train a machine learning model using previously seen raw machine data.