A method may include applying to at least a portion of historical runtimes associated with each of a plurality of tests included in a test suite, a machine learning model trained to identify one or more outlying runtime values. The portion of historical runtimes may include an n quantity of the most recent historical runtimes. An expected runtime for each test may be determined based on the portion of historical runtimes excluding the outlying runtime values. A schedule for executing each test in the test suite may be determined based on the expected runtime of each test. The test suite may be executed in accordance with the schedule. Moreover, the executing of the test suite may be monitored based on the expected runtime of each test. Related systems and computer program products are also provided.