Classification algorithm optimization is described. In an example, a classification algorithm is optimized by calculating an evaluation sequence for a set of weighted feature functions that orders the feature functions in accordance with a measure of influence on the classification algorithm. Classification thresholds are determined for each step of the evaluation sequence, which indicate whether a classification decision can be made early and the classification algorithm terminated without evaluating further feature functions. In another example, a classifier applies the weighted feature functions to previously unseen data in the order of the evaluation sequence and determines a cumulative value at each step. The cumulative value is compared to the classification thresholds at each step to determine whether a classification decision can be made early without evaluating further feature functions.