Systems and techniques for hierarchical tournament-based machine learning predictions are described herein. A machine learning selection model may be trained with training data. A configuration may be received that includes the metric and a target prediction. A set of evaluation component combinations may be selected using the machine learning selection model. Each evaluation component combination of the set of evaluation component combinations may include an algorithm, a hierarchical learning model corresponding to a level of a hierarchy, and a prediction model for the target prediction. The set of evaluation component combinations may be transmitted to a cluster of computing nodes. Output results may be received for the set of evaluation component combinations. The output results may be evaluated using the metric to determine a winning evaluation component combination. The winning evaluation component combination may be stored in storage for use in calculating future predictions for the target prediction.