In one embodiment, a system for enhancing predictive modeling includes an interface operable to receive a first dataset. The system may also include a processor communicatively coupled to the interface that is operable to generate a holdout dataset based on the first dataset. The processor may also train each of a plurality of boosting models in parallel using the first dataset, wherein for each of a number of iterations, training comprises: building a one-level binary decision tree to train a split-node variable; calculating an impurity of the split-node variable; and calculating an optimal split node, wherein the optimal split node is the split-node variable with a lowest impurity between the plurality of boosting models. The system may then determine a final model based on one of the plurality of boosting models that provides the lowest error rate when applied to the holdout dataset.