Computer-based systems and methods add extra terms to the objective function of machine learning systems (e.g., neural networks) in an ensemble for selected items of training data. This selective training is designed to penalize and decrease any tendency for two or more members of the ensemble to make the same mistake on any item of training data, which should result in improved performance of the ensemble in operation.