Methods, systems, and computer-readable storage media for tuning behavior of a machine learning (ML) model by providing an alternative loss function used during training of a ML model, the alternative loss function enhancing reliability of the ML model, calibrating the confidence of the ML model after training, and reducing risk in downstream tasks by providing a mapping between the confidence of the ML model to the expected accuracy of the ML model.