The present disclosure provides systems and methods that learn a loss function that, when (approximately) minimized over the training data, produces a model that performs well on test data according to some error metric. The error metric need not be differentiable and may be only loosely related to the loss function. In particular, the present disclosure presents a convex-programming-based algorithm that takes as input observed data from training a small number of models and produces as output a loss function. This algorithm can be used to tune loss function hyperparameters and/or to adjust the loss function on-the-fly during training.