A map labeling system trains a machine-learned model using a set of training data and generates a set of test predictions for a set of test properties by applying the machine-learned model to a set of testing data. Each prediction in the set of test predictions comprises a confidence score representing the machine-learned model's confidence in the prediction. The map labeling system determines a correctness of each prediction in the set of predictions and determines a relationship between the confidence scores and the correctness of the test predictions. The map labeling system establishes a confidence threshold for the machine-learned model based on the determined relationship and labels a production property by applying the machine-learned model to production data.