In one embodiment, a network assurance service applies labels to feature vectors of network characteristics associated with a plurality of wireless access points in the network. An applied label for a feature vector indicates whether the access point associated with the feature vector experienced a threshold number of onboarding delays within a given time window. The service, based on the feature vectors and labels, trains a plurality of machine learning-based classifiers to predict onboarding delays, and uses one or more of the trained plurality of classifiers to predict onboarding delays for a particular access point. The service calculates one or more classifier performance metrics for the one or more classifiers based on the predicted onboarding delays for the particular access point. The service selects a particular one of the classifiers to monitor the network characteristics associated with the particular access point, based on the one or more classifier performance metrics.