The described technology is generally directed towards predicting the survival of a storage device (e.g., a hard disk drive or a solid state drive) to a specified time point, expressed as a confidence score, via a prediction uncertainty quantifier framework with a machine learning classifier. The confidence score corresponds to the likelihood of a storage device surviving until a specified time point (e.g., for n hours). In one implementation, a conformal prediction framework provides the confidence score for a storage device, based on survival rate data predicted using recent telemetry data collected for that storage device by an online semi-parametric Mondrian survival forest classifier. Updated confidence scores based on updated telemetry data can be obtained at various evaluation stages to reevaluate whether to take remedial action with respect to a storage device (e.g., replace the storage device). Multiple storage devices can be ranked by their respective associated confidence scores.