A non-transitory computer-readable medium including instructions, which when executed by one or more processors of a computing system, causes the computing system to: access a machine learning model m, an input data point P to m, P including one or more features, and a prediction m(P) of m for P; create a set of perturbed input data points Pk from P by selecting a new value for at least one feature of P for each perturbed input data point; obtain a prediction m(Pk) for each of the perturbed input data points; analyze the predictions m(Pk) for the perturbed input data points to determine which features are most influential to the prediction; and output the analysis results to a user.