Systems, methods, and other embodiments associated with applying machine learning to below-the-line threshold tuning are described. In one embodiment, a method includes selecting a set of sampled events and labeling each event in the set of sampled events as either suspicious or not suspicious. Then, a machine learning model to calculate for a given event a probability that the given event is suspicious is built based on the set of sampled events. The machine learning model is trained, and its calibration validated. Based on probabilities calculated by the machine learning model, a scenario and segment combination to be tuned is determined. A tuned threshold value is generated, and an alerting engine is adjusted with the tuned parameter to reduce errors by the alerting engine in classifying events as not suspicious.