Embodiments are directed to monitoring network traffic associated with networks to provide metrics. A monitoring engine may determine an anomaly based on the metrics exceeding threshold values. An inference engine may be instantiated to provide an anomaly profile based on portions of the network traffic that are associated with the anomaly. The inference engine may provide an investigation profile based on the anomaly profile such that the investigation profile includes information associated with investigation activities associated with an investigation of the anomaly. The inference engine may monitor the investigation of the anomaly based on other portions of the network traffic such that the other portions of the network traffic are associated with monitoring an occurrence of the investigation activities. The inference engine may modify a performance score associated with the investigation profile based on the occurrence of the investigation activities and a completion status of the investigation.