In one embodiment, a device in a network receives feedback regarding an anomaly reporting mechanism used by the device to report network anomalies detected by a plurality of distributed learning agents to a user interface. The device determines an anomaly assessment rate at which a user of the user interface is expected to assess reported anomalies based in part on the feedback. The device receives an anomaly notification regarding a particular anomaly detected by a particular one of the distributed learning agents. The device reports, via the anomaly reporting mechanism, the particular anomaly to the user interface based on the determined anomaly assessment rate.