Data collected during runtime and associated with a service level agreement and operating level agreement for a network service are automatically correlated together. A network monitoring system monitors the network service during runtime to determine SLA and OLA violations. An SLA is tested against traffic monitoring data derived from monitoring traffic between a network service system and one or more users of the system. An OLA is tested against application runtime data generated from monitoring a transaction occurring between an application server and a backend or other internal entity of the network service system. The data collected during runtime can be correlated using a hierarchy based on business transactions or in some other manner. Because the traffic monitoring data and application runtime data are organized using a similar data hierarchy, the traffic monitoring data corresponding to an SLA may be correlated to the application runtime data corresponding to the same business transaction and one or more OLAs.