The described technology is generally directed towards latency diagnostics for multiparty systems, such as a system comprising a communication network, a content delivery network (CDN), and one or more internet service providers (ISPs). A latency analyzer component can process latency data in response to a latency diagnostic trigger, such as an alert from a video quality monitoring system. The latency analyzer can determine whether latency is attributable to the communication network. If not, the latency analyzer can determine whether the latency is attributable to the CDN. If the latency is not attributable to the communication network or the CDN, the latency analyzer can determine that the latency is attributable to the ISP, and the latency analyzer can identify the ISP and generate appropriate reports and notifications.