A system and method for identifying UDP packets on an IP network as candidates for characterization as packets of a RTP stream. UDP packets are identified at a point on the IP network, and for each identified UDP packet 1) it is determined if a version number in a RTP header field in the payload of the UDP packet equals a predetermined value, 2) determined if a packet length associated with the UDP packet is within a predetermined range, 3) determined if a payload type RTP header field within the payload of the UDP packet is within a predetermined range. If all the criteria are satisfied, then the identified UDP packet is characterized as a candidate RTP packet. It is then determined if the candidate RTP packet belongs to a previously detected RTP stream, or to a newly encountered RTP stream. A stream data store is then updated using the information in the RTP header of the candidate RTP packet.