A method for optimizing a jitter buffer includes receiving a first request to establish a video conference communication session with a remote endpoint. The method also includes sending a second request to a first network location for latency information for packet communications sent from a second network location. The method further includes receiving from the first network location the latency information for packet communications sent from the second network location. Additionally, the method includes setting a jitter buffer for the video conference communication session based on the latency information for packet communications sent from the second network location.