Time-varying latency is estimated based on the round-trip time between the time of sending a message and the time of receiving an acknowledgement of receipt of the message. The round-trip time relative to a transmitter is modeled as a combination of known, or determinable, delays, plus an unknown latency, plus a processing/acknowledgement delay at the receiver. The estimated time-varying latency is further refined to give more weight to estimates based on fewer unknowns or a lesser magnitude of unknowns, and to impose physical constraints, such as assuring that the estimate does not imply an unrealizable event. TCP-specific constraints and assumptions are also applied to further refine the latency estimates.