When making a decision about who should have control of a voice channel in a teleconference, one exemplary aspect is directed toward a voice switch mechanism that determines and takes into account a latency (such as a round trip latency) of each participant's path to the switch. The switch uses this information to ensure that short-latency paths do not have an unfair advantage over long-latency paths when individuals competitively seek control of the voice channel. Illustratively, if an individual is participating in the path that has a round trip latency of 300 ms or greater than that of other participants, the voice switch creates a level playing field even if it detects voice energy from a short-latency user first by granting control of the channel to the long-latency user if voice energy is detected from that user within 300 ms of the short-latency detection.