Disclosed are various systems and methods for controlling a voice activity detector. In one example approach, a method is provided in which a voice over internet protocol (VOIP) session is initiated in a first gateway with a second gateway. A plurality of jitter adaptation packets are transmitted from the first gateway to the second gateway. Also, the voice activity detector is disabled in the first gateway during the transmission of the jitter adaptation packets. The voice activity detector is enabled in the first gateway after the transmission of the jitter adaptation packets.