A system and method for discarding or inserting audio frames in a jitter buffer is described. The system and method provides improved audio quality as compared to conventional jitter buffer management systems. In one embodiment, buffer control logic determines whether to discard audio frames to be stored in a jitter buffer or to insert audio frames among audio frames to be output from a jitter buffer based not only on the number of audio frames currently stored in the jitter buffer but also based on the power of the current audio frame to be stored in or output from the jitter buffer. The system and method is generally applicable to any wireless or wired communication system in which audio signals are transmitted between entities operating in different clock domains.