Aspects of the disclosure include determining a data rate and a latency of one or more packets associated with a buffer, and adapting a size of the buffer based on the data rate or latency. The size of the buffer may also be adjusted differently based on its current size. For example, the size of the buffer may be increased by a greater amount if its current size is relatively small, or it may be decreased by a greater amount if its current size is relatively large. To determine how to adjust the buffer, the data rate may be compared to a data rate threshold and the latency may be compared to a latency threshold.