The present invention relates to a receiver system in a communication system supporting packet-based communicatior (e.g., an IP-network), including a receiver, speech decoder ( 40 ) and a jitter buffer ( 20 ) for handling delay variations in the reception of a speech signal consisting of packets containing frames with encoded speech. A jitter buffer controller ( 50 ) is provided for keeping information about the functional size of the jitter buffer ( 20 ) and for providing the speech decoder ( 40 ) with control information, such that the speech decoder ( 40 ), based on that information. provides a dynamic adaptation of the size of the jitter buffer ( 20 ) using the received encoded, packetized speech signal. The invention also relates to a method of adapting the functional size of the jitter buffer of a receiver system.