A sequence number is used to indicate where a payload of a voice data packet should fit in a data stream and a technique is described for retrieving the sequence number for redundant payloads. A receiver maintains a history of previously received timestamps and sequence numbers for previous payloads. A received packet is unpacked to obtain a primary payload and its associated sequence number and timestamp, and a redundant payload and its associated timestamp offset. The primary payload sequence number and timestamp are stored in the history. A time-span of the data stream covered by the packet is found using the timestamp offset, and a portion of the history selected based on the time-span. A timestamp parameter for the redundant payload is calculated using the primary payload timestamp and the timestamp offset, and is compared to timestamps in the selected portion of the history to derive the redundant payload sequence number. The history is updated to include the timestamp parameter and sequence number of the redundant payload.