A self-adaptive jitter buffer adjustment method for packet-switched network is presented in this invention, which involves both positive adjustment to jitter buffer and negative adjustment to jitter buffer. The positive adjustment part performs a positive adjustment to the jitter buffer according to an indication of a sequence number of the packet delaying, to absorb larger network jitters by dynamically detecting sequence numbers of the packets in the jitter buffer. The negative adjustment part performs a negative adjustment to the jitter buffer when the filling level descends, to match smaller network jitters by periodically detecting a filling level of the jitter buffer. This invention realizes the dynamic tracking of network jitter, so that a self-adaptive adjustment to the jitter buffer working parameters can be achieved. Comparing with traditional methods for self-adaptive buffer adjustment, the present method reduces the complicacy and computational costs of the self-adaptive jitter buffer adjustment algorithm. At the same time, the impact of self-adaptive jitter buffer adjustment on the performance of the entire system is greatly reduced.