A method and apparatus for in-loop processing of reconstructed video are disclosed. The reconstructed video data is partitioned into image units. The image unit corresponds to a Largest Coding Unit (LCU), a slice, or a tile. Horizontal deblocking is applied to the reconstructed video data across vertical image-unit boundaries and then vertical deblocking operates on n pixels of horizontal-deblocked video data on each side of horizontal image-unit boundaries. According to various embodiments of the present invention, the horizontal deblocking is applied to at least one of said n reconstructed lines to generate at least a horizontal-deblocked line for line pn−1. Sample adaptive offset (SAO) processing is then applied to at least a horizontal-deblocked line for line pn based on the horizontal-deblocked line for line pn−1. The above steps are performed before a neighboring image unit below the current image unit arrives.