A video processing apparatus and methodology are implemented as a combination of a processor and a video decoding hardware block to decode video data by performing piecewise processing of overlap smoothing and in-loop deblocking in a macroblock-based fashion. With this approach, a smaller on-board memory may be used for the in-loop filtering operations of the video decoding hardware block. By pipelining the piecewise processing operations, latency in the filtering operations is hidden and the filtering output is smoothed, thereby avoiding the need for bursts of fetching and storing of blocks.