A method is disclosed for the decoding and encoding of a block-based video bit-stream such as MPEG2, H.264-AVC, VC1, or VP6 using a system containing one or more high speed sequential processors, a homogenous array of software configurable general purpose parallel processors, and a high speed memory system to transfer data between processors or processor sets. This disclosure includes a method for load balancing between the two sets of processors.