This present invention brings to the multiprocessor what vectorization brought to the single processor. It provides similar tools to speed communication that have traditionally been used to speed computation; namely, the capability to program optimal communication algorithms on an architecture that can replicate their performance in terms of wall clock time. In addition to the usual complement of logic and arithmetic units, each processor contains a programmable communication unit that orchestrates traffic between the network and registers that communicate directly with comparable registers in neighboring processors. Communication tasks are performed out of these registers like computational tasks on a vector uniprocessor. The architecture is balanced and the hardware/software combination is scalable to any number of processors.