A processing element/unit can include a plurality of networks, a plurality of cores, crossbar interconnects, a plurality of memory controllers and local memory on an integrated circuit (IC) chip. The plurality of cores can be coupled together by the plurality of networks on chip. The crossbar interconnects can couple the networks of cores to the plurality of memory controllers. The plurality of memory controllers can be configured to access data stored in off-chip memory. The local memory can be configured to cache portions of the accessed data. The local memory can be directly accessible by the network of processing cores, or can be distributed across the plurality of memory controllers. The memory controllers can be narrow channel (NC) memory controllers having widths of 4, 8, 12, 16 or a multiple of 4 bits.