A NOR-type three-dimensional memory device includes a vertically alternating stack of insulating layers and electrically conductive layers located over a substrate, and laterally alternating sequences of respective active region pillars and respective memory stack structures. Each laterally alternating sequence is electrically isolated from the electrically conductive layers by a respective blocking dielectric layer at each level of the electrically conductive layers. Each memory stack structures include a memory film and a semiconductor channel material portion that vertically extend through the vertically alternating stack. The active region pillars include an alternating sequence of source pillar and drain pillars.