A three-dimensional memory device includes an alternating stack of source layers and drain layers located over a substrate, gate electrodes vertically extending through each of the source layers and the drain layers of the alternating stack, memory films laterally surrounding a respective one of the gate electrodes, and semiconductor channels laterally surrounding a respective one of the memory films and connected to a respective vertically neighboring pair of a source layer and a drain layer. An array of memory openings can vertically extend through the alternating stack, and each of the gate electrodes can be located within a respective one of the memory openings.