A semiconductor structure includes a peripheral circuit, a first three-dimensional memory array overlying the peripheral circuit and including a first alternating stack of first insulating layers and first electrically conductive layers containing first word lines and first select lines, and first memory stack structures vertically extending through the first alternating stack, and a second three-dimensional memory array overlying the first three-dimensional memory array and including a second alternating stack of second insulating layers and second electrically conductive layers containing second word lines and second select lines, and second memory stack structures vertically extending through the second alternating stack. The peripheral circuit includes a first word line driver circuit having first word line driver output nodes electrically connected to at least some of the first word lines and at least some of the second word lines, and each first word line is electrically connected to a respective second word line.