A three-dimensional memory device includes an alternating stack of electrically conductive layers and insulating layers located over a substrate, an array of memory stack structures. A source conductive line structure is provided between the substrate and the alternating stack. The source conductive line structure includes a plurality of parallel conductive rail structures extending along a same horizontal direction and adjoined to a common conductive straddling structure. Each memory stack structure straddles a vertical interface between a conductive rail structure and a support matrix. A semiconductor channel in each memory stack structure contacts a respective conductive rail structure and the support matrix.