A three-dimensional semiconductor device includes bit lines formed in the lower-interconnect-level dielectric material layers located over a substrate, bit-line-connection via structures contacting a respective one of the bit lines, pillar-shaped drain regions contacting a respective one of the bit-line-connection via structures, an alternating stack of insulating layers and electrically conductive layers located over the pillar-shaped drain regions, and memory stack structures extending through the alternating stack. A source layer overlies the alternating stack, and is electrically connected to an upper end of each vertical semiconductor channel within a subset of the vertical semiconductor channels. Vertical bit line interconnections structures extending through the levels of the alternating stack may be eliminated by forming the bit lines underneath the alternating stack, and the footprint of the layout of the three-dimensional memory device may be reduced.