A planar material layer stack including a lower etch stop dielectric layer, a sacrificial semiconductor layer, and an upper etch stop dielectric layer is formed over a source semiconductor layer on a substrate. An alternating stack of insulating layers and spacer material layers is formed. The spacer material layers are formed as, or are subsequently replaced with, electrically conductive layers. An array of memory stack structures is formed. A source cavity is formed by removing the sacrificial semiconductor layer and portions of the memory films. Source strap structures are formed by a selective semiconductor deposition process on the vertical semiconductor channels and the source semiconductor layer. A dielectric fill material layer fills a remaining volume of the source cavity.