A method of fabricating a memory device is provided. The method includes forming a first alternating stack of insulator layers and spacer material layers over a semiconductor substrate, etching the first alternating stack to expose a single crystalline semiconductor material, forming a first epitaxial semiconductor pedestal on the single crystalline semiconductor material, such that the first epitaxial semiconductor pedestal is in epitaxial alignment with the single crystalline semiconductor material, forming an array of memory stack structures through the first alternating stack, and forming at least one semiconductor device over the first epitaxial semiconductor pedestal.