A stack including an alternating plurality of first material layers and second material layers is provided. A memory opening is formed and at least a contiguous semiconductor material portion including a semiconductor channel is formed therein. The contiguous semiconductor material portion includes an amorphous or polycrystalline semiconductor material. A metallic material portion is provided at a bottom surface of the semiconductor channel, at a top surface of the semiconductor channel, or on portions of an outer sidewall surface of the semiconductor channel. An anneal is performed to induce diffusion of a metal from the metallic material portion through the semiconductor channel, thereby inducing conversion of the amorphous or polycrystalline semiconductor material into a crystalline semiconductor material. The crystalline semiconductor material has a relatively large grain size due to the catalytic crystallization process, and can provide enhanced charge carrier mobility.