A three-dimensional memory device includes an alternating stack of insulating layers and electrically conductive layers located over a substrate, memory openings extending through the alternating stack, memory opening fill structures located within a respective one of the memory openings, and a gate dielectric located between the memory opening fill structures and the electrically conductive layers. Each of the memory opening fill structures includes a vertical semiconductor channel, a conductive core electrode, and a memory film located between the vertical semiconductor channel and the conductive core electrode. The memory film contains a layer stack including a first tunneling dielectric contacting the vertical semiconductor channel, a second tunneling dielectric contacting the conductive core electrode, and a charge storage layer located between the first tunneling dielectric and the second tunneling dielectric.