A bonded assembly includes a memory die that is bonded to a logic die. The memory die includes a three-dimensional memory array located on a memory-side substrate, memory-side dielectric material layers located on the three-dimensional memory array and embedding memory-side metal interconnect structures and memory-side bonding pads, a backside peripheral circuit located on a backside surface of the memory-side substrate, and backside dielectric material layers located on a backside of the memory-side substrate and embedding backside metal interconnect structures. The logic die includes a logic-side peripheral circuit located on a logic-side substrate, and logic-side dielectric material layers located between the logic-side substrate and the memory die and embedding logic-side metal interconnect structures and logic-side bonding pads that are bonded to a respective one of the memory-side bonding pads.