A three-dimensional memory device includes a first three-dimensional memory plane including first alternating stacks of first insulating layers and first word lines, and first bit lines electrically connected first vertical semiconductor channels, and a second three-dimensional memory plane including second alternating stacks of second insulating layers and second word lines and second bit lines electrically connected to second vertical channels. An inter-array backside trench laterally extend between the first three-dimensional memory plane and the second three-dimensional memory plane, and filled with an inter-array backside insulating material portion that provides electrical isolation between the three-dimensional memory planes.