The total chip area for a three-dimensional memory device can be reduced employing a design layout in which the word line decoder circuitry is formed underneath an array of memory stack structures. The interconnection between the word lines and the word line decoder circuitry can be provided by forming discrete word line contact via structures. The discrete word line contact via structures can be formed by employing multiple sets of etch masks with overlapping opening areas and employed to etch a different number of pairs of insulating layers and electrically conductive layers, thereby obviating the need to form staircase regions having stepped surfaces. Sets of at least one conductive interconnection structure can be employed to provide vertical electrical connection to the word line decoder circuitry. Bit line drivers can also be formed underneath the array of memory stack structures to provide greater areal efficiency.