A three-dimensional mounting semiconductor device includes a layer structure including a plurality of first substrates with a trench-shaped concavity formed in and a plurality of second substrates with semiconductor elements formed in, which are alternately stacked, wherein an unevenness defined by a size difference between the first substrate and the second substrate is formed on a side surface, and a first through-hole are defined by an inside surface of the trench-shaped concavity and a surface of the second substrate, and a third substrate jointed to the side surface of the layer structure and having an unevenness formed on a surface jointed to the layer structure which are engaged with the unevenness formed on the side surface of the layer structure.