The stacked semiconductor device includes a semiconductor substrate, a multi-layered insulation layer pattern having at least two insulation layer patterns and an opening, an active layer pattern formed on each of the insulation layer patterns, a first plug including single crystalline silicon-germanium, a second plug including single crystalline silicon, and a wiring electrically connected to the first plug and sufficiently filling up the opening. The insulation layer patterns are vertically stacked on the semiconductor substrate and the opening exposes an upper face of the semiconductor substrate. A side portion of the active layer pattern is exposed by the opening. The first plug is formed on the upper face of the semiconductor substrate to partially fill the opening. The second plug is partially formed on the first plug, and has substantially the same interface as that of the first plug.