According to an embodiment, a method of manufacturing a semiconductor device includes forming a first opening that extends from a second surface of a semiconductor substrate opposite to a first surface toward the first surface and extending to a first insulating layer in the semiconductor substrate, performing a first annealing process in a first gas atmosphere that contains hydrogen after formation of the first opening, forming a second insulating layer on a side wall of the semiconductor substrate in the first opening, performing a second annealing process after formation of the second insulating layer, forming a second opening that extends to the conductive layer in the first insulating layer through the first opening, and forming a via that is connected to the conductive layer in the first and second openings.