A semiconductor device of which manufacturing steps can be simplified by doping impurities at a time, and a manufacturing method thereof. The manufacturing method of the semiconductor device comprises the steps of: forming first and second semiconductor layers over a substrate, forming a first insulating film over the first and second semiconductor layers, forming first and second conductive films thereover, forming a first gate electrode having a stacked layer of the first and second conductive films, in which a portion of the first conductive film is exposed from the second conductive film, over the first semiconductor layer with the first insulating film interposed therebetween, forming a second insulating film over the first insulating film, forming third and fourth conductive films thereover, and forming a second gate electrode having a stacked layer of the third and fourth conductive films, in which a portion of the third conductive film is exposed from the fourth conductive film, over the second semiconductor layer with the first and second insulating films interposed therebetween.