A method for fabricating semiconductor device comprising: providing a substrate having a gate structure thereon and a first interlayer dielectric (ILD) layer surrounding the gate structure; removing part of the gate structure; forming a first mask layer on the first ILD layer and the gate structure; removing the first mask layer on the first ILD layer and part of the first mask layer on the gate structure for forming a first hard mask on the gate structure; forming a second mask layer on the first ILD layer, the first hard mask, and the gate structure; and planarizing the second mask layer to form a second hard mask on the gate structure, in which the top surfaces of the first hard mask, the second hard mask, and the first ILD layer are coplanar.