A method of fabricating semiconductor device includes forming a plurality of gate structures on a semiconductor substrate. A first inter layer dielectric layer is deposited on the gate structures. A first contact plug is formed in the first inter layer dielectric layer in between every two immediately adjacent gate structures. An etch stop layer is deposited on the first inter layer dielectric layer. A second inter layer dielectric layer is deposited on the first inter layer dielectric layer. A second contact plug is formed in the second inter layer dielectric layer aligning with the first contact plug. A metal layer is deposited overlying the second inter layer dielectric layer and the second contact plug.