A method of fabricating a semiconductor device includes forming an inter-metal dielectric layer including a first trench and a second trench which are spaced from each other on a substrate, forming a first dielectric layer along the sides and bottom of the first trench, forming a second dielectric layer along the sides and bottom of the second trench, forming first and second lower conductive layers on the first and second dielectric layers, respectively, forming first and second capping layers on the first and second lower conductive layer, respectively, performing a heat treatment after the first and second capping layers have been formed, removing the first and second capping layers and the first and second lower conductive layers after performing the heat treatment, and forming first and second metal gate structures on the first and second dielectric layers, respectively.