A method of forming a semiconductor device includes forming a sacrificial gate structure over a substrate, depositing a spacer layer on the sacrificial gate structure in a conformal manner, performing a multi-step oxidation process to the spacer layer, etching the spacer layer to form gate sidewall spacers on opposite sidewalls of the sacrificial gate structure, removing the sacrificial gate structure to form a trench between the gate sidewalls spacers, and forming a metal gate structure in the trench.