Gate structures of CMOS device and the method for manufacturing the same are provided. A substrate having an NMOS region, a PMOS region, and a work function modulation layer disposed on the NMOS region and the PMOS region is provided. A nitrogen doping process is performed to dope nitrogen into a portion of the work function modulation layer disposed on the PMOS region so as to form an N-rich work function modulation layer disposed on the PMOS region. A nonmetallic conductive layer is formed blanketly covering the work function modulation layer and the N-rich work function modulation layer. A portion of the nonmetallic conductive layer, the work function modulation layer, and the N-rich work function modulation layer is removed to form a first gate in the NMOS region and a second gate in the PMOS region.