A method of fabricating a gate all around semiconductor device is provided. The method includes: providing a semiconductor substrate having a plurality of active fins extending in a first direction in a first region and a second region next to the first region, a plurality of gate all around channels stacked above each of the plurality of active fins, and a plurality of gate openings extending in a second direction across the first and second regions and crossing the plurality of active fins, in which the plurality of gate openings include cave-like gate spaces between each of the plurality of active fins and one adjacent gate all around channel and between two adjacent gate all around channels, forming a dielectric layer in the first and second regions on bottom and sidewalls of each of the plurality of gate openings, and on and surrounding each of the plurality of gate all around channels and filling a first portion of each of the cave-like gate spaces, forming first work function metal in the first and second regions on the dielectric layer with the first work function metal filling a second portion of each of the cave-like gate spaces, forming first carbon-based mask in the first and second regions by a chemical vapor deposition (CVD) process to fill the plurality of gate openings to a height at least covering all the plurality of gate all around channels, forming second carbon-based mask in the first and second regions on top of the first carbon-based mask to a height above the plurality of gate openings, removing the first and second carbon-based masks in the second region, removing the first work function metal in the second region through etching using remaining first and second carbon-based masks in the first region as an etching mask, removing the remaining first and second carbon-based masks in the first region, and forming second work function metal on the dielectric layer in the second region, and on the first work function metal in the first region.