A semiconductor device structure includes a first channel structure and a second channel structure over a substrate. The semiconductor device structure also includes a first gate stack over the first channel structure and a second gate stack over the second channel structure. The first gate stack and the second gate stack have a first work function layer and a second work function layer, respectively. The first work function layer and the second work function layer are made of a same material. The second gate stack has a first protruding portion and a second protruding portion, and each of the first protruding portion and the second protruding portion extends upwards and extend away from the second channel structure. The first protruding portion and the second protruding portion are spaced apart from each other, and half of the first gate stack is wider than the first protruding portion.