Semiconductor device and the manufacturing method thereof are disclosed. An exemplary semiconductor device includes a substrate including top portions isolated by an isolation structure, first semiconductor layers over a first top portion of the substrate in a first region, and a first gate structure wrapping each of the first semiconductor layers and covering a top surface and sidewalls of the first top portion of the substrate extending above the isolation structure. The first semiconductor layers are stacked up and separated from each other, and each first semiconductor layer has a first width. A bottom surface of the first gate structure is below the top surface of the substrate for a first depth which is at least half of the first width.