A semiconductor device structure, along with methods of forming such, are described. The semiconductor device structure includes a first P-type metal oxide semiconductor field effect transistor (p-MOSFET) having a first fin extending along a first direction and comprising a first semiconductor layer, wherein the first fin comprises a first recess formed in a top of the first fin, the first recess having a bottom surface and a sidewall surface extending upwardly from the bottom surface. The semiconductor device structure also includes a first gate structure disposed in the first recess and in contact with the bottom surface and the sidewall surface, the first gate structure extending along a second direction substantially perpendicular to the first direction. The semiconductor device structure further includes a first spacer disposed on opposite sidewalls of the first gate structure and in contact with the first fin and the first gate structure.