A structure of semiconductor device is provided, including a substrate. First and second trench isolations are disposed in the substrate. A height of a portion of the substrate is between a top and a bottom of the first and second trench isolations. A gate insulation layer is disposed on the portion of the substrate between the first and second trench isolations. A first germanium (Ge) doped layer region is disposed in the portion of the substrate just under the gate insulation layer. A second Ge doped layer region is in the portion of the substrate, overlapping with the first Ge doped layer region to form a Ge gradient from high to low along a depth direction under the gate insulation layer. A fluorine (F) doped layer region is in the portion of the substrate, lower than and overlapping with the first germanium doped layer region.