A semiconductor device structure is provided. The semiconductor device structure includes a fin structure protruding from a semiconductor substrate. The fin structure includes a first portion and an overlying second portion. The first portion is formed of a material that is the same as that of the semiconductor substrate and different from that of the second portion. The semiconductor device structure also includes a liner structure and an isolation feature. The liner structure includes a carbon-doped silicon oxide film covering the semiconductor substrate and the first portion of the first fin structure and a nitrogen-containing film over the carbon-doped silicon oxide film. The isolation feature is over the nitrogen-containing film and surrounded by the liner structure.