A semiconductor device is provided. The semiconductor device includes: a substrate and first gate structures and source/drain doped layers on the substrate. Each of the source/drain doped layers is located at two sides of one first gate structure. The semiconductor device further includes a dielectric layer on the substrate. The dielectric layer contains first grooves, exposing the source/drain doped layers, wherein each first groove includes a first-groove bottom part and a first-groove top part located above the first-groove bottom part, and a size of the first-groove top part is larger than a size of the first-groove bottom part. The semiconductor device further includes a first conductive structure located in the first-groove bottom part, an insulating layer located in the first-groove top part and on the first conductive structure, and a second conductive structure located in the dielectric layer and connected to the first gate structure.