A semiconductor device includes: an isolation insulating layer; fin structures protruding from the isolation insulating layer; gate structures, each having a metal gate and a cap insulating layer disposed over the metal gate; a first source/drain epitaxial layer and a second source/drain epitaxial layer disposed between two adjacent gate structures; and a first conductive contact disposed on the first source/drain epitaxial layer, and a second conductive contact disposed on the second source/drain epitaxial layer; a separation isolation region disposed between the first and second conductive contact; and an insulating layer disposed between the separation isolation region and the isolation insulating layer. The separation isolation region is made of a different material than the insulating layer.