A semiconductor device which includes: a substrate; a first set of fins above the substrate of a first semiconductor material; a second set of fins above the substrate and of a second semiconductor material different than the first semiconductor material; and an isolation region positioned between the first and second sets of fins, the isolation region having a nitride layer. The isolation region may be an isolation pillar or an isolation trench.