A semiconductor device includes a semiconductor substrate with an isolation layer formed in the semiconductor substrate to delimit active regions. Recess patterns for gates are defined in the active regions and the isolation layer. Gate patterns are formed in and over the recess patterns for gates, and a gate spacer is formed to cover the gate patterns. The recess patterns for gates have a first depth in the active regions and a second depth, which is greater than the first depth, in the isolation layer. Gaps are created between the gate patterns and upper parts of the recess patterns for gates that are defined in the isolation layer. The gate spacer fills the gaps and protects the gate spacer so as to prevent bridging.