A device includes a semiconductor substrate, a gate dielectric over the semiconductor substrate, and a gate electrode over the gate dielectric. The gate electrode has a first portion having a first thickness, and a second portion having a second thickness smaller than the first thickness. The device further includes a source/drain region on a side of the gate electrode with the source/drain region extending into the semiconductor substrate, and a device isolation region. The device isolation region has a part having a sidewall contacting a second sidewall of the source/drain region to form an interface. The interface is overlapped by a joining line of the first portion and the second portion of the gate electrode.