A method includes forming an opening in a dielectric to reveal a protruding semiconductor fin, forming a gate dielectric on sidewalls and a top surface of the protruding semiconductor fin, and forming a conductive diffusion barrier layer over the gate dielectric. The conductive diffusion barrier layer extends into the opening. The method further includes forming a silicon layer over the conductive diffusion barrier layer and extending into the opening, and performing a dry etch on the silicon layer to remove horizontal portions and vertical portions of the silicon layer. After the dry etch, a conductive layer is formed over the conductive diffusion barrier layer and extending into the opening.