An embodiment is a method including forming a first gate over a substrate, the first gate having first gate spacers on opposing sidewalls, forming a first hard mask layer over the first gate, forming a second hard mask layer over the first hard mask layer, the second hard mask layer having a different material composition than the first hard mask layer, forming a first dielectric layer adjacent and over the first gate, etching a first opening through the first dielectric layer to expose a portion of the substrate, at least a portion of the second hard mask layer being exposed in the first opening, filling the first opening with a conductive material, and removing the second hard mask layer and the portions of the conductive material and first dielectric layer above the first hard mask layer to form a first conductive contact in the remaining first dielectric layer.