A device includes a p-type metal-oxide-semiconductor (PMOS) device and an n-type metal-oxide-semiconductor (NMOS) device at a front surface of a semiconductor substrate. A first dielectric layer is disposed on a backside of the semiconductor substrate. The first dielectric layer applies a first stress of a first stress type to the semiconductor substrate, wherein the first dielectric layer is overlying the semiconductor substrate and overlapping a first one of the PMOS device and the NMOS device, and is not overlapping a second one of the PMOS device and the NMOS device. A second dielectric layer is disposed on the backside of the semiconductor substrate. The second dielectric layer applies a second stress to the semiconductor substrate, wherein the second stress is of a second stress type opposite to the first stress type. The second dielectric layer overlaps a second one of the PMOS device and the NMOS device.