A semiconductor device includes a magnetic tunneling junction (MTJ) on a substrate, a cap layer adjacent to and directly contacting the MTJ, a first inter-metal dielectric (IMD) layer around the MTJ, a top electrode on the MTJ, a metal interconnection under the MTJ, and a second IMD layer around the metal interconnection. Preferably, the cap layer is a single layer structure made of dielectric material and an edge of the cap layer contacts the first IMD layer directly.