According to one embodiment, a magnetic memory device includes a semiconductor substrate, a first lower area provided on the semiconductor substrate, and including a plurality of magnetoresistive effect elements, a second lower area provided on the semiconductor substrate, and being adjacent to the first lower area, a first upper area provided above the first lower area, and including a first material film formed of an insulating material or a semiconductor material, and a second upper area provided above the second lower area, being adjacent to the first upper area, and including a second material film formed of an insulating material different from a material of the first material film.