A magnetic memory element having a memory cell of size 4F2 is provided that realizes a crosspoint-type memory. In the magnetic memory element, a first magnetic layer, a third magnetic layer (spin polarization enhancement layer), an intermediate layer, a fourth magnetic layer (spin polarization enhancement layer), and a second magnetic layer are stacked in order. The intermediate layer is made of an insulating material or a nonmagnetic material. The second magnetic layer is composed of a ternary alloy of gadolinium, iron and cobalt, a binary alloy of gadolinium and cobalt, or a binary alloy of terbium and cobalt. Alternatively, the first magnetic layer is composed of a ternary alloy of terbium, iron and cobalt, or a binary alloy of terbium and cobalt.