According to one embodiment, a magnetic memory device including a stacked structure including a first magnetic layer having a variable magnetization direction, a second magnetic layer having a fixed magnetization direction, and a nonmagnetic layer provided between the first magnetic layer and the second magnetic layer, and containing magnesium (Mg) and oxygen (O). The nonmagnetic layer further contains a first additive element and a second additive element, the first additive element is at least one element selected from sulfur (S), gallium (Ga), aluminum (Al), titanium (Ti), vanadium (V), hydrogen (H), fluorine (F), manganese (Mn), lithium (Li), nitrogen (N) and magnesium (Mg), and the second additive element is lithium (Li).