A substrate with a multilayer reflective film, a reflective mask blank, a reflective mask and a method of manufacturing a semiconductor device that can prevent contamination of the surface of the multilayer reflective film even in the case of having formed reference marks on the multilayer reflective film. A substrate with a multilayer reflective film contains a substrate and a multilayer reflective film that reflects EUV light formed on the substrate. Reference marks are formed to a concave shape on the surface of the substrate with the multilayer reflective film. The reference marks have grooves or protrusions roughly in the center. The shape of the grooves or protrusions when viewed from overhead is similar or roughly similar to the shape of the reference marks.