In an optical element, a first diffraction grating, a second diffraction grating, a third diffraction grating, and a fourth diffraction grating are formed in each of a first surface of a first substrate, a fourth surface of a second substrate, a fifth surface of a third substrate, and an eighth surface of a fourth substrate. Each substrate is fixed by (a first adhesive layer, second adhesive layer, and a third adhesive layer) adhesive layers including a gap material. A first filler with a refractive index equal to that of the first substrate and the second substrate is filled between the first substrate and the second substrate and a third filler formed of air or a medium with a refractive index equal to that of air is filled between the second substrate and the third substrate.