An optical displacement measuring device includes a diffraction grating, a reflecting optical system configured to irradiate two one-time diffracted beams diffracted at the diffraction grating on the diffraction grating again, and the reflecting optical system includes a first imaging element, a second imaging element, a first reflector, and a second reflector, wherein the focal length of the first imaging element and the focal length of the second imaging element are the same, the diffraction grating and first reflector are disposed around the focal position of the first imaging element, and the diffraction grating and second reflector are disposed around the focal position of the second imaging element, thereby suppressing influence of displacement of the diffraction grating as to other than a direction where a movement position is detected.