A signal processing device includes: an amplitude reducing unit to reduce, when a phase shift control signal indicating a phase shift of a transmission wave whose frequency is modulated is output, among amplitudes of a beat signal having a difference frequency between the transmission wave and a reception wave, an amplitude of the beat signal at a timing at which a phase of the transmission wave is shifted in accordance with the phase shift control signal; and a signal converting unit to convert the beat signal whose amplitude at the timing at which the phase of the transmission wave is shifted is reduced by the amplitude reducing unit into a signal in a frequency domain.