A phase difference detection device able to detect a phase with a high precision is provided. A phase difference detection device 4 detecting a phase difference θ between an excitation signal Ss and a second detection signal Sd in accordance with a rotation position θ of a rotary body 101 has a reference counter 13 performing a count in synchronization with a first clock pulse CLK1 and reset when the excitation signal Ss causes zero cross down, a velocity detection unit 19 acquiring information capable of specifying the rotation velocity of the rotary body 101, a pulse conversion unit 21 outputting a second clock pulse CLK2 by using, as a pulse interval Tp, a time specified based on the information acquired by the velocity detection unit 19 and required for the rotation of the rotary body 101 by an angle corresponding to 1 count of the reference counter 13, and a phase counter 23 in which a count value counted by the reference counter 13 is set at an initial value for each fall of a second detection signal Sd, and which performs count in synchronization with the second clock pulse CLK2.