Provided are an analog digital converting device and a reference voltage controlling method thereof. The analog digital converting device includes: a first reference voltage generating circuit generating a first reference voltage; a second reference voltage generating circuit generating a second reference voltage; a first sub analog digital converter receiving an analog input signal and converting the analog input signal into a first digital signal by using the first reference voltage; an amplifier converting the first digital signal into a voltage corresponding to the first digital signal by using the first reference voltage and amplifying a difference between a voltage level of the analog input signal and a voltage level corresponding to the first digital signal to output a residual signal; and a second sub analog digital converter receiving the residual signal and converting the residual signal into a second digital signal by using the second reference voltage.