An inverse quantization processing method and associated device is provided comprising a determining unit, a bit reduction unit, a frequency/time-domain conversion unit, and a bit addition unit which collectively perform an inverse quantization processing method by receiving a plurality of quantized frequency-domain data, performing a bit reduction on a current quantized frequency domain data according to an adjusting factor, generating a bit-reduced quantized frequency-domain data when an accumulated count value of a plurality of valid bit numbers of the plurality of quantized frequency-domain data greater than a first threshold reaches a second threshold, performing a frequency/time domain conversion on the bit-reduced quantized frequency-domain data to obtain a bit-reduced time-domain PCM data, and performing a bit addition on the bit-reduced time-domain PCM data according to the adjusting factor, so as to obtain a time-domain PCM data.