An iterative detection and decoding (IDD) device in a receiving device of a wireless communication system by using multiple-input multiple-output (MIMO) and a channel code by connecting is provided. The IDD device includes an MIMO detector that generates an output log likelihood ratio (LLR) value by using signals received from a plurality of antennas and a feedback LLR value, a channel decoder for outputting a channel decoded LLR value through channel decoding and a second operation by using the LLR value of the MIMO detector, and a feedback compensator for generating the feedback LLR value so that the channel decoded LLR value is within the range of an upper threshold and a lower threshold determined based on a bit width of an LLR used by the receiving device.