A relay station and a method of operating a relay station are disclosed. The method includes decoding a received signal to restore the received signal into an information packet, and obtaining a first probability sequence representing probability information of respective information bits constituting the information packet, encoding the information packet to obtain a coded packet, and obtaining a second probability sequence representing probability information of respective coded bits constituting the coded packet by using the first probability sequence, and obtaining a modulated symbol by using the second probability sequence.