A method for transmitting multicast data in a WiMax/WiBro relay system, including transmitting carrier-interference-noise ratios from UEs to corresponding Relays; selecting the minimum value among the carrier-interference-noise ratios; determining the modulation and coding mode according to the selected minimum value; transmitting the modulation and coding mode to BS; selecting a modulation and coding mode with a highest rate for multicast data on an access link; transmitting the final modulation and coding mode to the Relays; and transmitting data via the access link with the modulation and coding mode determined by the BS.