A method of decoding an image including performing entropy-decoding to obtain quantized transformation coefficients of at least one transformation unit in a coding unit of the image, performing inverse-quantization and inverse-transformation on the quantized transformation coefficients of the at least one transformation unit to obtain residuals, and performing inter prediction for at least one prediction unit in the coding unit to generate a predictor and restoring the image by using the residuals and the predictor.