A method for decoding a residual block, includes: obtaining residual data from encoded data; identifying one or more conditions for enabling an inverse quantization and an inverse transform; skipping both of the inverse quantization and the inverse transform for the obtained residual data to thereby reconstruct a residual block directly from the obtained residual data, when the one or more identified conditions indicate skipping of both the inverse quantization and the inverse transform; inversely quantizing the obtained residual data and skipping the inverse transform for the inversely quantized residual data to thereby reconstruct a residual block using the inverse-quantized residual data, when the one or more identified conditions indicate skipping of the inverse transform; and inversely quantizing the obtained residual data and inversely transforming the inversely quantized residual data to thereby reconstruct a residual block using the inverse transformed residual data.