Provided are an apparatus for encoding an image and an apparatus for decoding an image. The apparatus for encoding the image includes a grouping unit to group transform coefficients according to layers and generate groups, a pattern encoding unit to generate and encode the pattern information of each group by using the quantized transform coefficients existing in the groups of each layer in an order from an upper layer to a lower layer, and a level encoding unit to encode levels of the quantized transform coefficients corresponding to the pattern information of the groups of each layer and generate bitstreams along with the pattern information of the groups of each layer.