Provided is a method that de-multiplexes a bit stream to extract intra prediction mode information and residual signals, restores an intra prediction mode of a current prediction unit using the intra prediction information and intra prediction modes of prediction units adjacent to the current prediction unit, generates reference pixels using one or more available reference pixel if there exist unavailable reference pixels of the prediction unit, adaptively filters the reference pixels based on the restored intra prediction mode, generates a prediction block using the restored intra prediction mode and the reference pixels, decodes the residual signal to generating a residual block, and generates a reconstructed block using the prediction block and the residual block. Accordingly, additional bits resulted from increase of a number of intra prediction mode are effectively reduced. Also, an image compression ratio can be improved by generating a prediction block similar to an original block.