A method and an apparatus for efficiently enlarging an image, where the outermost Discrete Cosine Transform (DCT) components from a DC value are regarded as high frequency components in DCT components expressed in one two-dimensional coordinate system, the edge signals of an image are restored by means of only the high frequency components, and then a binary edge mask, which represents the distribution of high frequency components within the image, is generated from the restored edge signals. Accordingly, an interpolation order can be determined based on the binary edge mask and interpolation can be performed. A higher order low pass filter is applied to an area having many high frequency signal components, so that it is possible to output more precisely enlarged image signals. In this way, that it is possible to improve the overall picture quality of an enlarged image, and to properly adjust the amount of calculation and buffer capacity, which are used when interpolation is performed.