Disclosed is a method for encoding/decoding a still image or moving image based on an intra-prediction mode and an apparatus supporting the same. Particularly, a method for processing an image based on an intra prediction mode may include determining an intra prediction mode set used for an intra prediction in a current prediction processing unit among multiple intra prediction mode sets having different resolutions with each other, wherein the resolution represents a degree of distribution of the intra prediction mode in the intra prediction mode set; deriving an intra prediction mode applied to the current prediction processing unit in the determined intra prediction mode set using index information received from an encoder; and generating a prediction sample of the current prediction processing unit based on the derived intra prediction mode, and each of the intra prediction mode sets includes one or more intra prediction modes.