In an image processing apparatus for encoding image data and a method of controlling the same, whether an attribute of each of a plurality of areas in image data corresponds to an edge in an image based on the image data is determined, and one of a plurality of sub-sampling processes is selected according to the determination for each of the plurality of areas. Note that the plurality of sub-sampling processes can sub-sample color difference components of each of the plurality of areas by different processes. By the sub-sampling process selected as that corresponding to each of the plurality of areas, each of the plurality of areas is sub-sampled to encode the image data.