An image processing apparatus for controlling brightness of an image to be output to a display unit includes a separation unit which separates an input image into images in a plurality of areas, and an entropy calculation unit which calculates, for the image in each area, entropy information on at least one of distribution of luminosity, character quantity, and motion quantity. In addition, a brightness control unit controls brightness for the image in each area so that a first reduction rate of brightness in an area of which entropy is relatively small is greater than a second reduction rate of brightness in an area of which entropy is relatively large.