Provided are a method and apparatus for processing an image to remove fog from the image. The apparatus for processing an image includes: a transmission estimation unit configured to calculate a transmission of an input image by synthesizing pixel-based transmissions of current pixels of the current image including a fog component, and block-based transmissions of a plurality of blocks of a previous image, each block including a plurality of pixels; an airlight estimation unit configured to calculate airlight of the input image; and an image restoration unit configured to generate, based on the transmission and the airlight, a restored image in which the fog component is removed from the current image.