An image processing apparatus and a method of processing an image are provided. The image processing apparatus includes a depth estimation unit for estimating a depth of an input three-dimensional (3D) image; a text area detection unit for detecting a text area included in the 3D image; a mask generation unit for generating a text mask for the text area; and a depth correction unit for correcting a depth of the text area based on the estimated depth of the input 3D image and the text mask.