Disclosed herein is an image processing apparatus, including: a projection section adapted to alternately project an image for a right eye and another image for a left eye; a separation section adapted to separate a projected image of the right eye image and another projected image of the left eye image from each other; a gaze detection section adapted to detect a gaze direction of an observer to which the right eye image and the left eye image are to be given; and a luminance adjustment section adapted to set a range with respect to the gaze direction to the right eye image and the left eye image and adjust a luminance value of pixels so that a portion of the right and left eye images other than the set range exhibits an invisible state.