A three-dimensional image processing apparatus includes: a distance image generating part capable of generating a distance image based on a plurality of images captured in an image capturing part; a pattern generating part for generating a first projection pattern and a second projection pattern whose fringe direction is different from that of the first projection pattern as a plurality of projection patterns obtained by changing a fringe direction of a projection pattern; and an incorrect-height determining part for making comparison in height information of a corresponding portion of an inspection target between a first distance image, generated in the distance image generating part based on a first pattern projected image, and a second distance image, generated based on a second pattern projected image, to determine height information of a portion where a difference not smaller than a predetermined value has occurred as incorrect.