An electron beam inspection apparatus according to one aspect of the present invention includes an image acquisition mechanism to acquire a secondary electron image by scanning a substrate, on which a figure pattern is formed, with an electron beam, and detecting a secondary electron emitted due to irradiation with the electron beam by the scanning, a resize processing unit to perform, using design pattern data being a basis of the figure pattern, resize processing on the figure pattern to enlarge its size in a scan direction of the electron beam, a first developed image generation unit to generate, using the design pattern data which has not been resized, a first developed image by developing an image of a design pattern of a region corresponding to the secondary electron image, a second developed image generation unit to generate, using partial patterns enlarged by the resize processing in the figure pattern having been resized, a second developed image by developing an image of partial patterns in a region corresponding to the secondary electron image, a map generation unit to generate a pseudo defect candidate pixel map which can identify a pseudo defect candidate pixel that has no pattern in the first developed image and has a pattern in the second developed image, a reference image generation unit to generate a reference image of the region corresponding to the second electron image, and a comparison unit to compare, using the pseudo defect candidate pixel map, the second electron image with the reference image of the region corresponding to the second electron image.