A charged particle beam inspection apparatus comprises: an electron gun for irradiating an electron beam onto a sample; a detector for detecting a signal obtained from the sample; an image processor for forming an image from the signal obtained from the detector, and an energy controller for controlling the beam energy of the electron beam to be irradiated onto the sample. An identical charged particle beam inspection apparatus carries out a plurality of types of inspections. An inspection apparatus of a projection type may be applied thereto. A pattern defect inspection, a foreign material inspection, and an inspection for a defect in a multilayer are carried out. Beam energies E1, E2, and E3 in those inspections have a relation E1>E2 and E3>E2. Charge removal is performed in a transport chamber or other vacuum chamber before an inspection.