A charged particle beam inspection apparatus includes a first deflector to deflect N×N′ multiple beams collectively to N×N′ small regions having a size p/M in the first direction and arrayed at the pitch p in the first direction, perform tracking deflection, and re-deflect the multiple beams collectively to next N×N′ small regions away from the N×N′ small regions by N small regions in the first direction, by the stage completes a movement of a distance of N/M×p so as to reset the tracking deflection; and a second deflector to deflect the multiple beams collectively to scan the N×N′ small regions concerned while the tracking deflection is performed.