An interference optical system unit includes at least one electromagnetic lens that forms an image of a charged particle beam, at least one charged particle beam biprism, and a support member for the electromagnetic lens and the charged particle beam biprism. The electromagnetic lens, the charged particle beam biprism, the support member, and a space to an image plane of the electromagnetic lens are integrally configured as one unit. The interference optical system unit is disposed to have an optical axis coaxialized with an optical axis of an imaging optical system of an upstream stage that is disposed on an upstream side of the unit in a flow direction of the charged particle beam. A focal length of the electromagnetic lens and a deflection angle of the charged particle beam given by the charged particle beam biprism are controlled to generate an interference fringe of the charged particle beam on the image plane of the electromagnetic lens.