An amount of pattern position displacement between observation images acquired by irradiating from two different directions is changed depending on beam deflection for moving an image acquisition position. In a pattern evaluation method that measures astigmatic difference or focus position displacement having a small amount of dose at a high speed using parallax caused by the tilted beam, a correction value obtained in advance by measurement is reflected in an amount of pattern position displacement between observation images obtained by irradiating from at least two different directions and generated in accordance with the amount of beam deflection for moving an image acquisition position. A processing unit calculates an amount of correction of an amount of pattern position displacement depending on beam deflection of a beam deflecting unit for moving an image acquisition position on the sample at a high speed.