An apparatus for inspecting pattern defects, the apparatus including: a defect candidate extraction unit configured to perform a defect candidate extraction process by comparing a detected image signal with a reference image signal; and a defect detection unit configured to perform a defect detection process and a defect classification process based on a partial image containing a defect candidate that is extracted by the defect candidate extraction unit, wherein the processes performed by the defect candidate extraction unit and/or the defect detection unit are performed asynchronously with an image acquisition process.