A method of detecting a defect, including the steps of: illuminating step for illuminating a sample with a light; detecting step for detecting light from the specimen which is illuminated by the light and forming an image by processing the detected light; processing step for extracting a defect candidate by processing the image of the sample formed in the detecting step and determining an inspection condition by using images including the image of the sample acquired in the detecting step, a partial image including the extracted defect candidate and a reference image which corresponds to the partial image including the defect candidate.