There is provided an interference removing method of an image system including the steps of: respectively acquiring different numbers of images within two brightness intervals having different brightness variations using an image sensor having a sampling frequency; lighting a light source with a lighting frequency, which is a half of the sampling frequency, and synchronizing to the sampling frequency; and subtracting a first image, which is associated with the lighting of the light source, acquired in a later brightness interval of two adjacent brightness intervals having identical brightness variations by a second image, which is associated with a sampling time of the first image, acquired in an earlier brightness interval of the two adjacent brightness intervals. The present disclosure further provides an image system.