Techniques and systems are described for automatic artifact removal in a digital image. A segmentation map is generated that describes a magnitude of difference among pixels in a digital image. Contours may be generated that describe boundaries of objects described in the segmentation map. The contours may be filtered according to two-dimensional and three-dimensional cues to identify contours corresponding to artifacts in the digital image. For each contour corresponding to an artifact, an object mask and a sampling mask may be generated. The object mask and the sampling mask may be utilized as part of a content filling operation upon the digital image to remove the artifact, and a corrected digital image is generated that does not include the artifact.