A digital medium environment is described to automatically generate a trimap and segment a digital image, independent of any user intervention. An image processing system receives an image and a low-resolution mask for the image, which provides a probability map indicating a likelihood that a pixel in the image mask corresponds to a foreground object in the image. The image processing system analyzes the image to identify content in the image's foreground and background portions, and adaptively generates a trimap for the image based on differences between the identified foreground and background content. By identifying content of the image prior to generating the trimap, the techniques described herein can be applied to a wide range of images, such as images where foreground content is visually similar to background content, and vice versa. Thus, the image processing system can automatically generate trimaps for images having diverse visual characteristics.