The present disclosure relates to an object selection system that automatically detects and selects objects in a digital image utilizing a large-scale object detector. For instance, in response to receiving a request to automatically select a query object with an unknown object class in a digital image, the object selection system can utilize a large-scale object detector to detect potential objects in the image, filter out one or more potential objects, and label the remaining potential objects in the image to detect the query object. In some implementations, the large-scale object detector utilizes a region proposal model, a concept mask model, and an auto tagging model to automatically detect objects in the digital image.