Systems and methods are described for a face annotation framework with partial clustering and interactive labeling. In one implementation, an exemplary system automatically groups some images of a collection of images into clusters, each cluster mainly including images that contain a person's face associated with that cluster. After an initial user-labeling of each cluster with the person's name or other label, in which the user may also delete/label images that do not belong in the cluster, the system iteratively proposes subsequent clusters for the user to label, proposing clusters of images that when labeled, produce a maximum information gain at each iteration and minimize the total number of user interactions for labeling the entire collection of images.