A mechanism is provided that implements a cognitive data processing system for automatically processing ambiguously labeled data associated with a medical image. The cognitive data processing system receives an ambiguously labeled set of training data in which the ambiguously labeled set of training data comprises portions of data and associated labels, and wherein at least one portion of data in the ambiguously labeled set of training data has a plurality of different labels that together render the portion of data ambiguously labeled. The cognitive data processing system configures an implementation of a model that comprises a loss term, a maximizing term, and a sparsity term. The cognitive data processing system processes the ambiguously labeled set of training data based on the model to identifying a mapping that minimizes a loss function and thereby train the cognitive data processing system.