A cross descriptor learning system, method and program product therefor. The system extracts descriptors from unlabeled exemplars. For each unlabeled exemplar, a cross predictor uses each descriptor to generate labels for other descriptor. An automatic label generator also generates labels for the same unlabeled exemplars or, optionally, for labeled exemplars. A label predictor results for each descriptor by combining labels from the cross predictor with labels from the automatic label generator.