Methods, systems, and storage media for classifying content across media formats based on weak supervision and cross-modal training are disclosed. The system can maintain a first feature classifier and a second feature classifier that classifies features of content having a first and second media format, respectively. The system can extract a feature space from a content item using the first feature classifier and the second feature classifier. The system can apply a set of content rules to the feature space to determine content metrics. The system can correlate a set of known labelled data to the feature space to construct determinative training data. The system can train a discrimination model using the content item and the determinative training data. The system can classify content using the discrimination model to assign a content policy to the second content item.