The present disclosure provides systems and methods for building an inventory database with automatic labeling. A system can maintain a hierarchical concept tree including labels. Each of the labels is associated with a set of attributes and a respective embedding. The system can receive, from a provider device, a request to generate labels for an item of media content. The request can include a request attribute. The system can generate, using a gated categorical model, document embeddings for the item of media content. The system can select a subset of the labels based on the request attribute. The system can determine a respective label score for each label of the subset of the labels based on the document embeddings and the respective embedding of the label. The system can provide a selected label of the subset of the labels based on the respective label score of the selected label.