This document describes a search retrieval system for automatically indexing data representing audio-visual recordings and for querying, responsive to a search query, that indexed data representing the audio-visual recordings. The search retrieval system defines weights for semantic features of an audio-visual recording and extracts, based on execution of a first set of rules, one or more semantic features. The system determines a weight for each of the one or more semantic features. A search engine searches nodes in the graph of the semantic features to identify one or more logical relationships for the one or more semantic features extracted. The weights for each of the one or more semantic features are adjusted based on the graph. The data is indexed in association with the one or more adjusted weights for the one or more semantic features, respectively.