Producing an event related video content data structure includes processing a video feed through a spatiotemporal pattern recognition algorithm that uses machine learning to develop an understanding of an event within the video feed. Developing the understanding includes identifying context information relating to the event and identifying an entry in a relationship library at least detailing a relationship between two visible features of the video feed. Content of the video feed that displays the event is automatically extracted by a computer and associated with the context information. A video content data structure that includes the context information is produced.