A system and method of data correlation and analysis of past activities and prediction of future activity that includes establishing a database comprising structured metadata on at least one computer and a distributed system of networked computers having a plurality of access levels, configuring a database front-end to parse incoming data, the database front-end defining a plurality of entities and events, each having one or more attributes, and correlation IDs, which are user defined attribute relationships that correlate events and entities, collecting data, parsing the incoming data in the database front-end into structured metadata, the distributed analysis engine correlating the incoming structured metadata with structured metadata in the database and determining micro-patterns in the structured metadata, generating a graphical representation of a composite network of the determined micro-patterns as super-objects, and predicting future activity from the one or more super-objects representing the determined micro-patterns.