A method and structure for storing information for one or more semantic objects derived from raw data. A semantic object extracted from the raw data and classified to comprise the semantic object is received, the received semantic object having one or more attributes. A summary of attributes of the semantic object by calculating one or more statistics of one or more of the one or more attributes of the received semantic object, a confidence level of the received semantic object that quantifies a degree of certainty that the received semantic object has been correctly classified and/or labeled; and a compact representation of raw data of the received semantic object are generated. Indexing information for one or more of the summary of attributes and the compact representation of the semantic object is generated. The semantic object, along with its associated summary of attributes, confidence level, compact representation, and indexing information, stored in a semantic object database associated with a database storing the raw data.