Embodiments of the present invention combine the flexibility of the computer in segmenting, indexing, and retrieving data with the strengths of people in analyzing scenes. They provide a mechanism that allows a group of analyzers at distributed locations to collaborate in generating pieces of semantic information of a data stream, integrates these pieces into a summary representation that can be used for subsequent computation, and allows related hypotheses and information requests to be presented back to the analyzers. This description is not intended to be a complete description of, or limit the scope of, the invention. Other features, aspects, and objects of the invention can be obtained from a review of the specification, the figures, and the claims.