A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types. In one embodiment of the present invention, the MMR system includes an action processor and method, and MMR documents with an associated action. The MMR document structure specifies different actions for different MMR documents combined with the ability to create any number of MMR documents for a particular location on any media, which allows the MMR architecture to serve as a universal trigger or initiator for additional processing. In another embodiment, the MMR system, in particular MMR recognition and the input/output capabilities of the capture device, are used as a control mechanism for a third party system.