A method and system for defining a model by analyzing images of a physical space. In some embodiments the images of a physical space contain distinctive visual features with associated semantic information, and the model is defined using image feature detection techniques to identify distinctive visual features and a rich marker-based markup language to give meaning to the distinctive visual features. In some embodiments the distinctive visual features are predefined markers, and the markup language specifies model aspects and rules for combining semantic information from a plurality of markers to define the model.