The present invention relates generally to a system and methods for medical reporting. More specifically, the invention is directed to a system that includes a structured reference frame component through the use of which one or more anatomical features or pathologies that are shown on an actual image taken of a patient can be associated with the data and other information provided in a structured knowledge base. Certain embodiments of the system include a structured reference frame component that can be used for a plurality of medical images taken of a patient, the plurality of images being ones taken, for example, of different views or projections or at different times to provide what is termed for purposes of this application a structured image set. Advantageously, such a structured image set places the subject anatomical feature or pathology in a greater spatial and/or temporal context. Additionally, embodiments of the invention allow users through the use of structured image set to annotate, augment, and overall interact with the clinically more accurate and detailed medical image, not the conventional anatomical cartoons.