A system and method for registration and coordinated manipulation of augmented reality image components includes registering a model of patient anatomy to a first image of the patient anatomy captured using an imaging device to determine a baseline relationship between the model and the first image, tracking movement of a computer-assisted device used to manipulate the imaging device, updating the baseline relationship based on the tracked movement, and generating a composite image by overlaying the model on a second image of the patient anatomy according to the updated relationship. In some embodiments, the model is semi-transparent. In some embodiments, registering the model to the first image includes adjusting the model relative to the first image based on one or more inputs received from a user and generating a model to image transformation based on the adjustments to the model. The model to image transformation captures the baseline relationship.