A system and method for fine-grained privacy control of image and video recordings uses a model-based approach to solve privacy issues. The method includes steps of: receiving a digital capture of a subject bound by privacy constraints; receiving information regarding the subject, the environment, and the privacy constraints; using the information to generate a model image of the subject within the environment; associating a disposition tag with an appearance of the subject in the model image; and comparing the model image with the digital capture to determine coordinates in the recording where a privacy control action, such as a blocking action, should be applied.