A tool for providing contextual policy-based computer vision control. The tool identifies one or more camera devices in an environment. The tool determines a plurality of vision areas in a field of view within the environment. The tool determines one or more contextual policies for the plurality of vision areas in the field of view. The tool determines one or more vision augmentations for the plurality of vision areas based, at least in part, on an aggregate computer vision capability of the one or more camera devices in the environment and the one or more contextual policies for the plurality of vision areas in the field of view. The tool applies the one or more vision augmentations to at least one of the one or more camera devices in the environment.