Mechanisms are provided to provide an improved computer tool for determining and mitigating the presence of adversarial inputs to an image classification computing model. A machine learning computer model processes input data representing a first image to generate a first classification output. A cohort of second image(s), that are visually similar to the first image, is generated based on a comparison of visual characteristics of the first image to visual characteristics of images in an image repository. A cohort-based machine learning computer model processes the cohort of second image(s) to generate a second classification output and the first classification output is compared to the second classification output to determine if the first image is an adversarial image. In response to the first image being determined to be an adversarial image, a mitigation operation by a mitigation system is initiated.