An image-based overlay metrology system is disclosed. The system includes a controller couplable to a metrology sub-system. The controller is configured to receive a set of image signals of a first metrology target disposed on the sample from the metrology sub-system and determine a plurality of harmonic detectivity metric values by calculating a harmonic detectivity metric value for each of the plurality of image signals. The controller is also configured to identify a set of optical measurement conditions of the metrology sub-system based on the plurality of harmonic detectivity metric values, wherein the set of optical measurement conditions define a recipe for optical metrology measurements of the metrology sub-system. The controller then provides the recipe to the metrology sub-system for execution of one or more optical metrology measurements of one or more additional metrology targets.