An image of a portion of a semiconductor die is obtained that shows one or more structures in a first process layer and one or more structures in a second process layer. Using machine learning, a first region is defined on the image that at least partially includes the one or more structures in the first process layer. Also using machine learning, a second region is defined on the image that at least partially includes the one or more structures in the second process layer. An overlay offset between the one or more structures in the first process layer and the one or more structures in the second process layer is calculated using the first region and the second region.