Tracking patterns during a semiconductor fabrication process includes: obtaining an image of a portion of a fabricated device; extracting contours of the portion of the fabricated device from the obtained image; aligning the extracted contour to a matching section of a reference design; decomposing the matching section of the reference design into one or more patterns; and updating a pattern tracking database with information pertaining to at least one pattern in the one or more patterns generated as a result of the decomposition.