A method, includes, in part, defining a continuous signal, defining a threshold value, calibrating the continuous signal and the threshold value from measurements made on edges of one or more patterns on a mask and corresponding edges of the patterns on a wafer, convolving the continuous signal with a kernel to form a corrected signal, and establishing, by a processor, a probability of forming an edge at a point along the corrected signal in accordance with a difference between the value of the corrected signal at the point and the calibrated threshold value. The kernel is calibrated using the same measurements made on the patterns' edges.