A system for calculating mask data to create a desired layout pattern on a wafer reads all or a portion of a desired layout pattern. Mask data having pixels with transmission values is defined along with corresponding optimal mask data pixel transmission values. An objective function is defined that compares image intensities as would be generated on a wafer with an optimal image intensity at a point corresponding to a pixel. The objective function is minimized to determine the transmission values of the mask pixels that will reproduce the desired layout pattern on a wafer.