Techniques are disclosed for determining if the decomposition of layout design data is feasible, and for optimizing the segmentation of polygons in decomposable layout design data. Layout design data is analyzed to identify the edges of polygons that should be imaged by separate lithographic masks. In addition, proposed cut paths are generated to cut the polygons in the layout design data into a plurality of polygon segments. Once the separated edges and cut paths have been selected, a conflict graph is constructed that reflects these relationships. Next, a dual of the conflict graph is constructed. This dual graph will have a corresponding separation dual graph edge for each separated polygon edge pair in the layout design data. The dual graph also will have a corresponding cut path dual graph edge for each proposed cut path generated for the layout design data. After the dual graph has been constructed, it is analyzed to determine which of the proposed cut paths should be kept and which should be discarded. The layout design data is then modified to include the cut paths corresponding to the selected cut path dual graph edges. Alternately or additionally, separate sets of layout design data may be decomposed from original layout design data using the selected cut paths.