Systems and methods according to the present invention provide techniques to reliably detect edges, lines and quadrilaterals, especially those with low local contrast, in color images. Edges can be detected using a color gradient operator is based on color distance with a non-linear weight determined by the consistency of local gradient orientations, thereby significantly improving the signal/noise ratio. In detecting lines, a variant of the Gradient Weighted Hough Transform can be used employing both the edge strength and orientation. Multiple non-overlapping quadrilaterals can be detected using a process which includes quality metrics (for both individual quadrilaterals and for a set of non-overlapping quadrilaterals) and a graph-searching method.