A system and method of adaptive edge detection and noise reduction in an image where edge information is detected for each color component of each pixel, whether sensed or synthesized. In some embodiments, the filter applied to a selected non-edge pixel may be determined by the ultimate size of a region around the selected pixel, where the size of the region may be increased if a count of the non-edge pixels in the region is less than a threshold value.