A method and apparatus for determining missing color pixel values in image data (i.e., for demosaicing image data) using a small number of lines in a buffer memory. Pixel values are interpolated in a horizontal or a vertical direction based on gradient scores. Interpolation direction selection can be refined based on a chroma scores or morphological operations. A chroma blurring technique to reduce false color artifacts is also disclosed. A disclosed embodiment can be implemented in an imager having as few as three buffer lines.