Certain aspects relate to systems and techniques for correction of color artifacts including both color aberration and color spot artifacts using a unified framework. For example, both color aberration and color spot artifacts can be corrected using a post-processing method implementing directional median filtering on chroma channels. A pixel-by-pixel correction ratio map can be built by analyzing the luma and chroma components of the image to indicate a type of color artifact associated with each pixel in the image, and a directional median filter can be selected for each pixel based on the corresponding correction ratio map value.