A system and method for performing integral histogram convolution for filtering image data is disclosed. The method may include generating an initial histogram for a kernel region of an input image comprising two or more column histograms. Each histogram may include a plurality of bucket values, each representing a sum of weights associated with pixels having the same value or a count of pixels having the same value. The method may include incrementally generating a region histogram for each successive region of the image as the kernel is passed over the image and applying a convolution operation to it. Generating the new region histogram may involve merging an additional column histogram with the initial or current region histogram, and may also involve subtracting values of a preceding column histogram from the initial or current histogram. The methods may be implemented by program instructions executing in parallel on CPU(s) or GPUs.