An image sensor may include an array of image pixels arranged in rows and columns. The columns of pixels are coupled to corresponding delta sigma modulators. Each group of delta sigma modulators may be coupled to a column memory circuit. The column memory circuit may receive bits serially from each pixel column in the group. Once all bits in a bit stream from each pixel column have been stored into the column memory circuit, the column memory circuit may output one bit stream at a time to a shared filter circuit. The shared filter circuit may process an entire bit stream associated with a given column in one cycle. Sharing the filter circuit among multiple pixel columns can dramatically reduce circuit area for the image sensor.