Described is a system for multispectral image processing with spiking dynamics. For example, the system receives an input image and compresses the image through space and spectrally variant sampling. Center-surround dynamics are modeled to control high dynamic ranges of the image and provide gain control. Further, habituative dynamics are modeled to produce outputs specialized for static or dynamic image content. Finally, neural spikes are generated based on the habituative dynamics. The neural spikes are saved or provided to other systems for further image processing.