A buffered scene-based non-uniformity correction method includes receiving a plurality of frames of video image data from an image detector; determining relative movement of a current image frame with respect to a previous image frame and responsive to a determination of substantial movement, adding the current image frame to a buffer memory sized to store a predetermined number of video frames; averaging pixel values of the frames in the buffer to determine a mean (or weighted mean) value for each pixel of a reference image; determining correction terms for each pixel of the current image frame by determining the difference between the current image frame pixel values and the corresponding reference image pixels; and correcting the current image frame using the correction terms. A scene-based non-uniformity correction system is also disclosed.