A method of operating a neural network device configured to perform a neural network operation on successively input image frames includes generating, by a processing circuit, a second delta feature map by performing a linear operation on a first delta feature map generated based on a difference between a current image frame and a previous image frame; loading feature values as a second previous feature map onto the processing circuit from at least one memory, the loaded feature values being feature values corresponding to a first partial region to be updated in a first feature map stored in the at least one memory; generating, at the processing circuit, a second current feature map based on the second delta feature map and the second previous feature map; and updating the first feature map by storing the second current feature map in the at least one memory.