An object detection algorithm is selectively applied to frames in a video. A frame in the video is analyzed using a set of neural networks of the object detection algorithm to detect a location of an object in the frame and predict a bounding box for the object in the frame. A magnitude of a delta between the frame and a second frame is determined. The magnitude of the delta is determined based on a difference between the two frames in values of at least one parameter of their respective sets of pixels. Responsive to the magnitude of the delta being less than a threshold, a new bounding box is predicted for the object in the second frame without analyzing the second frame using the set of neural networks.