Disclosed herein are systems and methods for classifying traffic indicators to detect intersection crossings by a vehicle. A computing device can receive a sequence of frames captured by a capture device mounted to the vehicle, and generate an intersection status data structure for each frame in the sequence of frames using a full-frame classification model. The computing device can also classify a set of features detected in each frame using an object detection model, and detect an intersection crossing event based on the intersection status data structure of each frame and the classification of each of the set of features detected in each frame. The systems and methods described herein improve upon existing traffic light classification models by using a full-frame intersection classification model in connection with traditional object tracking techniques, thereby reducing false-positive crossing-event detection and improving the accuracy of classification in a variety of environmental conditions.