A system and method for determining object-wise situational awareness that includes receiving data associated with a driving scene of a vehicle, an eye gaze of a driver of the vehicle, and alerts that are provided to the driver of the vehicle. The system and method also includes analyzing the data and extracting features associated with dynamic objects located within the driving scene, the eye gaze of the driver of the vehicle, and the alerts provided to the driver of the vehicle. The system and method additionally includes determining a level of situational awareness of the driver with respect to the each of the dynamic objects based on the features. The system and method further includes communicating control signals to electronically control at least one component of the vehicle based on the situational awareness of the driver.