A yield determination system for automatically collecting, determining, and labeling yield behaviors of vehicles during cooperative driving scenarios. The system includes sensors for detecting the start and stop of the scenario, a data recorder for automatically collecting the data, an annotation unit for automatically labeling features of interest about the vehicle and surrounding vehicles during the scenario. The labeled file may be automatically uploaded and processed to insert the labelled features into a learning model to predict vehicle behavior in cooperative driving scenarios.