Systems and methods for scenario marker infrastructure are disclosed. In one embodiment, a method comprises receiving driving scenario data, extracting simulated sensor data from the driving scenario data, extracting scenario markers from the driving scenario data, inputting the simulated sensor data into autonomous driving software, receiving driving instructions output by the driving software, updating a driving status of the autonomous vehicle based on the driving instructions output by the autonomous driving software, creating a simulation log comprising the simulated sensor data and the driving instructions, and inserting the scenario markers into the simulation log.