A domain specific language for use in constructing simulations within real environments is described. In an example, a computing device associated with a vehicle can receive, from one or more sensors associated with the vehicle, sensor data associated with an environment within which the vehicle is positioned. In an example, the vehicle can be an autonomous vehicle. The computing device associated with the vehicle can receive simulated data associated with one or more primitives that are to be instantiated as a scenario in the environment. The computing device can merge the sensor data and the simulated data to generate aggregated data and determine a trajectory along which the vehicle is to drive based at least in part on the aggregated data. The computing device can determine instructions for executing the trajectory and can assess the performance of the vehicle based on how the vehicle responds to the scenario.