An autonomous vehicle simulation system for analyzing motion planners is disclosed. A particular embodiment includes: receiving map data corresponding to a real world driving environment; obtaining perception data and configuration data including pre-defined parameters and executables defining a specific driving behavior for each of a plurality of simulated dynamic vehicles; generating simulated perception data for each of the plurality of simulated dynamic vehicles based on the map data, the perception data, and the configuration data; receiving vehicle control messages from an autonomous vehicle control system; and simulating the operation and behavior of areal world autonomous vehicle based on the vehicle control messages received from the autonomous vehicle control system.