A method and apparatus for generating adversarial scenarios and training an autonomous driving agent for an autonomous vehicle, using one or more sets of parameters, each set of parameters defining a respective driving scenario. A new set of parameters is generated by changing one or more parameters of one of the sets of parameters to define a new driving scenario, and performance of the autonomous driving agent is evaluated on the new driving scenario. The generating and evaluating is repeated until the autonomous driving agent fails to satisfy a predefined performance threshold for the new driving scenario. Each instance of changing the one or more parameters is based on a prior evaluated performance of the autonomous driving agent. The autonomous driving agent is trained to update a learned policy of the autonomous driving agent using at least one set of parameters, including the new set of parameters.