Upon receiving instructions from a release orchestrator to validate a second application version prior to a transition to sending production traffic to the second application version instead of a first application version, a determination of an application version identifier for a second application version is performed. Tests are selected based on the application version identifier and version rules. The tests are performed by sending test traffic to the second application version via a routing engine while the routing engine routes production traffic to the first application version. Based on the responses to the test traffic, a determination that any failure rules for any of the tests that failed indicate that on a failure the transition should not occur is performed. Responsive to that determination, a communication to the release orchestrator that the second application version is not validated for production traffic and the transition should not occur is performed.