Techniques are described herein for implementing a testing and evaluation pipeline. The techniques include receiving testing specifications for validating an XR application executing on XR capable devices and mapping individual testing specifications to a corresponding XR capable device including the XR application. Upon mapping the individual testing specifications, testing configurations for an evaluation pipeline is determined. The evaluation pipeline may include one or more computing instances that execute one or more validation tests for the XR application executing on the corresponding XR capable device according to the individual testing specifications and the testing configurations. The one or more computing instances may operate in parallel to perform the one or more validation tests concurrently. Based at least on test results generated from the one or more computing instances and one or more evaluation criteria, the XR application executing on the corresponding XR capable device may be validated.