Method and system are provided for automated system testing in a complex software environment. The method includes: collecting data during a test run of a product under test that involves multiple systems and resources where a test run introduces one or more product code change sets, the collected data including a product trace of the execution of the product under test at a product code level; and storing the last good collected data. Based on the test failing, comparing a current product trace with a last good product trace; determining a difference between the current and last good traces and identifying a failing product program referenced in the product trace; looking through the introduced product code change sets to determine if a code change is found for the failing program indicating that the error is likely to be in the code change set, and generating an appropriate defect prompt.