The disclosed embodiments provide a system that facilitates the development and testing of a software program. During operation, the system receives and stores a plurality of defect reports associated with a software program, wherein each defect report documents a previously discovered defect in the software program. In response to detecting a given test failure of the software program while testing the software program in a test environment, the system then eliminates, based on attributes of the test environment that are selected by the defect reports, one or more of the defect reports from the plurality of defect reports, wherein at least one of the selected attributes is retrieved from the test environment during the determination by invoking a provider script that is specified by one of the defect reports. Next, the system associates the test failure with a defect report that remains after the one or more eliminations.