A system for synthesizing real IVR scenarios, automatically reproducing test scenarios, and providing accurate reports on those scenarios. An IVR tester uses a telephony device (website, mobile app, VOIP, etc.) and an automated IVR test case generator to place test calls to an IVR system. The automated IVR test case generator intercepts and collects the phone call's audio stream, dual-tone multi-frequency signals, and available metadata (e.g., response times, post-speech silence timeout, etc.). Once the call is complete, the automated IVR test case generator pre-populates a webpage or other user-interface technology and a database with the audio stream, dual-tone multi-frequency signals, and available metadata from the test scenario phone call. The IVR tester may then save that test scenario and continue to the next scenario. The saved scenario may now be used automatically by the IVR test case generator for regression testing or duplicated and edited for use in other systems.