Methods, systems and computer program products for automated testing of dialog systems are provided herein. A computer-implemented method includes receiving information pertaining to a given conversation workspace of an automated dialog system and identifying test case inputs to the automated dialog system, the test case inputs comprising user input for the given conversation workspace that has portions thereof modified and which the automated dialog system maps to a different intent and/or a different entity relative to the user input. The method further includes generating human-interpretable explanations of mappings of portions of the test case inputs to the different intent and/or entity, generating suggestions for modifying intents, entities and dialog flows of the given conversation workspace such that the test case inputs map to the same intent and/or the same entity as their corresponding user input, and outputting the suggestions and the human-interpretable explanations to a user.