Methods and apparatus, including computer program products, implementing and using techniques for identifying candidate answer gaps within a corpus of a question and answer system. An original question posed to the question and answer system is analyzed to identify an object and a semantic type for the question. Concepts having a same or similar semantic type are retrieved from an ontology or dictionary. For at least one retrieved concept, one or more altered questions are created by replacing the object of the original question with a preferred term of the retrieved concept. The one or more altered questions are submitted to the question and answer system. The answers to the altered questions are analyzed to identify gaps within the corpus of the question and answer system.