Answer effectiveness evaluations include providing, by a computing device, an answer to a search query received from a user, and in response to receiving a subsequent search query from the user, determining by the computing device a level of effectiveness of the answer to the search query with respect to the user. The determination includes comparing aspects of the search query to aspects of the subsequent search query, calculating, based on the comparing, a relevance score that indicates a measure of similarity between the aspects of the search query and the aspects of the subsequent search query, and determining that the answer effectively answers the search query when the relevance score exceeds a threshold value.