Embodiments are directed to a watched questions threshold filtering system that functions to determine and deliver to a user relevant and significant data changes with respect to a user's goals, as defined by a notification threshold value provided by the user. The user is provided with an option to flag one or more queries for automatic re-querying. Confidence scores are processed on new data (i.e., data ingested after the original question was asked) by utilizing a confidence threshold for indicating if the new data warrants alerting a user.