Embodiments are directed to managing documents where clauses in a document may be identified. Evaluations of the clauses may be provided based on evaluators and machine learning (ML) models that assign each of the clauses to a category and a confidence score. Actions associated with the clauses may be monitored including updates to content of the clauses. Inconsistent evaluations associated with the clauses be identified. The ML models may be retrained based on the content of the clauses associated with the inconsistent evaluations.