A mechanism is provided in a data processing system for identifying nonsense passages. An annotator in a natural language processing pipeline configured to execute in the data processing system annotates an input passage in a corpus with linguistic features to form an annotated passage. A domain-specific policy is associated with a domain of the corpus. A metric counters component in the natural language processing pipeline counts a number of instances of each type of linguistic feature in the annotated passage to form a set of feature counts. The metric counters component of the natural language processing pipeline determines a value for a metric based on the set of feature counts. The metric is specified in the domain-specific policy. A comparator component of the natural language processing pipeline compares the value for the metric to a predetermined model threshold. The threshold is specified in the domain-specific policy. A filter component of the natural language processing pipeline identifies whether the input passage is a nonsense passage based on a result of the comparison.