Techniques are described for extracting structured information from unstructured documents based on validation of the structured information as applied to a domain problem associated with the unstructured documents. In one embodiment, a computer program product for automated information extraction is provided. The computer program product comprising a computer readable storage medium having program executable by a processing component to cause the processing component to extract structured candidate interpretations of a rule from unstructured information that defines a plurality of rules intended to control operations of a system, and determine measures of validity of the structured candidate interpretations based on application of the candidate policy interpretations to historical operational data for the system that represents operations performed by the system. The measures of validity respectively can represent degrees to which the historical operational data reflects operation of the system in accordance with the structured candidate interpretations.