A document information extraction system determines a structure of an electronic document based on characteristics of the document's constituent elements. The system segments the document to generate elements with each element having similar characteristics. Elements may be clustered to assist in determining the document structure. The system determines directional relationships between elements (e.g., above, below, etc.). The system then employs a master comparator to determine familial relationships between adjacent elements. The master comparator includes a set of unit comparators and each unit comparator compares a specific characteristic between two elements. The master comparator sequentially applies the unit comparators to determine the familial relationship based on the comparisons. The system outputs a document hierarchy tree reflecting the determined familial relationships. The hierarchy tree represents the structure of the document.