Techniques are described herein for maintaining a directory map that contains a data chunk-to-server mapping. Directory maps, maintained within the volatile memory of each server node in the set of multiple server nodes, include a node hierarchy represented as a tree structure in which each node represents a range of an on-disk address range. Each child node, within the node hierarchy, represents a sub-range of the on-disk address range represented by the parent node of the child node. Once a data chunk is loaded into volatile memory of a particular server, mapping information relating the particular on-disk starting address of the data chunk loaded into volatile memory and the pointer address to the location in volatile memory is loaded into one or more node in the directory map. Loading mapping information into the directory map includes first, locating a target child node, in the node hierarchy that has a range into which the particular on-disk starting address falls, where the target node is the Nth child node of a particular parent node. Then inserting a mapping entry into the target child node. Then determining a set of parent nodes, other than the particular parent node of the target child node, that has ranges that overlap with the particular on-disk address range. Then inserting into each Nth child node of each parent node in the set of parents nodes, an entry corresponding to mapping information that includes an indication of the on-disk starting address and the particular in-memory address.