In one embodiment of the present invention, storing a plurality of key value pairs may be accomplished by first, for each of two or more quantities of most significant bits, determining how much overall memory usage will be saved upon removal of the corresponding quantity of most significant bits from each key in the plurality of key value pairs. Then, for the quantity of most significant bits determined to have the most overall memory usage savings, the quantity of most significant bits may be removed from each key in the plurality of key value pairs. Then a first auxiliary data structure may be formed, wherein the first auxiliary data structure contains the removed quantity of most significant bits from each key in the plurality of key value pairs and pointers to the remaining bits of each key of the plurality of keys in a primary data structure.