A system and method for constructing a multi-scale voxel representation of a three-dimensional (3D) sparse point map. The method includes: providing the 3D sparse point map; receiving coordinates of a 3D point from the 3D sparse point map; converting the coordinates of the 3D point to integer coordinates; and when the integer coordinates are not encompassed by any voxel entries of a hash table: creating a new unit voxel at scale level 0, scale level 0 being a minimum voxel size; calculating a hash value using the integer coordinates; inserting the new unit voxel into the hash table; and merging the new unit voxel with neighboring voxels along at least one axis of the integer coordinates, so as to obtain the multi-scale voxel representation of the 3D sparse point map.