A system and a method provide in memory two level of indices to a data set, respectively referred to as an instance layer and an entry layer. The instance layer is brought from disk into memory when an application is initialized to allow insertion of new data into the data set. A temporary data structure augments the instance layer and is provided to keep track of the new data. When query processing is required, the entry layer is brought in, so that query processing may be achieved using both the instance layer indices and the entry layer indices. The temporary data is merged into the instance and entry layers and the data set is updated upon termination of the application or at predetermined times, whereupon the resulting data set and index layers are written out to disk.