One embodiment of the present invention provides a system for facilitating a distributed storage system. The system receives, by a first client-serving machine, a first request to write data. The system writes the data to a first persistent cache associated with the first client-serving machine, wherein a persistent cache includes non-volatile memory. The system records, in an entry in a global data structure, a status for the data prior to completing a write operation for the data in a storage server, wherein the status indicates that the data has been stored in the first persistent cache but has not yet been stored in the storage server.