Techniques are provided for implementing asynchronous checkpointing of in-memory data in a distributed computing system. For example, a method includes processing a stream of data records by an operator executing on a computing node, maintaining in a system memory, an operator state which is generated in response to the operator processing the stream of data records, and performing an asynchronous checkpointing process. The asynchronous checkpointing process includes enqueuing a checkpoint of the operator state in a first queue, wherein the first queue is maintained in the system memory, and executing a background worker thread to dequeue the checkpoint of the operator state from the first queue and store the checkpoint of the operator state in a data store. The operator continues with processing the stream of data records during the asynchronous checkpointing process.