Disclosed is a method and system for dynamically persisting log data from member computers in a network. The log data is transmitted as a plurality of shards to a router of router-writer pairs which transmits the shards to writers of the router-writer pairs that persist the shards into a storage unit. A shard contains a distinct subset of the log data. The shards are transmitted to the writers based on a writer assignment map that identifies a writer assigned to persist each of the shards. The assignment of the shards to a writer is performed based on assignment criteria that include at least one of (i) number of writers, (ii) a load of the writers, (iii) number of shards, (iv) size of a particular shard, or (v) estimated load capacity of writers.