Persisting distributed data sets into eventually consistent storage systems
Abstract:
Embodiments of the present systems and methods may provide the capability ensure that data is persisted and accessed correctly without depending on eventually consistent list operations on the object store. For example, in an embodiment, a computer-implemented method for data distribution may comprise attempting to persist a plurality of data parts from a plurality of processing tasks, generating a manifest including information indicating those attempts to persist data parts that have succeeded, and persisting the manifest with the data parts that have been successfully persisted.
Information query
Patent Agency Ranking
0/0