Abstract:
A method begins by a dispersed storage (DS) processing module determining to remove data object index entries from a data object level index node, identifying data object index entries to extract, and creating a temporary sibling data object level index node to include the identified data object index entries. The method continues with the DS processing module initiating updating of an hierarchical ordered index structure by identifying an address for storing the temporary sibling data object level index node, setting up linking the temporary sibling data object level index node to a next level node and to an adjacent data object level index node, and determining whether a change has occurred. When the change has not occurred, the method continues with the DS processing module removing the data object index entries from the data object level index node and commencing the updating of the hierarchical ordered index structure.
Abstract:
A method begins with a computing device of a dispersed storage network (DSN) determining that an encoded data slice of a set of encoded data slices requires rebuilding and sending partial rebuild requests to storage units of the DSN. The method continues with one of the storage units generating a partial rebuilt slice based one or more encoded data slices of the set of encoded data slices stored by the one of the storage units and securing the partial rebuilt slice using a shared secret scheme that is shared among the storage units to produce a secured partial rebuilt slice. The method continues with the computing device receiving a set of secured partial rebuilt slices from the storage units, recovering a set of partial rebuilt slices from the set of secured partial rebuilt slices, and rebuilding the encoded data slice from the set of partial rebuilt slices.
Abstract:
A dispersed storage (DS) method begins by issuing a plurality of write commands to a plurality of DS storage units. The method continues by receiving a write acknowledgement from one of the plurality of DS storage units to produce a received write acknowledgement. The method continues by issuing a plurality of commit commands to the plurality of DS storage units when a write threshold number of the received write acknowledgements have been received. The method continues by receiving a commit acknowledgement from a DS storage unit of the plurality of DS storage units to produce a received commit acknowledgement. The method continues by issuing a plurality of finalize commands to the plurality of DS storage units when a write threshold number of the received commit acknowledgements have been received.
Abstract:
A method begins by a dispersed storage (DS) processing module receiving data for storage and generating a dispersed storage network (DSN) source name for the data. The method continues with the DS processing module determining whether substantially identical data to the data has been previously stored in memory of the DSN. When the substantially identical data has been previously stored in the memory of the DSN, the method continues with the DS processing module generating an object linking file that links the data to the substantially identical data, dispersed storage error encoding the object linking file to produce a set of encoded link file slices, and outputting the set of encoded link file slices for storage in the memory of the DSN.
Abstract:
A method begins by a computing device of a dispersed storage network (DSN) determining whether a DSN access request involves an iterative storage unit access process. When the DSN access request involves the iterative storage unit access process, the method continues with the computing device determining, based on configuration of a plurality of storage units storing data objects associated with the DSN access request, that a storage unit of the plurality of storage units is capable of executing at least a portion of the iterative storage unit access process better than the computing device, sending the DSN access request and a control command to the storage unit, where the control command instructs the storage unit to perform at least a portion of the iterative storage unit access process, and receiving, from the storage unit, at least a partial response to the DSN access request.
Abstract:
A dispersed storage system includes a plurality of storage units that each include a partial rebuild grid module. The partial rebuild grid module includes partial rebuilding functionality to reconstruct one of a plurality of encoded data slices wherein the plurality of encoded data slices are generated from a data segment based on an error encoding dispersal function. In the partial rebuilding process, a data slice is rebuilt by combining in any order slice partials generated from at least a threshold number T of the plurality of data slices.
Abstract:
A method for writing a set of encoded data slices to memory of a dispersed storage network (DSN) begins by a processing module identifying an encoded data slice of the set of encoded data slices for a redundant write operation to produce an identified encoded data slice. The method continues with the processing module generating a set of first write requests regarding the set of encoded data slices less the identified encoded data slice and generating a set of second write requests regarding the identified encoded data slice. The method continues with the processing module sending the set of first write requests to storage units of the DSN and sending the set of second write requests to a set of storage units of the DSN, where each storage unit of the set of storage units is sent a corresponding one of the set of second write requests.
Abstract:
A method begins with storage units receiving a request to transfer a copy of a set of encoded data slices from two or more source virtual storage vaults to a destination virtual storage vault. The method continues with each storage unit obtaining a slice transfer map and determining whether the storage unit supports one of the two or more source virtual storage vaults. For each supporting storage unit, the method continues with each supporting storage unit determining, based on the slice transfer map, a sub-set of encoded data slices of the set of encoded data slices that is stored within the one of the two or more source virtual storage vaults by the supporting storage unit. The method continues with the supporting storage unit sending the sub-set of encoded data slices to a corresponding storage unit that is supporting the destination virtual storage vault.
Abstract:
A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.
Abstract:
A method begins by a dispersed storage (DS) processing module transmitting a set of write commands for storing a set of encoded data slices in storage units of a dispersed storage network (DSN) and determining whether at least a first threshold number of write responses have been received within a response time period. When the at least the first threshold number of the write responses have been received within the response time period, the method continues with the DS processing module determining whether a total number of responses have been received within another response time period. When the total number of responses have not been received within the other response time period, the method continues with the DS processing module issuing a sub-set of write commit commands corresponding to a response number of encoded data slices for which a response was received.