ERASURE CODING CONTENT DRIVEN DISTRIBUTION OF DATA BLOCKS

    公开(公告)号:WO2020081491A1

    公开(公告)日:2020-04-23

    申请号:PCT/US2019/056200

    申请日:2019-10-15

    Applicant: NETAPP, INC.

    Abstract: A technique is configured to provide data protection, such as replication and erasure coding, of content driven distribution of data blocks served by storage nodes of a cluster. When providing data protection in the form of replication (redundancy), a slice service of the storage node generates one or more copies or replicas of a data block for storage on the cluster. Each replicated data block is illustratively organized within a bin that is maintained by block services of the nodes for storage on storage devices. When providing data protection in the form of erasure coding, the block services may select data blocks to be erasure coded. A set of data blocks for erasure coding may then be grouped together to form a write group. According to the technique, EC group membership is guided by varying bin groups so the data is resilient against failure. Slice services of the storage nodes assign data blocks of different bins and replicas to a write group.

    EFFICIENT MEMORY FOOTPRINT IN DEDUPLICATED SYSTEM STORING WITH CONTENT BASED ADDRESSING

    公开(公告)号:WO2020061398A1

    公开(公告)日:2020-03-26

    申请号:PCT/US2019/052061

    申请日:2019-09-20

    Applicant: NETAPP, INC.

    Abstract: A technique is configured to reduce an amount of memory (i.e., memory footprint) usage by each storage node of a cluster needed to store metadata while providing fast and efficient servicing of data in accordance with storage requests issued by a client of the cluster. Illustratively, a block identifier (ID) is used to identify a block of data serviced by the storage node. Metadata embodied as mappings between block IDs and locations of data blocks in the cluster are illustratively maintained in map fragments. A map fragment may be embodied as "active" map fragment or a "frozen" map fragment. An active map fragment refers to a map fragment that has space available to store a mapping, whereas a frozen map fragment refers to a map fragment that is full, i.e., has no available space for storing a mapping. In order to reduce the memory footprint of each storage node, yet still provide fast and efficient servicing of data by the node, the active map fragments are preferably maintained in memory as "in-core" data structures, whereas the frozen map fragments are paged-out and stored on storage devices of the cluster as "on-disk" map fragment structures.

Patent Agency Ranking