公开(公告)号：WO2015105665A1

公开(公告)日：2015-07-16

申请号：PCT/US2014/071465

申请日：2014-12-19

Applicant: NETAPP, INC.

Inventor： SUNDARAM, Rajesh ,  KIMMEL, Jeffrey S. ,  LEWIS, Blake H.

IPC: G06F3/06

CPC classification number: G06F3/0641 ,  G06F3/0604 ,  G06F3/0608 ,  G06F3/0643 ,  G06F3/067 ,  G06F3/0688 ,  G06F2003/0697

Abstract: In one embodiment, a layered file system of a storage input/output (I/O) stack executes on one or more nodes of a cluster. The layered file system includes a flash-optimized, log-structured layer configured to provide sequential storage of data and metadata (i.e., a log-structured layout) on solid state drives (SSDs) of storage arrays in the cluster to reduce write amplification, while leveraging a data de-duplication feature of the storage I/O stack. An extent store layer of the file system performs and maintains mappings of the extent keys to SSD storage locations, while a volume layer of the file system performs and maintains mappings of the LUN offset ranges to the extent keys. Separation of the mapping functions between the volume and extent store layers enables different volumes with different offset ranges to reference a same extent key (and thus a same extent).

Abstract translation: 在一个实施例中，存储输入/输出（I / O）堆栈的分层文件系统在集群的一个或多个节点上执行。 分层文件系统包括闪存优化的日志结构化层，其被配置为在集群中的存储阵列的固态驱动器（SSD）上提供对数据和元数据的顺序存储（即，日志结构化布局），以减少写入放大， 同时利用存储I / O堆栈的重复数据删除功能。 文件系统的扩展存储层执行并维护扩展密钥到SSD存储位置的映射，而文件系统的卷层执行并维护LUN偏移范围到扩展密钥的映射。 卷和卷存储层之间的映射函数的分离使得具有不同偏移范围的不同卷可以引用相同的扩展密钥（并因此相同的扩展）。

公开(公告)号：WO2015105671A1

公开(公告)日：2015-07-16

申请号：PCT/US2014/071581

申请日：2014-12-19

Applicant: NETAPP, INC.

Inventor： KIMMEL, Jeffrey S.

IPC: G06F12/08 ,  G06F3/06

CPC classification number: G06F12/0246 ,  G06F3/0614 ,  G06F3/0655 ,  G06F3/0679 ,  G06F3/0688 ,  G06F11/2094 ,  G06F12/0802 ,  G06F12/0804 ,  G06F12/0815 ,  G06F12/0866 ,  G06F12/0891 ,  G06F12/0893 ,  G06F12/0895 ,  G06F2212/1032 ,  G06F2212/214 ,  G06F2212/221 ,  G06F2212/222 ,  G06F2212/2228

Abstract: Non-volatile random access memory (NVRAM) caching and logging may be configured to deliver low latency acknowledgements of input/output (I/O) requests, such as write requests, while avoiding loss of data associated with the requests that may occur as a result of power failures. Write data associated with one or more write requests may be received at a node of a cluster. The write data may be stored in a portion of an NVRAM configured as, e.g., a persistent write-back cache of the node, while parameters of the request may be stored in another portion of the NVRAM configured as one or more logs, e.g., NVLogs. The write data may be organized into separate variable length blocks or extents and "written back" out-of-order from the write back cache to storage devices, such as solid state drives (SSDs). The write data may be preserved in the write-back cache until each extent is safely and successfully stored on SSD (i.e., in the event of power loss), or operations associated with the write request are sufficiently logged on NVLog, to thereby provide efficient recovery when attempting to restore the write data preserved in the cache to the SSDs.

Abstract translation: 可以将非易失性随机存取存储器（NVRAM）高速缓存和记录配置为提供输入/输出（I / O）请求（例如写入请求）的低延迟确认，同时避免与可能发生的请求相关联的数据丢失 电源故障的结果。 与一个或多个写入请求相关联的写入数据可以在集群的节点处被接收。 写入数据可以存储在被配置为例如节点的持久回写缓存的NVRAM的一部分中，而请求的参数可以存储在被配置为一个或多个日志的NVRAM的另一部分中，例如， NVLogs。 写数据可以被组织成单独的可变长度块或扩展区，并且从写回高速缓存“无回写”到诸如固态驱动器（SSD）的存储设备。 写入数据可以保留在回写缓存中，直到每个区段安全地并且成功地存储在SSD上（即，在电力丢失的情况下），或者与写入请求相关联的操作被充分地记录在NVLog上，从而提供有效的 在尝试将缓存中保留的写入数据恢复到SSD时恢复。

公开(公告)号：WO2015105666A1

公开(公告)日：2015-07-16

申请号：PCT/US2014/071484

申请日：2014-12-19

Applicant: NETAPP, INC.

Inventor： SUNDARAM, Rajesh ,  DANIEL, Stephen ,  KIMMEL, Jeffrey S. ,  LEWIS, Blake H.

IPC: G06F3/06

CPC classification number: G06F17/30138 ,  G06F3/0604 ,  G06F3/0608 ,  G06F3/0616 ,  G06F3/064 ,  G06F3/0643 ,  G06F3/0644 ,  G06F3/0647 ,  G06F3/0659 ,  G06F3/0679 ,  G06F3/0685 ,  G06F3/0688 ,  G06F12/0246 ,  G06F17/30185 ,  G06F17/30218 ,  G06F2003/0697 ,  G06F2212/2022 ,  G06F2212/7208

Abstract: In one embodiment, a flash-optimized, log-structured layer of a file system of a storage input/output (I/O) stack executes on one or more nodes of a cluster. The log-structured layer of the file system provides sequential storage of data and metadata on solid state drives (SSDs) to reduce write amplification, while leveraging variable compression and variable length data features of the storage I/O stack. The data may be organized as an arbitrary number of variable-length extents of one or more host-visible logical units (LUNs). The metadata may include mappings from host-visible logical block address ranges of a LUN to extent keys, as well as mappings of the extent keys to SSD storage locations of the extents. The storage location of an extent on SSD is effectively "virtualized" by its mapped extent key such that relocation of the extent on SSD does not require update to volume layer metadata.

Abstract translation: 在一个实施例中，存储输入/输出（I / O）堆栈的文件系统的闪存优化的日志结构化层在集群的一个或多个节点上执行。 文件系统的日志结构化层在固态驱动器（SSD）上提供数据和元数据的顺序存储，以减少写入放大，同时利用存储I / O堆栈的可变压缩和可变长度数据特征。 数据可以被组织为一个或多个主机可见逻辑单元（LUN）的任意数量的可变长度盘区。 元数据可以包括从LUN到扩展密钥的主机可视逻辑块地址范围的映射，以及扩展密钥到扩展区的SSD存储位置的映射。 SSD上的盘区的存储位置被其映射的盘区密钥有效地“虚拟化”，使得SSD上盘区的重新定位不需要更新到卷层元数据。

公开(公告)号：WO2015108667A1

公开(公告)日：2015-07-23

申请号：PCT/US2014/071446

申请日：2014-12-19

Applicant: NETAPP, INC.

Inventor： KIMMEL, Jeffrey S. ,  RAKITZIS, T. Byron

IPC: G06F3/06

CPC classification number: G06F3/061 ,  G06F3/0608 ,  G06F3/0611 ,  G06F3/0638 ,  G06F3/064 ,  G06F3/0665 ,  G06F3/0688 ,  G06F17/30067 ,  G06F17/30097 ,  G06F2003/0692

Abstract: The embodiments described herein are directed to the use of hashing in a file system metadata arrangement that reduces an amount of metadata stored in a memory of a node in a cluster and that reduces the amount of metadata needed to process an input/output (I/O) request at the node. Illustratively, the embodiments are directed to cuckoo hashing and, in particular, to a manner in which cuckoo hashing may be modified and applied to construct the file system metadata arrangement. In an embodiment, the file system metadata arrangement may be illustratively configured as a key-value extent store embodied as a data structure, e.g., a cuckoo hash table, wherein a value, such as a hash table index, may be configured as an index and applied to the cuckoo hash table to obtain a key, such as an extent key, configured to reference a location of an extent on one or more storage devices, such as solid state drives.

Abstract translation: 这里描述的实施例涉及在文件系统元数据排列中使用散列的方式，其减少了存储在群集中的节点的存储器中的元数据量，并且减少了处理输入/输出所需的元数据量（I / O）请求。 示例性地，实施例针对杜鹃哈希，并且特别地涉及可以修改和应用杜鹃散列以构造文件系统元数据布置的方式。 在一个实施例中，文件系统元数据布置可以被示意性地配置为体现为数据结构的键值区段存储，例如布鲁克哈希表，其中诸如散列表索引的值可以被配置为索引 并且将其应用于布鲁克哈希表以获得诸如扩展密钥的密钥，其被配置为引用诸如固态驱动器之类的一个或多个存储设备上的盘区的位置。

公开(公告)号：WO2015051059A1

公开(公告)日：2015-04-09

申请号：PCT/US2014/058728

申请日：2014-10-02

Applicant: NETAPP, INC.

Inventor： KIMMEL, Jeffrey S. ,  LEWIS, Blake H.

IPC: G06F3/06 ,  G06F17/30

CPC classification number: G06F17/3033 ,  G06F3/0608 ,  G06F3/0641 ,  G06F3/067 ,  G06F17/30097 ,  G06F17/30949

Abstract: In one embodiment, a technique is provided for distributing data and associated metadata within a distributed storage architecture. A set of hash tables that embody mappings of cluster-wide identifiers associated with storage locations are stored for write data of write requests organized into extents. A hash value is generated from a hash function applied to each extent. The hash value is overloaded and used for multiple purposes within the distributed storage architecture, including (i) a remainder computation on the hash value to select a bucket of a plurality of buckets representative of the extents, (ii) a hash table selector of the hash value to select a hash table from the set of hash tables, and (iii) a hash table index computed from the hash value to select an entry from a plurality of entries of the selected hash table having a cluster-wide identifier identifying a storage location for the extent.

Abstract translation: 在一个实施例中，提供了用于在分布式存储架构内分发数据和相关元数据的技术。 存储一组哈希表，其中包含与存储位置相关联的群集范围标识符的映射，用于组织到扩展区中的写入请求的写入数据。 从应用于每个区段的散列函数生成哈希值。 散列值被重载并用于分布式存储体系结构内的多个目的，包括（i）散列值的余数计算，以选择代表扩展区的多个存储桶的桶，（ii）哈希表选择器 散列值以从所述散列表集合中选择散列表，以及（iii）根据所述散列值计算的散列表索引，以从所选择的哈希表的多个条目中选择具有标识存储器的集群范围标识符的标识符的条目 位置的程度。

公开(公告)号：EP3095029A1

公开(公告)日：2016-11-23

申请号：EP14828399.7

申请日：2014-12-19

Applicant: NetApp, Inc.

Inventor： KIMMEL, Jeffrey S. ,  RAKITZIS, T. Byron

IPC: G06F3/06

CPC classification number: G06F3/061 ,  G06F3/0608 ,  G06F3/0611 ,  G06F3/0638 ,  G06F3/064 ,  G06F3/0665 ,  G06F3/0688 ,  G06F17/30067 ,  G06F17/30097 ,  G06F2003/0692

Abstract: In one embodiment, an extent key reconstruction technique is provided for use with a set of hash tables embodying metadata. The metadata includes an extent key associated with a storage location on storage devices for write data of one or more write requests organized into an extent. Each hash table has a plurality of entries, and each entry includes a plurality of slots. A first field of the extent key is recreated implicitly from an entry in a first address space portion of a hash table. A second field of the extent key is stored in the slot. A third field of the extent key is stored in the slot. A fourth field of the extent key is recreated implicitly from the hash table of the set of hash tables.

Abstract translation: 在一个实施方式中，使用在文件系统元数据排列的散列减少对存储在集群中的一个节点的存储器中的元数据的量和减少了在节点处理到输入/输出（I / O）请求所需的元数据的量 ， 说明性地，杜鹃散列可以被修改和应用来构造文件系统元数据的安排。 的文件系统元数据布置可以说明性地配置为体现为数据结构，例如，杜鹃哈希表，worin的值，检查作为哈希表索引键值程度商店，可以被配置为以索引和施加到 杜鹃哈希表来获得密钥，颜色：诸如程度键，配置为在一个或多个存储设备，颜色引用的程度的位置：例如固态驱动器。

公开(公告)号：EP3095029B1

公开(公告)日：2018-07-18

申请号：EP14828399.7

申请日：2014-12-19

Applicant: NetApp, Inc.

Inventor： KIMMEL, Jeffrey S. ,  RAKITZIS, T. Byron

IPC: G06F3/06

CPC classification number: G06F3/061 ,  G06F3/0608 ,  G06F3/0611 ,  G06F3/0638 ,  G06F3/064 ,  G06F3/0665 ,  G06F3/0688 ,  G06F17/30067 ,  G06F17/30097 ,  G06F2003/0692

Abstract: In one embodiment, an extent key reconstruction technique is provided for use with a set of hash tables embodying metadata. The metadata includes an extent key associated with a storage location on storage devices for write data of one or more write requests organized into an extent. Each hash table has a plurality of entries, and each entry includes a plurality of slots. A first field of the extent key is recreated implicitly from an entry in a first address space portion of a hash table. A second field of the extent key is stored in the slot. A third field of the extent key is stored in the slot. A fourth field of the extent key is recreated implicitly from the hash table of the set of hash tables.

公开(公告)号：EP3186727A1

公开(公告)日：2017-07-05

申请号：EP15751194.0

申请日：2015-07-31

Applicant: NetApp, Inc.

Inventor： KIMMEL, Jeffrey S. ,  COATNEY, Susan M. ,  MU, Yuedong ,  RAO, Santosh

IPC: G06F17/30 ,  H04L29/08

CPC classification number: H04L67/1095 ,  G06F3/0619 ,  G06F3/065 ,  G06F3/067 ,  G06F17/30215 ,  H04L67/1097

Abstract: Data consistency and availability can be provided at the granularity of logical storage objects in storage solutions that use storage virtualization in clustered storage environments. To ensure consistency of data across different storage elements, synchronization is performed across the different storage elements. Changes to data are synchronized across storage elements in different clusters by propagating the changes from a primary logical storage object to a secondary logical storage object. To satisfy the strictest RPOs while maintaining performance, change requests are intercepted prior to being sent to a filesystem that hosts the primary logical storage object and propagated to a different managing storage element associated with the secondary logical storage object.

Abstract translation: 数据一致性和可用性可以在使用集群存储环境中的存储虚拟化的存储解决方案中以逻辑存储对象的粒度来提供。 为确保跨不同存储元素的数据一致性，跨不同存储元素执行同步。 通过将更改从主逻辑存储对象传播到辅助逻辑存储对象，跨不同群集中的存储元素对数据更改进行同步。 为了在保持性能的同时满足最严格的RPO，变更请求在被发送到托管主逻辑存储对象并传播到与次逻辑存储对象相关联的不同管理存储元件的文件系统之前被拦截。

公开(公告)号：EP3036617A1

公开(公告)日：2016-06-29

申请号：EP14796316.9

申请日：2014-10-02

Applicant: NetApp, Inc.

Inventor： KIMMEL, Jeffrey S. ,  LEWIS, Blake H.

IPC: G06F3/06 ,  G06F17/30

CPC classification number: G06F17/3033 ,  G06F3/0608 ,  G06F3/0641 ,  G06F3/067 ,  G06F17/30097 ,  G06F17/30949

Abstract: In one embodiment, a technique is provided for distributing data and associated metadata within a distributed storage architecture. A set of hash tables that embody mappings of cluster-wide identifiers associated with storage locations are stored for write data of write requests organized into extents. A hash value is generated from a hash function applied to each extent. The hash value is overloaded and used for multiple purposes within the distributed storage architecture, including (i) a remainder computation on the hash value to select a bucket of a plurality of buckets representative of the extents, (ii) a hash table selector of the hash value to select a hash table from the set of hash tables, and (iii) a hash table index computed from the hash value to select an entry from a plurality of entries of the selected hash table having a cluster-wide identifier identifying a storage location for the extent.