Abstract:
Transportable storage devices are communicatively coupled behind one or more dedicated “edge” data servers to take advantage of the benefits of the data servers and transportable storage devices while limiting their disadvantages. Each edge data server ingests data from one or more client devices and copies the data to one or more of the transportable storage devices for eventual transport to a cloud data center for upload to a cloud storage system. Multiple versions of a segment of data (e.g., of a file or a blob) are maintained at various points in the disclosed system and the disclosed technology ensures that the most recent version is the version ultimately stored on the cloud storage system.
Abstract:
Embodiments are directed to communicating between computing nodes in a cluster of nodes. In one scenario, a computer system receives a data packet from a worker node including the worker node's current workload identifiers and health status, where the data packet includes an associated version number. The computer system determines that the version number in the received data packet is different than a previously received data packet and evaluates the worker node's current workload configuration to determine whether workload changes are to be made on the worker node. Then, upon determining that workload changes are to be made on the worker node, the computer system selects a subset of workload changes to apply to the worker node, generates an indication of the selected subset of workload changes to the worker node and sends the generated indication of workload changes to the worker node.
Abstract:
Embodiments provide workload processing for clustered systems. In an illustrative, non-limiting embodiment, a computer-implemented method may include identifying a server as an active node of a cluster; assigning a workload to the server in response to the identification; determining, after the assignment, that the server is no longer an active node of the cluster; calculating, in response to the determination, a probability that the server is capable of continuing to execute the workload; and deciding, based upon the probability, whether to allow the workload to remain assigned to the server.
Abstract:
Embodiments are directed to creating global, aggregated namespaces for storage management and to providing consistent namespaces in a distributed storage system. In one scenario, a computer system defines data storage objects for each data storage node. The data storage objects uniquely identify storage elements of the data storage nodes, where each data storage object includes various associated attributes. The computer system replicates the defined data storage objects and any associated attributes from a first data storage node to a second, different data storage node among the data storage nodes. As such, the defined data storage objects are visible from any node in the data storage nodes. The computer system also aggregates the defined data storage objects for each of the data storage nodes and creates a global, aggregated namespace that includes the aggregated data storage objects for each of the data storage nodes.
Abstract:
Recovery requests are scheduled and prioritized according to priority valuations of the minimum time to next failure relative to the minimum time to next recovery for corresponding storage data sets. The prioritization can be performed by an upper layer recovery scheduler that dispatches requests to different storage units and/or by lower layers and individual storage unit schedulers. Prioritizations can be reflected in tagging on recovery requests, as determined by a first entity, and/or determined dynamically at the point of dispatch and processing.
Abstract:
Embodiments are directed to organizing computing nodes in a cluster federation and to reassigning roles in a cluster federation. In one scenario, a computer system identifies computing nodes that are to be part of a cluster federation which includes a master cluster and worker clusters. The computer system assigns a director role to a master node in the master cluster which governs decisions that affect consistency within the federation, and further assigns a leader role to at least one master node which monitors and controls other master nodes in the master cluster. The computer system assigns a worker agent role to a worker node which receives workload assignments from the master cluster, and further assigns a worker role to a worker node which processes the assigned workload. The organized cluster federation provides fault tolerance by allowing roles to be dynamically reassigned to computing nodes in different master and worker clusters.
Abstract:
A system is configured to use a de-randomizer and budget data structure to economize I/O operations for a shared storage device while still allowing access to the device to a number of different entities. Embodiments can identify a comparatively low cost next operation as compared to other I/O operations, including a cost for seek time, for a first entity to dispatch to the storage device when the first entity has sufficient budget to have the I/O operation performed on its behalf and to identify an I/O operation for a second entity to dispatch to the storage device when there is insufficient budget for the first entity.
Abstract:
The targeted recovery of application-specific data corresponding to an application without performing recovery of the entire volume. The recovery is initiated by beginning to copy the prior state of the content of an application-specific data container from a prior snapshot to the application-specific data container in an operation volume accessible by the application. However, while the content of the application-specific data container is still being copied from the snapshot to the application-specific data container, the application is still permitted to perform read and write operations on the application-specific data container. Thus, the application-specific data container appears to the application to be fully accessible even though recovery of the content of the application-specific data container is still continuing in the background.
Abstract:
Transportable storage devices are communicatively coupled behind one or more dedicated “edge” data servers to take advantage of the benefits of the data servers and transportable storage devices while limiting their disadvantages. Each edge data server ingests data from one or more client devices and copies the data to one or more of the transportable storage devices for eventual transport to a cloud data center for upload to a cloud storage system. Multiple versions of a segment of data (e.g., of a file or a blob) are maintained at various points in the disclosed system and the disclosed technology ensures that the most recent version is the version ultimately stored on the cloud storage system.
Abstract:
Services from domainless machines are made available in a security domain under a virtual name. Each machine is not joined to the domain but can reach a security domain controller. The controller controls at least one security domain using an authentication protocol, such as a modified Kerberos protocol. One obtains a set of security domain credentials, generates a cluster name secret, gives the cluster a virtual name, and authenticates the machines to the domain controller using these items. In some cases, authentication uses a ticket-based protocol which accepts the cluster name secret in place of a proof of valid security domain membership. In some, the domain controller uses a directory service which is compatible with an active directory service; the cluster virtual name is provisioned as an account in the directory service. The cluster virtual name may concurrently serve clients on different security domains of the directory service.