Abstract:
Embodiments of the present disclosure relate to a data analysis system that may automatically generate memory-efficient clustered data structures, automatically analyze those clustered data structures, and provide results of the automated analysis in an optimized way to an analyst. The automated analysis of the clustered data structures (also referred to herein as data clusters) may include an automated application of various criteria or rules so as to generate a compact, human-readable analysis of the data clusters. The human-readable analyses (also referred to herein as “summaries” or “conclusions”) of the data clusters may be organized into an interactive user interface so as to enable an analyst to quickly navigate among information associated with various data clusters and efficiently evaluate those data clusters in the context of, for example, a fraud investigation. Embodiments of the present disclosure also relate to automated scoring of the clustered data structures.
Abstract:
In various embodiments, systems, methods, and techniques are disclosed for generating a collection of clusters of related data from a seed. Seeds may be generated based on seed generation strategies or rules. Clusters may be generated by, for example, retrieving a seed, adding the seed to a first cluster, retrieving a clustering strategy or rules, and adding related data and/or data entities to the cluster based on the clustering strategy. Various cluster scores may be generated based on attributes of data in a given cluster. Further, cluster metascores may be generated based on various cluster scores associated with a cluster. Clusters may be ranked based on cluster metascores. Various embodiments may enable an analyst to discover various insights related to data clusters, and may be applicable to various tasks including, for example, tax fraud detection, beaconing malware detection, malware user-agent detection, and/or activity trend detection, among various others.
Abstract:
In various embodiments, systems, methods, and techniques are disclosed for generating a collection of clusters of related data from a seed. Seeds may be generated based on seed generation strategies or rules. Clusters may be generated by, for example, retrieving a seed, adding the seed to a first cluster, retrieving a clustering strategy or rules, and adding related data and/or data entities to the cluster based on the clustering strategy. Various cluster scores may be generated based on attributes of data in a given cluster. Further, cluster metascores may be generated based on various cluster scores associated with a cluster. Clusters may be ranked based on cluster metascores. Various embodiments may enable an analyst to discover various insights related to data clusters, and may be applicable to various tasks including, for example, tax fraud detection, beaconing malware detection, malware user-agent detection, and/or activity trend detection, among various others.
Abstract:
A history preserving data pipeline computer system and method. In one aspect, the history preserving data pipeline system provides immutable and versioned datasets. Because datasets are immutable and versioned, the system makes it possible to determine the data in a dataset at a point in time in the past, even if that data is no longer in the current version of the dataset.
Abstract:
Embodiments of the present disclosure relate to a data analysis system that may automatically generate memory-efficient clustered data structures, automatically analyze those clustered data structures, and provide results of the automated analysis in an optimized way to an analyst. The automated analysis of the clustered data structures (also referred to herein as data clusters) may include an automated application of various criteria or rules so as to generate a compact, human-readable analysis of the data clusters. The human-readable analyses (also referred to herein as “summaries” or “conclusions”) of the data clusters may be organized into an interactive user interface so as to enable an analyst to quickly navigate among information associated with various data clusters and efficiently evaluate those data clusters in the context of, for example, a fraud investigation. Embodiments of the present disclosure also relate to automated scoring of the clustered data structures.
Abstract:
A fair scheduling system with methodology for fairly scheduling queries for execution by a database management system. The system obtains query jobs for execution by the database management system and cost estimates to execute the query jobs. The cost estimate can be a number of results the query is expected to return. Based on the cost estimates, the system causes the database management system to execute the query jobs as separately sub-query tasks in a round-robin fashion. By doing so, the execution latency of “low cost” query jobs that return few results is reduced when the query jobs are concurrently executed with “high cost” query jobs that return a large number of results.
Abstract:
Techniques are disclosed for generating a collection of clusters of related data from a seed. Doing so may generally include retrieving a seed and adding the seed to a first cluster and include retrieving a cluster strategy referencing one or more data bindings. Each data binding specifies a search protocol for retrieving data. For each of the one or more data bindings, data parameters input to the search protocol are identified, the search protocol is performed using the identified data parameters, and data returned by the search protocol is evaluated for inclusion in the first cluster.
Abstract:
Systems and methods including a framework for migration of live data. The method may comprised, by one or more hardware processors executing program instructions, receiving, at a migration proxy of the framework, code for reading data and writing data compatible with each of a plurality of states of a migration of data in a data store, wherein a service is at least intermittently reading data from and writing data to the data store; determining, by a migration runner of the framework, to perform the migration of the data; initiating, by the migration runner, the migration of the data, wherein the migration comprises a plurality of stages; storing, as the migration progresses through the plurality of stages, and at a migration data store of the framework, a current stage of the migration; and during the migration, using the migration proxy to read data from and write data to the data store.
Abstract:
Systems and methods including a framework for migration of live data. The method may comprised, by one or more hardware processors executing program instructions, receiving, at a migration proxy of the framework, code for reading data and writing data compatible with each of a plurality of states of a migration of data in a data store, wherein a service is at least intermittently reading data from and writing data to the data store; determining, by a migration runner of the framework, to perform the migration of the data; initiating, by the migration runner, the migration of the data, wherein the migration comprises a plurality of stages; storing, as the migration progresses through the plurality of stages, and at a migration data store of the framework, a current stage of the migration; and during the migration, using the migration proxy to read data from and write data to the data store.
Abstract:
Embodiments of the present disclosure relate to a data analysis system that may automatically generate memory-efficient clustered data structures, automatically analyze those clustered data structures, and provide results of the automated analysis in an optimized way to an analyst. The automated analysis of the clustered data structures (also referred to herein as data clusters) may include an automated application of various criteria or rules so as to generate a compact, human-readable analysis of the data clusters. The human-readable analyses (also referred to herein as “summaries” or “conclusions”) of the data clusters may be organized into an interactive user interface so as to enable an analyst to quickly navigate among information associated with various data clusters and efficiently evaluate those data clusters in the context of, for example, a fraud investigation. Embodiments of the present disclosure also relate to automated scoring of the clustered data structures.