Big data processing method based on direct computation of compressed data
Abstract:
A big data processing method based on direct computation of compressed data. The method includes 1) compressing, based on a modified Sequitur compression method, original input data according to a smallest compression granularity given by an user, and transforming them into a directed acyclic graph, DAG, consisting of digits; and 2) determining an optimal traversal pattern, and performing a top-downward traversal or a bottom-upward traversal on the DAG in the step 1) based on the determined optimal traversal pattern so as to enable direct processing of the compressed data. By providing a modified Sequitur algorithm and top-downward and bottom-upward traversal strategies in the disclosure, direct processing of compressed data is enabled, significant improvement in time and space has been gained with broad applicability, and certain representations with respect to more advanced document analytics can still be derived on the basis of these.
Information query
Patent Agency Ranking
0/0