Effective method to compress tabular data export files for data movement
Abstract:
Compression of data for database movement, includes: selecting a first group of categorical columns for compression; selecting a next group of categorical columns from remaining columns for compression; repeating the selecting of the next group until a predetermined compression threshold is met; creating first compression files comprising compressed representations of the columns in the first group; creating next compression files comprising compressed representations of the columns in each of the next groups; storing initial row sort order, group identity, and column positions corresponding to each of the next groups; and storing any columns not selected for compression in an uncompressed file in the original row sort order. Decompression of the data includes: rebuilding categorical columns in each group of compression files using group identity and column positions corresponding to the group; and sorting rows comprising the rebuilt categorical columns to the initial row sort order.
Information query
Patent Agency Ranking
0/0