-
公开(公告)号:US20220121683A1
公开(公告)日:2022-04-21
申请号:US17463313
申请日:2021-08-31
Applicant: Snowflake Inc.
Inventor: Vasile Paraschiv , Saurin Shah , Marianne Shaw , Nileema Shingte
IPC: G06F16/27 , G06F16/2455 , G06F16/22 , G06F9/30 , G06F16/28
Abstract: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.
-
公开(公告)号:US20240211491A1
公开(公告)日:2024-06-27
申请号:US18599647
申请日:2024-03-08
Applicant: Snowflake Inc.
Inventor: Vasile Paraschiv , Saurin Shah , Marianne Shaw , Nileema Shingte
IPC: G06F16/27 , G06F9/30 , G06F16/11 , G06F16/13 , G06F16/182 , G06F16/22 , G06F16/2455 , G06F16/25 , G06F16/28
CPC classification number: G06F16/278 , G06F9/3009 , G06F16/137 , G06F16/182 , G06F16/2282 , G06F16/24554 , G06F16/283 , G06F16/116 , G06F16/254
Abstract: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.
-
公开(公告)号:US11347775B2
公开(公告)日:2022-05-31
申请号:US17463313
申请日:2021-08-31
Applicant: Snowflake Inc.
Inventor: Vasile Paraschiv , Saurin Shah , Marianne Shaw , Nileema Shingte
IPC: G06F16/27 , G06F16/28 , G06F9/30 , G06F16/22 , G06F16/2455
Abstract: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.
-
公开(公告)号:US11899646B2
公开(公告)日:2024-02-13
申请号:US18193069
申请日:2023-03-30
Applicant: Snowflake Inc.
Inventor: Selcuk Aya , Thierry Cruanes , Istvan Cseri , Benoit Dageville , Marcia Feitel , Steven P. Herbert , Xinglian Liu , James Malone , Subramanian Muralidhar , Muthunagappan Muthuraman , Polita Paulus , Marianne Shaw , Nileema Shingte , Wai Sing Wong , Jiaqi Yan
CPC classification number: G06F16/2282 , G06F16/2379 , G06F16/258
Abstract: The subject technology receives a command to commit a table in a different table format on an external volume. The subject technology generates a first snapshot of the table on internal storage. The subject technology generates a first list of metadata files on the internal storage. The subject technology generates, based on the first list of metadata files, a first set of metadata files on the internal storage. The subject technology generates a second snapshot of the table on the external volume. The subject technology generates a second list of metadata files on the external volume. The subject technology generates, based on the second list of metadata files, a second set of metadata files on the external volume. The subject technology generates a first set of data files in a different file format on the external volume.
-
公开(公告)号:US20230401193A1
公开(公告)日:2023-12-14
申请号:US18193069
申请日:2023-03-30
Applicant: Snowflake Inc.
Inventor: Selcuk Aya , Thierry Cruanes , Istvan Cseri , Benoit Dageville , Marcia Feitel , Steven P. Herbert , Xinglian Liu , James Malone , Subramanian Muralidhar , Muthunagappan Muthuraman , Polita Paulus , Marianne Shaw , Nileema Shingte , Wai Sing Wong , Jiaqi Yan
CPC classification number: G06F16/2282 , G06F16/2379 , G06F16/258
Abstract: The subject technology receives a command to commit a table in a different table format on an external volume. The subject technology generates a first snapshot of the table on internal storage. The subject technology generates a first list of metadata files on the internal storage. The subject technology generates, based on the first list of metadata files, a first set of metadata files on the internal storage. The subject technology generates a second snapshot of the table on the external volume. The subject technology generates a second list of metadata files on the external volume. The subject technology generates, based on the second list of metadata files, a second set of metadata files on the external volume. The subject technology generates a first set of data files in a different file format on the external volume.
-
公开(公告)号:US20220277024A1
公开(公告)日:2022-09-01
申请号:US17664144
申请日:2022-05-19
Applicant: Snowflake Inc.
Inventor: Vasile Paraschiv , Saurin Shah , Marianne Shaw , Nileeman Shingte
IPC: G06F16/27 , G06F16/28 , G06F9/30 , G06F16/22 , G06F16/2455 , G06F16/182 , G06F16/13 , G06F16/11 , G06F16/25
Abstract: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.
-
公开(公告)号:US11354331B2
公开(公告)日:2022-06-07
申请号:US17086221
申请日:2020-10-30
Applicant: Snowflake Inc.
Inventor: Vasile Paraschiv , Saurin Shah , Marianne Shaw , Nileema Shingte
IPC: G06F16/25 , G06F16/11 , G06F16/182 , G06F16/13
Abstract: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.
-
8.
公开(公告)号:US20240427755A1
公开(公告)日:2024-12-26
申请号:US18517744
申请日:2023-11-22
Applicant: Snowflake Inc.
Inventor: Selcuk Aya , Thierry Cruanes , Istvan Cseri , Benoit Dageville , Marcia Feitel , Steven P. Herbert , Dennis Huo , Xinglian Liu , Nithin Mahesh , James Malone , Subramanian Muralidhar , Muthunagappan Muthuraman , Ronald Lee Ortloff , Polita Paulus , Marianne Shaw , Nileema Shingte , Wai Sing Wong , Jiaqi Yan
IPC: G06F16/22 , G06F16/215 , G06F16/2457
Abstract: The subject technology provides embodiments for supporting a unified table which may be a managed table or an unmanaged table. Managed tables are those where the subject technology manages the metastore/catalog for the table, whereas unmanaged tables are tables where an external catalog controls the table and the subject technology integrates with that catalog to work with the table, but does not assume control of the table.
-
9.
公开(公告)号:US12050582B1
公开(公告)日:2024-07-30
申请号:US18498463
申请日:2023-10-31
Applicant: Snowflake Inc.
Inventor: Selcuk Aya , Thierry Cruanes , Istvan Cseri , Benoit Dageville , Marcia Feitel , Steven P. Herbert , Dennis Huo , Xinglian Liu , Nithin Mahesh , James Malone , Subramanian Muralidhar , Muthunagappan Muthuraman , Ronald Lee Ortloff , Polita Paulus , Marianne Shaw , Nileema Shingte , Wai Sing Wong , Jiaqi Yan
IPC: G06F16/22 , G06F16/215 , G06F16/2457
CPC classification number: G06F16/2282 , G06F16/215 , G06F16/24573
Abstract: The subject technology provides embodiments for supporting a unified table which may be a managed table or an unmanaged table. Managed tables are those where the subject technology manages the metastore/catalog for the table, whereas unmanaged tables are tables where an external catalog controls the table and the subject technology integrates with that catalog to work with the table, but does not assume control of the table.
-
公开(公告)号:US11960505B2
公开(公告)日:2024-04-16
申请号:US17664144
申请日:2022-05-19
Applicant: Snowflake Inc.
Inventor: Vasile Paraschiv , Saurin Shah , Marianne Shaw , Nileema Shingte
IPC: G06F16/27 , G06F9/30 , G06F16/13 , G06F16/182 , G06F16/22 , G06F16/2455 , G06F16/28 , G06F16/11 , G06F16/25
CPC classification number: G06F16/278 , G06F9/3009 , G06F16/137 , G06F16/182 , G06F16/2282 , G06F16/24554 , G06F16/283 , G06F16/116 , G06F16/254
Abstract: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.
-
-
-
-
-
-
-
-
-