-
公开(公告)号:US20220327132A1
公开(公告)日:2022-10-13
申请号:US17647500
申请日:2022-01-10
Applicant: Snowflake Inc.
Inventor: Tyler Arthur Akidau , Istvan Cseri , Tyler Jones , Daniel E. Sotolongo , Zhuo Zhang
IPC: G06F16/2455 , G06F16/2453
Abstract: A streaming ingest platform can improve latency and expense issues related to uploading data into a cloud data system. The streaming ingest platform can organize the data to be ingested into per-table chunks and per-account blobs. This data may be committed and may be made available for query processing before it is ingested into the target source tables. This significantly improves latency issues. The streaming ingest platform can also accommodate uploading data from various sources with different processing and communication capabilities, such as Internet of Things (IOT) devices.
-
公开(公告)号:US11250006B1
公开(公告)日:2022-02-15
申请号:US17386258
申请日:2021-07-27
Applicant: Snowflake Inc.
Inventor: Tyler Arthur Akidau , Istvan Cseri , Tyler Jones , Daniel E. Sotolongo , Zhuo Zhang
IPC: G06F16/245 , G06F16/2455 , G06F16/2453
Abstract: A streaming ingest platform can improve latency and expense issues related to uploading data into a cloud data system. The streaming ingest platform can organize the data to be ingested into per-table chunks and per-account blobs. This data may be committed and may be made available for query processing before it is ingested into the target source tables. This significantly improves latency issues. The streaming ingest platform can also accommodate uploading data from various sources with different processing and communication capabilities, such as Internet of Things (IOT) devices.
-
公开(公告)号:US11100111B1
公开(公告)日:2021-08-24
申请号:US17226423
申请日:2021-04-09
Applicant: Snowflake Inc.
Inventor: Tyler Arthur Akidau , Istvan Cseri , Tyler Jones , Daniel E. Sotolongo , Zhuo Zhang
IPC: G06F16/245 , G06F16/2455 , G06F16/2453
Abstract: A streaming ingest platform can improve latency and expense issues related to uploading data into a cloud data system. The streaming ingest platform can organize the data to be ingested into per-table chunks and per-account blobs. This data may be committed and may be made available for query processing before it is ingested into the target source tables. This significantly improves latency issues. The streaming ingest platform can also accommodate uploading data from various sources with different processing and communication capabilities, such as Internet of Things (IOT) devices.
-
公开(公告)号:US20250086168A1
公开(公告)日:2025-03-13
申请号:US18955789
申请日:2024-11-21
Applicant: Snowflake Inc.
Inventor: Tyler Arthur Akidau , Tyler Jones , Daniel Mills , Leon Papke , Prasanna Rajaperumal , Daniel E. Dotolongo
Abstract: A method for query execution includes configuring, by at least one hardware processor, a materialized table (MT) based on the query and a base table. The MT includes a lag duration indicating a maximum time period that a result of a prior refresh of the query on the base table can lag behind a current time instance. The method further includes generating a query plan for the query. The method further includes determining multiple sets of data processing operations included in the query plan. The method further includes generating a plurality of intermediate MTs. The one or more intermediate MTs hold an intermediate processing state for the multiple sets of data processing operations. The method further includes configuring a refresh of the MT based on the intermediate processing state for the multiple sets of data processing operations.
-
公开(公告)号:US12216654B2
公开(公告)日:2025-02-04
申请号:US18362898
申请日:2023-07-31
Applicant: Snowflake Inc.
Inventor: Tyler Arthur Akidau , Fabian Hueske , Tyler Jones , Daniel Mills , Leon Papke , Prasanna Rajaperumal , Daniel E. Sotolongo
IPC: G06F16/2453 , G06F7/14
Abstract: A system for a materialized table (MT) refresh using multiple processing pipelines includes at least one hardware processor coupled to memory storing instructions. The instructions cause the at least one hardware processor to perform operations including determining dependencies among a plurality of intermediate MTs generated from a source MT. The source MT uses a table definition with a query on one or more base tables and a lag duration value. A graph snapshot of dependencies among the plurality of intermediate MTs is generated. Processing pipelines are configured. Each of the processing pipelines corresponds to a subset of the plurality of intermediate MTs indicated by the graph snapshot. Responsive to detecting an instruction for a refresh operation on the source MT, refreshes on corresponding intermediate MTs of the plurality of intermediate MTs in each processing pipeline of the processing pipelines are performed to complete the refresh operation on the source MT.
-
公开(公告)号:US20240232224A1
公开(公告)日:2024-07-11
申请号:US18610863
申请日:2024-03-20
Applicant: Snowflake Inc.
Inventor: Tyler Arthur Akidau , Istvan Cseri , Tyler Jones , Dinesh Chandrakant Kulkarni , Daniel Mills , Daniel E. Sotolongo , Di Fei Zhang
IPC: G06F16/27
CPC classification number: G06F16/273
Abstract: Techniques for triggering pipeline execution based on data change (transaction commit) are described. The pipelines can be used for data ingestion or other specified tasks. These tasks can be operational across account, organization, cloud region, and cloud provider boundaries. The tasks can be triggered by commit post-processing. Gates in the tasks can be set up to reference change data capture information. If the gate is satisfied, tasks can be executed to set up data pipelines.
-
公开(公告)号:US20230367757A1
公开(公告)日:2023-11-16
申请号:US18359322
申请日:2023-07-26
Applicant: Snowflake Inc.
Inventor: Tyler Arthur Akidau , Istvan Cseri , Tyler Jones , Daniel E. Sotolongo
IPC: G06F16/22 , G06F16/23 , G06F16/2455
CPC classification number: G06F16/2282 , G06F16/2358 , G06F16/24568
Abstract: A system or persistent table may be generated storing changelog information of a primary base table. The system table may then be used to create streams of relevant information. In some examples, the streams may read from the system table for information past a retention period of the primary table while reading from the primary table information in the retention period.
-
公开(公告)号:US20230315755A1
公开(公告)日:2023-10-05
申请号:US18103977
申请日:2023-01-31
Applicant: Snowflake Inc.
Inventor: Tyler Arthur Akidau , Istvan Cseri , Tyler Jones , Dinesh Chandrakant Kulkarni , Daniel Mills , Daniel E. Sotolongo , Di Fei Zhang
IPC: G06F16/27
CPC classification number: G06F16/273
Abstract: Techniques for triggering pipeline execution based on data change (transaction commit) are described. The pipelines can be used for data ingestion or other specified tasks. These tasks can be operational across account, organization, cloud region, and cloud provider boundaries. The tasks can be triggered by commit post-processing. Gates in the tasks can be set up to reference change data capture information. If the gate is satisfied, tasks can be executed to set up data pipelines.
-
公开(公告)号:US20230297579A1
公开(公告)日:2023-09-21
申请号:US18324710
申请日:2023-05-26
Applicant: Snowflake Inc.
Inventor: Tyler Arthur Akidau , Thierry Cruanes , Istvan Cseri , Benoit Dageville , Tyler Jones , Dinesh Chandrakant Kulkarni
IPC: G06F16/2455 , G06F16/2453
CPC classification number: G06F16/24568 , G06F16/24544
Abstract: Hybrid tables can be used in different use-case scenarios. Hybrid tables provide a flexible mechanism to support files and data in different formats while providing access to the different types of data as part of one table. This flexibility can allow the use of hybrid tables in data lake or other similar environments.
-
公开(公告)号:US11727013B2
公开(公告)日:2023-08-15
申请号:US17930150
申请日:2022-09-07
Applicant: Snowflake Inc.
Inventor: Tyler Arthur Akidau , Thierry Cruanes , Istvan Cseri , Benoit Dageville , Tyler Jones , Dinesh Chandrakant Kulkarni
IPC: G06F16/245 , G06F16/2455 , G06F16/2453
CPC classification number: G06F16/24568 , G06F16/24544
Abstract: Hybrid tables can be used in different use-case scenarios. Hybrid tables provide a flexible mechanism to support files and data in different formats while providing access to the different types of data as part of one table. This flexibility can allow the use of hybrid tables in data lake or other similar environments.
-
-
-
-
-
-
-
-
-