公开(公告)号：US20250086168A1

公开(公告)日：2025-03-13

申请号：US18955789

申请日：2024-11-21

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Dotolongo

IPC: G06F16/23 ,  G06F11/34

Abstract: A method for query execution includes configuring, by at least one hardware processor, a materialized table (MT) based on the query and a base table. The MT includes a lag duration indicating a maximum time period that a result of a prior refresh of the query on the base table can lag behind a current time instance. The method further includes generating a query plan for the query. The method further includes determining multiple sets of data processing operations included in the query plan. The method further includes generating a plurality of intermediate MTs. The one or more intermediate MTs hold an intermediate processing state for the multiple sets of data processing operations. The method further includes configuring a refresh of the MT based on the intermediate processing state for the multiple sets of data processing operations.

公开(公告)号：US12242457B2

公开(公告)日：2025-03-04

申请号：US18459256

申请日：2023-08-31

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Tyler Jones ,  Daniel Mills ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F16/22 ,  G06F16/2455 ,  G06F16/27

Abstract: Provided herein are systems and methods for a stream object configuration, including query processing of stream objects using stream expansion. For example, a method includes decoding a query to obtain a first data processing operation and a first stream object. The first stream object is associated with a view on a base table. A first stream expansion on the first stream object is performed. The first stream expansion is based on generating a second stream object on the base table. A second stream expansion of the second stream object is performed. The second stream expansion is based on replacing the second stream object with at least a second data processing operation. The query is executed based on completing the first data processing operation and the at least a second data processing operation.

公开(公告)号：US12216654B2

公开(公告)日：2025-02-04

申请号：US18362898

申请日：2023-07-31

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Fabian Hueske ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/2453 ,  G06F7/14

Abstract: A system for a materialized table (MT) refresh using multiple processing pipelines includes at least one hardware processor coupled to memory storing instructions. The instructions cause the at least one hardware processor to perform operations including determining dependencies among a plurality of intermediate MTs generated from a source MT. The source MT uses a table definition with a query on one or more base tables and a lag duration value. A graph snapshot of dependencies among the plurality of intermediate MTs is generated. Processing pipelines are configured. Each of the processing pipelines corresponds to a subset of the plurality of intermediate MTs indicated by the graph snapshot. Responsive to detecting an instruction for a refresh operation on the source MT, refreshes on corresponding intermediate MTs of the plurality of intermediate MTs in each processing pipeline of the processing pipelines are performed to complete the refresh operation on the source MT.

公开(公告)号：US20240232224A1

公开(公告)日：2024-07-11

申请号：US18610863

申请日：2024-03-20

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Istvan Cseri ,  Tyler Jones ,  Dinesh Chandrakant Kulkarni ,  Daniel Mills ,  Daniel E. Sotolongo ,  Di Fei Zhang

IPC: G06F16/27

CPC classification number: G06F16/273

Abstract: Techniques for triggering pipeline execution based on data change (transaction commit) are described. The pipelines can be used for data ingestion or other specified tasks. These tasks can be operational across account, organization, cloud region, and cloud provider boundaries. The tasks can be triggered by commit post-processing. Gates in the tasks can be set up to reference change data capture information. If the gate is satisfied, tasks can be executed to set up data pipelines.

公开(公告)号：US20230315755A1

公开(公告)日：2023-10-05

申请号：US18103977

申请日：2023-01-31

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Istvan Cseri ,  Tyler Jones ,  Dinesh Chandrakant Kulkarni ,  Daniel Mills ,  Daniel E. Sotolongo ,  Di Fei Zhang

IPC: G06F16/27

CPC classification number: G06F16/273

Abstract: Techniques for triggering pipeline execution based on data change (transaction commit) are described. The pipelines can be used for data ingestion or other specified tasks. These tasks can be operational across account, organization, cloud region, and cloud provider boundaries. The tasks can be triggered by commit post-processing. Gates in the tasks can be set up to reference change data capture information. If the gate is satisfied, tasks can be executed to set up data pipelines.

公开(公告)号：US20250117382A1

公开(公告)日：2025-04-10

申请号：US18988025

申请日：2024-12-19

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Fabian Hueske ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/2453

Abstract: A system includes at least one hardware processor and at least one memory storing instructions that cause the at least one hardware processor to perform operations. The operations include generating a log of changes posted to a plurality of intermediate materialized tables (MTs) during execution of a query in a network-based database system. The query is associated with a source MT that the intermediate MTs depend on. The operations include rendering the log of changes into a dependency graph. The operations include configuring a plurality of processing pipelines based on the dependency graph. The operations include performing refreshes on one or more of the plurality of intermediate MTs in at least one of the plurality of processing pipelines to complete the refresh operation. The refreshes are performed responsive to detecting an instruction for a refresh operation on the source MT.

公开(公告)号：US12189616B2

公开(公告)日：2025-01-07

申请号：US18353317

申请日：2023-07-17

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F11/34

Abstract: A method includes retrieving a plurality of materialized tables (MTs). Each of the plurality of MTs includes a lag duration and refers to a corresponding base table of a plurality of base tables. The lag duration indicates a maximum time period that a result of a prior refresh of a query on the corresponding base table can lag behind a current time instance. A plurality of time instances for the MT is determined based on the lag duration and a number of prior refreshes of the corresponding base table. A plurality of aligned time instances for the plurality of MTs is determined based on the plurality of time instances for each of the plurality of MTs. Refresh operations are scheduled for the plurality of MTs at one or more of the plurality of aligned time instances that are within the maximum time period.

公开(公告)号：US20230401199A1

公开(公告)日：2023-12-14

申请号：US18353317

申请日：2023-07-17

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F11/34

CPC classification number: G06F16/2393 ,  G06F11/3419

Abstract: A method includes retrieving a plurality of materialized tables (MTs). Each of the plurality of MTs includes a lag duration and refers to a corresponding base table of a plurality of base tables. The lag duration indicates a maximum time period that a result of a prior refresh of a query on the corresponding base table can lag behind a current time instance. A plurality of time instances for the MT is determined based on the lag duration and a number of prior refreshes of the corresponding base table. A plurality of aligned time instances for the plurality of MTs is determined based on the plurality of time instances for each of the plurality of MTs. Refresh operations are scheduled for the plurality of MTs at one or more of the plurality of aligned time instances that are within the maximum time period.

公开(公告)号：US11301451B1

公开(公告)日：2022-04-12

申请号：US17460974

申请日：2021-08-30

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Tyler Jones ,  Daniel Mills ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F16/2455 ,  G06F16/27 ,  G06F16/22

Abstract: Provided herein are systems and methods for a stream object configuration, including configuring and processing streams on views. For example, a method includes detecting a first stream object on a view. The view includes a query associated with a source table. A syntax tree of the query is determined based on a definition of the view. The view is expanded based on replacing the first stream object with the syntax tree. The syntax tree includes a second stream object on the source table. Stream expansion of the second stream object is performed based on computing changes on the source table.

公开(公告)号：US20250068622A1

公开(公告)日：2025-02-27

申请号：US18948663

申请日：2024-11-15

Applicant: Snowflake Inc.

Inventor： Abdullah Al Mahmood ,  Tyler Jones ,  Xin Huang ,  Ganeshan Ramachandran Iyer ,  Jiaxing Liang ,  Daniel Mills ,  Subramanian Muralidhar ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F11/14 ,  G06F16/22

Abstract: Techniques for creating and using error tables to track errors associated with a base table are described. A command to perform an operation on a base table stored in a network-based data system can be received and executed, causing at least one error. At least one error record corresponding to the at least one error can be inputted into an error table, which is nested with the base table. Contextual information can be added to the at least one error record.