公开(公告)号：US11762855B1

公开(公告)日：2023-09-19

申请号：US17931590

申请日：2022-09-13

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Fabian Hueske ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/2453 ,  G06F7/14

CPC classification number: G06F16/24539 ,  G06F7/14 ,  G06F16/24542

Abstract: Provided herein are systems and methods for a database object (e.g., materialized table) configuration including using the database object for incremental maintenance of query results. For example, a method includes creating a materialized table using a table definition. The table definition includes a query on one or more base tables and a lag duration value. The lag duration value indicates a maximum time period that a result of a prior refresh of the query can lag behind a current time instance. A delta set for the one or more base tables is determined. The delta set includes at least one change applied to the one or more base tables since the prior refresh of the query. The delta set is merged with the materialized table to perform an incremental update of the materialized table.

公开(公告)号：US20250086168A1

公开(公告)日：2025-03-13

申请号：US18955789

申请日：2024-11-21

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Dotolongo

IPC: G06F16/23 ,  G06F11/34

Abstract: A method for query execution includes configuring, by at least one hardware processor, a materialized table (MT) based on the query and a base table. The MT includes a lag duration indicating a maximum time period that a result of a prior refresh of the query on the base table can lag behind a current time instance. The method further includes generating a query plan for the query. The method further includes determining multiple sets of data processing operations included in the query plan. The method further includes generating a plurality of intermediate MTs. The one or more intermediate MTs hold an intermediate processing state for the multiple sets of data processing operations. The method further includes configuring a refresh of the MT based on the intermediate processing state for the multiple sets of data processing operations.

公开(公告)号：US12216654B2

公开(公告)日：2025-02-04

申请号：US18362898

申请日：2023-07-31

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Fabian Hueske ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/2453 ,  G06F7/14

Abstract: A system for a materialized table (MT) refresh using multiple processing pipelines includes at least one hardware processor coupled to memory storing instructions. The instructions cause the at least one hardware processor to perform operations including determining dependencies among a plurality of intermediate MTs generated from a source MT. The source MT uses a table definition with a query on one or more base tables and a lag duration value. A graph snapshot of dependencies among the plurality of intermediate MTs is generated. Processing pipelines are configured. Each of the processing pipelines corresponds to a subset of the plurality of intermediate MTs indicated by the graph snapshot. Responsive to detecting an instruction for a refresh operation on the source MT, refreshes on corresponding intermediate MTs of the plurality of intermediate MTs in each processing pipeline of the processing pipelines are performed to complete the refresh operation on the source MT.

公开(公告)号：US20230409574A1

公开(公告)日：2023-12-21

申请号：US18362898

申请日：2023-07-31

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Fabian Hueske ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/2453 ,  G06F7/14

CPC classification number: G06F16/24539 ,  G06F7/14 ,  G06F16/24542

Abstract: A system for a materialized table (MT) refresh using multiple processing pipelines includes at least one hardware processor coupled to memory storing instructions. The instructions cause the at least one hardware processor to perform operations including determining dependencies among a plurality of intermediate MTs generated from a source MT. The source MT uses a table definition with a query on one or more base tables and a lag duration value. A graph snapshot of dependencies among the plurality of intermediate MTs is generated. Processing pipelines are configured. Each of the processing pipelines corresponds to a subset of the plurality of intermediate MTs indicated by the graph snapshot. Responsive to detecting an instruction for a refresh operation on the source MT, refreshes on corresponding intermediate MTs of the plurality of intermediate MTs in each processing pipeline of the processing pipelines are performed to complete the refresh operation on the source MT.

公开(公告)号：US11755568B1

公开(公告)日：2023-09-12

申请号：US17931705

申请日：2022-09-13

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F11/34

CPC classification number: G06F16/2393 ,  G06F11/3419

Abstract: Provided herein are systems and methods for a database object (e.g., materialized table) configuration including scheduling refreshes of the materialized table. For example, a method includes determining a dependency graph for a first MT. The dependency graph comprises a second MT from which the first MT depends. The first MT includes a query on one or more base tables and a lag duration value. The lag duration value indicates a maximum time period that a result of a prior refresh of the query can lag behind a current time instance. A tick period is selected for a set of ticks based on the lag duration value. The set of ticks corresponds to a set of aligned time instances. Refresh operations are scheduled for the first and second MTs at corresponding time instances from the set of aligned time instances. The corresponding time instances are separated by the tick period.

公开(公告)号：US20250117382A1

公开(公告)日：2025-04-10

申请号：US18988025

申请日：2024-12-19

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Fabian Hueske ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/2453

Abstract: A system includes at least one hardware processor and at least one memory storing instructions that cause the at least one hardware processor to perform operations. The operations include generating a log of changes posted to a plurality of intermediate materialized tables (MTs) during execution of a query in a network-based database system. The query is associated with a source MT that the intermediate MTs depend on. The operations include rendering the log of changes into a dependency graph. The operations include configuring a plurality of processing pipelines based on the dependency graph. The operations include performing refreshes on one or more of the plurality of intermediate MTs in at least one of the plurality of processing pipelines to complete the refresh operation. The refreshes are performed responsive to detecting an instruction for a refresh operation on the source MT.

公开(公告)号：US12189616B2

公开(公告)日：2025-01-07

申请号：US18353317

申请日：2023-07-17

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F11/34

Abstract: A method includes retrieving a plurality of materialized tables (MTs). Each of the plurality of MTs includes a lag duration and refers to a corresponding base table of a plurality of base tables. The lag duration indicates a maximum time period that a result of a prior refresh of a query on the corresponding base table can lag behind a current time instance. A plurality of time instances for the MT is determined based on the lag duration and a number of prior refreshes of the corresponding base table. A plurality of aligned time instances for the plurality of MTs is determined based on the plurality of time instances for each of the plurality of MTs. Refresh operations are scheduled for the plurality of MTs at one or more of the plurality of aligned time instances that are within the maximum time period.

公开(公告)号：US20230401199A1

公开(公告)日：2023-12-14

申请号：US18353317

申请日：2023-07-17

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F11/34

CPC classification number: G06F16/2393 ,  G06F11/3419

Abstract: A method includes retrieving a plurality of materialized tables (MTs). Each of the plurality of MTs includes a lag duration and refers to a corresponding base table of a plurality of base tables. The lag duration indicates a maximum time period that a result of a prior refresh of a query on the corresponding base table can lag behind a current time instance. A plurality of time instances for the MT is determined based on the lag duration and a number of prior refreshes of the corresponding base table. A plurality of aligned time instances for the plurality of MTs is determined based on the plurality of time instances for each of the plurality of MTs. Refresh operations are scheduled for the plurality of MTs at one or more of the plurality of aligned time instances that are within the maximum time period.