公开(公告)号：US20250117382A1

公开(公告)日：2025-04-10

申请号：US18988025

申请日：2024-12-19

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Fabian Hueske ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/2453

Abstract: A system includes at least one hardware processor and at least one memory storing instructions that cause the at least one hardware processor to perform operations. The operations include generating a log of changes posted to a plurality of intermediate materialized tables (MTs) during execution of a query in a network-based database system. The query is associated with a source MT that the intermediate MTs depend on. The operations include rendering the log of changes into a dependency graph. The operations include configuring a plurality of processing pipelines based on the dependency graph. The operations include performing refreshes on one or more of the plurality of intermediate MTs in at least one of the plurality of processing pipelines to complete the refresh operation. The refreshes are performed responsive to detecting an instruction for a refresh operation on the source MT.

公开(公告)号：US12216638B2

公开(公告)日：2025-02-04

申请号：US18158627

申请日：2023-01-24

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Istvan Cseri ,  Fabian Hueske ,  Tyler Jones ,  Yevgeniy Kogan ,  Dzmitry Pauliukevich ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F16/22 ,  G06F16/2455

Abstract: Techniques described herein can accelerate change data capture determinations such as stream reads, which show changes made to a table between two points in time. Three distinct row bitests that mark deleted, updated, inserted, rows in micro-partitions can be added as metadata for the table. These bitsets can be generated during DML operations and then stored as metadata of the new partition generated by the DML operations. The bitsets can then be used to generate streams showing the changes in the table between two points in time (changes interval).

公开(公告)号：US20250124014A1

公开(公告)日：2025-04-17

申请号：US18987679

申请日：2024-12-19

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Istvan Cseri ,  Fabian Hueske ,  Tyler Jones ,  Yevgeniy Kogan ,  Dzmitry Pauliukevich ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F16/22 ,  G06F16/2455

Abstract: Techniques described herein can accelerate change data capture determinations such as stream reads, which show changes made to a table between two points in time. Three distinct row bitsets that mark deleted, updated, inserted, rows in micro-partitions can be added as metadata for the table. These bitsets can be generated during DML operations and then stored as metadata of the new partition generated by the DML operations. The bitsets can then be used to generate streams showing the changes in the table between two points in time (changes interval).

公开(公告)号：US11762855B1

公开(公告)日：2023-09-19

申请号：US17931590

申请日：2022-09-13

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Fabian Hueske ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/2453 ,  G06F7/14

CPC classification number: G06F16/24539 ,  G06F7/14 ,  G06F16/24542

Abstract: Provided herein are systems and methods for a database object (e.g., materialized table) configuration including using the database object for incremental maintenance of query results. For example, a method includes creating a materialized table using a table definition. The table definition includes a query on one or more base tables and a lag duration value. The lag duration value indicates a maximum time period that a result of a prior refresh of the query can lag behind a current time instance. A delta set for the one or more base tables is determined. The delta set includes at least one change applied to the one or more base tables since the prior refresh of the query. The delta set is merged with the materialized table to perform an incremental update of the materialized table.

公开(公告)号：US20230409574A1

公开(公告)日：2023-12-21

申请号：US18362898

申请日：2023-07-31

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Fabian Hueske ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/2453 ,  G06F7/14

CPC classification number: G06F16/24539 ,  G06F7/14 ,  G06F16/24542

Abstract: A system for a materialized table (MT) refresh using multiple processing pipelines includes at least one hardware processor coupled to memory storing instructions. The instructions cause the at least one hardware processor to perform operations including determining dependencies among a plurality of intermediate MTs generated from a source MT. The source MT uses a table definition with a query on one or more base tables and a lag duration value. A graph snapshot of dependencies among the plurality of intermediate MTs is generated. Processing pipelines are configured. Each of the processing pipelines corresponds to a subset of the plurality of intermediate MTs indicated by the graph snapshot. Responsive to detecting an instruction for a refresh operation on the source MT, refreshes on corresponding intermediate MTs of the plurality of intermediate MTs in each processing pipeline of the processing pipelines are performed to complete the refresh operation on the source MT.

公开(公告)号：US12216654B2

公开(公告)日：2025-02-04

申请号：US18362898

申请日：2023-07-31

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Fabian Hueske ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/2453 ,  G06F7/14

Abstract: A system for a materialized table (MT) refresh using multiple processing pipelines includes at least one hardware processor coupled to memory storing instructions. The instructions cause the at least one hardware processor to perform operations including determining dependencies among a plurality of intermediate MTs generated from a source MT. The source MT uses a table definition with a query on one or more base tables and a lag duration value. A graph snapshot of dependencies among the plurality of intermediate MTs is generated. Processing pipelines are configured. Each of the processing pipelines corresponds to a subset of the plurality of intermediate MTs indicated by the graph snapshot. Responsive to detecting an instruction for a refresh operation on the source MT, refreshes on corresponding intermediate MTs of the plurality of intermediate MTs in each processing pipeline of the processing pipelines are performed to complete the refresh operation on the source MT.

公开(公告)号：US20230237043A1

公开(公告)日：2023-07-27

申请号：US18158627

申请日：2023-01-24

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Istvan Cseri ,  Fabian Hueske ,  Tyler Jones ,  Yevgeniy Kogan ,  Dzmitry Pauliukevich ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F16/2455 ,  G06F16/22

CPC classification number: G06F16/2358 ,  G06F16/2282 ,  G06F16/2456

Abstract: Techniques described herein can accelerate change data capture determinations such as stream reads, which show changes made to a table between two points in time. Three distinct row bitests that mark deleted, updated, inserted, rows in micro-partitions can be added as metadata for the table. These bitsets can be generated during DML operations and then stored as metadata of the new partition generated by the DML operations. The bitsets can then be used to generate streams showing the changes in the table between two points in time (changes interval).

公开(公告)号：US11593345B1

公开(公告)日：2023-02-28

申请号：US17648621

申请日：2022-01-21

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Istvan Cseri ,  Fabian Hueske ,  Tyler Jones ,  Yevgeniy Kogan ,  Dzmitry Pauliukevich ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F16/2455 ,  G06F16/22

Abstract: Techniques described herein can accelerate change data capture determinations such as stream reads, which show changes made to a table between two points in time. Three distinct row bitests that mark deleted, updated, inserted, rows in micro-partitions can be added as metadata for the table. These bitsets can be generated during DML operations and then stored as metadata of the new partition generated by the DML operations. The bitsets can then be used to generate streams showing the changes in the table between two points in time (changes interval).