公开(公告)号：US20250117382A1

公开(公告)日：2025-04-10

申请号：US18988025

申请日：2024-12-19

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Fabian Hueske ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/2453

Abstract: A system includes at least one hardware processor and at least one memory storing instructions that cause the at least one hardware processor to perform operations. The operations include generating a log of changes posted to a plurality of intermediate materialized tables (MTs) during execution of a query in a network-based database system. The query is associated with a source MT that the intermediate MTs depend on. The operations include rendering the log of changes into a dependency graph. The operations include configuring a plurality of processing pipelines based on the dependency graph. The operations include performing refreshes on one or more of the plurality of intermediate MTs in at least one of the plurality of processing pipelines to complete the refresh operation. The refreshes are performed responsive to detecting an instruction for a refresh operation on the source MT.

公开(公告)号：US12189616B2

公开(公告)日：2025-01-07

申请号：US18353317

申请日：2023-07-17

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F11/34

Abstract: A method includes retrieving a plurality of materialized tables (MTs). Each of the plurality of MTs includes a lag duration and refers to a corresponding base table of a plurality of base tables. The lag duration indicates a maximum time period that a result of a prior refresh of a query on the corresponding base table can lag behind a current time instance. A plurality of time instances for the MT is determined based on the lag duration and a number of prior refreshes of the corresponding base table. A plurality of aligned time instances for the plurality of MTs is determined based on the plurality of time instances for each of the plurality of MTs. Refresh operations are scheduled for the plurality of MTs at one or more of the plurality of aligned time instances that are within the maximum time period.

公开(公告)号：US20240126765A1

公开(公告)日：2024-04-18

申请号：US18392327

申请日：2023-12-21

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Istvan Cseri ,  Tyler Jones ,  Daniel E. Sotolongo ,  Zhuo Zhang

IPC: G06F16/2455 ,  G06F16/22 ,  G06F16/2453 ,  G06F16/25

CPC classification number: G06F16/24568 ,  G06F16/2219 ,  G06F16/24544 ,  G06F16/2456 ,  G06F16/258

Abstract: A streaming ingest platform can improve latency and expense issues related to uploading data into a cloud data system. The streaming ingest platform can organize the data to be ingested into per-table chunks and per-account blobs. This data may be committed and may be made available for query processing before it is ingested into the target source tables. This significantly improves latency issues. The streaming ingest platform can also accommodate uploading data from various sources with different processing and communication capabilities, such as Internet of Things (IOT) devices.

公开(公告)号：US11893029B2

公开(公告)日：2024-02-06

申请号：US18049325

申请日：2022-10-25

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Istvan Cseri ,  Tyler Jones ,  Daniel E. Sotolongo ,  Zhuo Zhang

IPC: G06F16/245 ,  G06F16/25 ,  G06F16/22 ,  G06F16/2455 ,  G06F16/2453

CPC classification number: G06F16/24568 ,  G06F16/2219 ,  G06F16/2456 ,  G06F16/24544 ,  G06F16/258

Abstract: A streaming ingest platform can improve latency and expense issues related to uploading data into a cloud data system. The streaming ingest platform can organize the data to be ingested into per-table chunks and per-account blobs. This data may be committed and may be made available for query processing before it is ingested into the target source tables. This significantly improves latency issues. The streaming ingest platform can also accommodate uploading data from various sources with different processing and communication capabilities, such as Internet of Things (IOT) devices.

公开(公告)号：US20230401199A1

公开(公告)日：2023-12-14

申请号：US18353317

申请日：2023-07-17

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Tyler Jones ,  Daniel Mills ,  Leon Papke ,  Prasanna Rajaperumal ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F11/34

CPC classification number: G06F16/2393 ,  G06F11/3419

Abstract: A method includes retrieving a plurality of materialized tables (MTs). Each of the plurality of MTs includes a lag duration and refers to a corresponding base table of a plurality of base tables. The lag duration indicates a maximum time period that a result of a prior refresh of a query on the corresponding base table can lag behind a current time instance. A plurality of time instances for the MT is determined based on the lag duration and a number of prior refreshes of the corresponding base table. A plurality of aligned time instances for the plurality of MTs is determined based on the plurality of time instances for each of the plurality of MTs. Refresh operations are scheduled for the plurality of MTs at one or more of the plurality of aligned time instances that are within the maximum time period.

公开(公告)号：US11748327B2

公开(公告)日：2023-09-05

申请号：US17648622

申请日：2022-01-21

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Istvan Cseri ,  Tyler Jones ,  Daniel E. Sotolongo

IPC: G06F16/22 ,  G06F16/23 ,  G06F16/2455

CPC classification number: G06F16/2282 ,  G06F16/2358 ,  G06F16/24568

Abstract: A system or persistent table may be generated storing changelog information of a primary base table. The system table may then be used to create streams of relevant information. In some examples, the streams may read from the system table for information past a retention period of the primary table while reading from the primary table information in the retention period.

公开(公告)号：US11734301B1

公开(公告)日：2023-08-22

申请号：US17656116

申请日：2022-03-23

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Tyler Jones ,  Daniel E. Sotolongo ,  Boyuan Zhang

IPC: G06F16/30 ,  G06F16/27

CPC classification number: G06F16/27

Abstract: Techniques described herein can enable stream replication. A first deployment can store a table including one or more streams. The techniques described herein can be used to replicate the table at a second deployment while replicating the one or more streams associated with the table. Select prior table versions and partitions in the table are copied to the second deployment to enable stream replication.

公开(公告)号：US11301451B1

公开(公告)日：2022-04-12

申请号：US17460974

申请日：2021-08-30

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Tyler Jones ,  Daniel Mills ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F16/2455 ,  G06F16/27 ,  G06F16/22

Abstract: Provided herein are systems and methods for a stream object configuration, including configuring and processing streams on views. For example, a method includes detecting a first stream object on a view. The view includes a query associated with a source table. A syntax tree of the query is determined based on a definition of the view. The view is expanded based on replacing the first stream object with the syntax tree. The syntax tree includes a second stream object on the source table. Stream expansion of the second stream object is performed based on computing changes on the source table.

公开(公告)号：US20250124014A1

公开(公告)日：2025-04-17

申请号：US18987679

申请日：2024-12-19

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Istvan Cseri ,  Fabian Hueske ,  Tyler Jones ,  Yevgeniy Kogan ,  Dzmitry Pauliukevich ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F16/22 ,  G06F16/2455

Abstract: Techniques described herein can accelerate change data capture determinations such as stream reads, which show changes made to a table between two points in time. Three distinct row bitsets that mark deleted, updated, inserted, rows in micro-partitions can be added as metadata for the table. These bitsets can be generated during DML operations and then stored as metadata of the new partition generated by the DML operations. The bitsets can then be used to generate streams showing the changes in the table between two points in time (changes interval).

公开(公告)号：US20250068622A1

公开(公告)日：2025-02-27

申请号：US18948663

申请日：2024-11-15

Applicant: Snowflake Inc.

Inventor： Abdullah Al Mahmood ,  Tyler Jones ,  Xin Huang ,  Ganeshan Ramachandran Iyer ,  Jiaxing Liang ,  Daniel Mills ,  Subramanian Muralidhar ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F11/14 ,  G06F16/22

Abstract: Techniques for creating and using error tables to track errors associated with a base table are described. A command to perform an operation on a base table stored in a network-based data system can be received and executed, causing at least one error. At least one error record corresponding to the at least one error can be inputted into an error table, which is nested with the base table. Contextual information can be added to the at least one error record.