Task scheduling using a stream of committed transactions

    公开(公告)号:US12153569B2

    公开(公告)日:2024-11-26

    申请号:US17653491

    申请日:2022-03-04

    Applicant: Snowflake Inc.

    Abstract: A method includes generating a task using a plurality of logical statements embedded in a database, the plurality of logical statements corresponding to a data modification. Database data is ingested into a staging table that is configured within the database. The task is executed based on applying the data modification to a first set of partitions storing the database data and generating a second set of partitions. The second set of partitions store modified data corresponding to the database data. A stream of committed transactions is advanced at least in part by adding an entry into the stream. The entry corresponds to committed transactions performed on the first set of partitions during the data modification. A data processing task is scheduled for execution on the modified data based on the advancing of the stream offset.

    Tracking changes in database data
    118.
    发明授权

    公开(公告)号:US11874818B2

    公开(公告)日:2024-01-16

    申请号:US17660132

    申请日:2022-04-21

    Applicant: Snowflake Inc.

    CPC classification number: G06F16/2358 G06F16/2455 G06F16/278

    Abstract: A method includes partitioning a database table into a first plurality of partitions. A plurality of changes are executed on the database table in response to a transaction. The changes occur at a corresponding plurality of timestamps and result in a second plurality of partitions. The database table is updated to include a log with the plurality of changes. For each change of the plurality of changes, the log includes and an identification of a portion of the database table the change is applied to. A selection of a first timestamp and a second timestamp of the plurality of timestamps is detected. A delta is generated in response to the selection. The delta indicates a total change occurring to the database table between a first change of the plurality of changes corresponding to the first timestamp and a second change of the plurality of changes corresponding to the second timestamp.

    QUERY PROCESSING OF STREAM OBJECTS USING STREAM EXPANSION

    公开(公告)号:US20230418807A1

    公开(公告)日:2023-12-28

    申请号:US18459256

    申请日:2023-08-31

    Applicant: Snowflake Inc.

    Abstract: Provided herein are systems and methods for a stream object configuration, including query processing of stream objects using stream expansion. For example, a method includes decoding a query to obtain a first data processing operation and a first stream object. The first stream object is associated with a view on a base table. A first stream expansion on the first stream object is performed. The first stream expansion is based on generating a second stream object on the base table. A second stream expansion of the second stream object is performed. The second stream expansion is based on replacing the second stream object with at least a second data processing operation. The query is executed based on completing the first data processing operation and the at least a second data processing operation.

    SCHEMA EVOLUTION
    120.
    发明公开
    SCHEMA EVOLUTION 审中-公开

    公开(公告)号:US20230401180A1

    公开(公告)日:2023-12-14

    申请号:US18345987

    申请日:2023-06-30

    Applicant: Snowflake Inc.

    CPC classification number: G06F16/211

    Abstract: Techniques for schema mismatch detection and evolution are described. When data is being uploaded into a source table, schema of the data to be uploaded can be compared with the schema for the source table. If a schema mismatch is detected, the schema of the source table can be modified, and the upload can be continued without data loss.

Patent Agency Ranking