公开(公告)号：US12287808B2

公开(公告)日：2025-04-29

申请号：US18472912

申请日：2023-09-22

Applicant: SNOWFLAKE INC.

Inventor： Thierry Cruanes ,  Benoit Dageville ,  Marcin Zukowski

IPC: G06F16/27 ,  A61F5/56 ,  G06F9/48 ,  G06F9/50 ,  G06F16/14 ,  G06F16/182 ,  G06F16/21 ,  G06F16/22 ,  G06F16/23 ,  G06F16/2453 ,  G06F16/2455 ,  G06F16/2458 ,  G06F16/25 ,  G06F16/28 ,  G06F16/951 ,  G06F16/9535 ,  G06F16/9538 ,  H04L67/1095 ,  H04L67/1097 ,  H04L67/568

Abstract: Example resource management systems and methods are described. In one implementation, a resource manager is configured to manage data processing tasks associated with multiple data elements. An execution platform is coupled to the resource manager and includes multiple execution nodes configured to store data retrieved from multiple remote storage devices. Each execution node includes a cache and a processor, where the cache and processor are independent of the remote storage devices. A metadata manager is configured to access metadata associated with at least a portion of the multiple data elements.

公开(公告)号：US20240330319A1

公开(公告)日：2024-10-03

申请号：US18738875

申请日：2024-06-10

Applicant: Snowflake Inc.

Inventor： Benoit Dageville ,  Thierry j:;ruanes ,  Marcin Zukowski

IPC: G06F16/27 ,  A61F5/56 ,  G06F9/48 ,  G06F9/50 ,  G06F16/14 ,  G06F16/182 ,  G06F16/21 ,  G06F16/22 ,  G06F16/23 ,  G06F16/2453 ,  G06F16/2455 ,  G06F16/2458 ,  G06F16/25 ,  G06F16/28 ,  G06F16/951 ,  G06F16/9535 ,  G06F16/9538 ,  H04L67/1095 ,  H04L67/1097 ,  H04L67/568

CPC classification number: G06F16/273 ,  A61F5/566 ,  G06F9/4881 ,  G06F9/5016 ,  G06F9/5044 ,  G06F9/5083 ,  G06F9/5088 ,  G06F16/148 ,  G06F16/1827 ,  G06F16/211 ,  G06F16/221 ,  G06F16/2365 ,  G06F16/24532 ,  G06F16/24545 ,  G06F16/24552 ,  G06F16/2456 ,  G06F16/2471 ,  G06F16/254 ,  G06F16/27 ,  G06F16/283 ,  G06F16/951 ,  G06F16/9535 ,  G06F16/9538 ,  H04L67/1095 ,  H04L67/1097 ,  H04L67/568

Abstract: Example resource management systems and methods are described. In one implementation, a system includes a memory and a processing device operatively coupled to the memory. The processing device is to receive a query referencing database data stored in a storage platform, determine a task associated with processing the received query, and create an execution node comprising cache resources and processing resources. Furthermore, a size of the cache resources of the execution node is determined upon creation of the execution node, based at least in part on the task, and processing resources of the execution node are determined upon creation of the execution node, based at least in part on the task. The execution node is included within a plurality of execution nodes to process the task associated with processing the received query.

公开(公告)号：US11977560B2

公开(公告)日：2024-05-07

申请号：US17563418

申请日：2021-12-28

Applicant: Snowflake Inc.

Inventor： Benoit Dageville ,  Thierry Cruanes ,  Marcin Zukowski

IPC: G06F16/27 ,  A61F5/56 ,  G06F9/48 ,  G06F9/50 ,  G06F16/14 ,  G06F16/182 ,  G06F16/21 ,  G06F16/22 ,  G06F16/23 ,  G06F16/2453 ,  G06F16/2455 ,  G06F16/2458 ,  G06F16/25 ,  G06F16/28 ,  G06F16/951 ,  G06F16/9535 ,  G06F16/9538 ,  H04L67/1095 ,  H04L67/1097 ,  H04L67/568

CPC classification number: G06F16/273 ,  A61F5/566 ,  G06F9/4881 ,  G06F9/5016 ,  G06F9/5044 ,  G06F9/5083 ,  G06F9/5088 ,  G06F16/148 ,  G06F16/1827 ,  G06F16/211 ,  G06F16/221 ,  G06F16/2365 ,  G06F16/24532 ,  G06F16/24545 ,  G06F16/24552 ,  G06F16/2456 ,  G06F16/2471 ,  G06F16/254 ,  G06F16/27 ,  G06F16/283 ,  G06F16/951 ,  G06F16/9535 ,  G06F16/9538 ,  H04L67/1095 ,  H04L67/1097 ,  H04L67/568

Abstract: Example resource management systems and methods are described. In one implementation, a resource manager is to be coupled to one or more virtual data warehouse instances that each comprise a plurality of computing resources. A resource manager receives a query directed to database data. The resource manager determines a plurality of tasks associated with the query. The resource manager distributes the plurality of tasks among the one or more virtual data warehouse instances to cause each computing resource of the plurality of computing resources of the one or more virtual data warehouse instances to process a respective task of the plurality of tasks. The resource manager receives results obtained from the processing of the plurality of tasks. The resource manager uses the results to respond to the query.

公开(公告)号：US20240111787A1

公开(公告)日：2024-04-04

申请号：US18539079

申请日：2023-12-13

Applicant: Snowflake Inc.

Inventor： Benoit Dageville ,  Thierry Cruanes ,  Marcin Zukowski ,  Allison Waingold Lee ,  Philipp Thomas Unterbrunner

IPC: G06F16/27 ,  A61F5/56 ,  G06F9/48 ,  G06F9/50 ,  G06F16/14 ,  G06F16/182 ,  G06F16/21 ,  G06F16/22 ,  G06F16/23 ,  G06F16/2453 ,  G06F16/2455 ,  G06F16/2458 ,  G06F16/25 ,  G06F16/28 ,  G06F16/951 ,  G06F16/9535 ,  G06F16/9538 ,  H04L67/1095 ,  H04L67/1097 ,  H04L67/568

CPC classification number: G06F16/273 ,  A61F5/566 ,  G06F9/4881 ,  G06F9/5016 ,  G06F9/5044 ,  G06F9/5083 ,  G06F9/5088 ,  G06F16/148 ,  G06F16/1827 ,  G06F16/211 ,  G06F16/221 ,  G06F16/2365 ,  G06F16/24532 ,  G06F16/24545 ,  G06F16/24552 ,  G06F16/2456 ,  G06F16/2471 ,  G06F16/254 ,  G06F16/27 ,  G06F16/283 ,  G06F16/951 ,  G06F16/9535 ,  G06F16/9538 ,  H04L67/1095 ,  H04L67/1097 ,  H04L67/568

Abstract: A method, apparatus, and system for join operations of a plurality of relations that are distributed over a plurality of storage locations over a network of computing components.

公开(公告)号：US11494337B2

公开(公告)日：2022-11-08

申请号：US17508705

申请日：2021-10-22

Applicant: SNOWFLAKE INC.

Inventor： Thierry Cruanes ,  Benoit Dageville ,  Ashish Motivala ,  Marcin Zukowski

IPC: G06F16/00 ,  G06F16/16 ,  G06F16/22 ,  G06F16/215 ,  G06F16/9535 ,  G06F16/2453

Abstract: A system and method for pruning data based on metadata. The method may include receiving a query with a plurality of predicates and identifying one or more applicable files that includes database data satisfying at least one of the plurality of predicates. The identifying the one or more applicable files including reading metadata stored in a metadata store that is separate from the database data. The method further includes pruning inapplicable files comprising database data that does not satisfy at least one of the plurality of predicates to create a reduced set of files and reading the reduced set of files to execute the query.

公开(公告)号：US11321352B2

公开(公告)日：2022-05-03

申请号：US17361210

申请日：2021-06-28

Applicant: SNOWFLAKE INC.

Inventor： Thierry Cruanes ,  Benoit Dageville ,  Marcin Zukowski

IPC: G06F16/27 ,  G06F16/21 ,  G06F16/23 ,  G06F16/2458 ,  G06F9/50 ,  G06F16/14 ,  G06F16/22 ,  G06F16/951 ,  G06F16/182 ,  G06F16/2455 ,  G06F16/9535 ,  G06F16/2453 ,  G06F9/48 ,  H04L67/1095 ,  H04L67/568 ,  H04L67/1097

Abstract: A method and apparatus managing a set of processors for a set of queries is described. In an exemplary embodiment, a device receives a set of queries for a data warehouse, the set of queries including one or more queries to be processed by the data warehouse. The device further provisions a set of processors from a first plurality of processors, where the set of processors to process the set of queries, and a set of storage resources to store data for the set of queries. In addition, the device monitors a utilization of the set of processors as the set of processors processes the set of queries. The device additionally updates a number of the processors in the set of processors provisioned based on the utilization/Furthermore, the device processes the set of queries using the updated set of processors.

公开(公告)号：US20220067067A1

公开(公告)日：2022-03-03

申请号：US17524439

申请日：2021-11-11

Applicant: SNOWFLAKE INC.

Inventor： Benoit Dageville ,  Thierry Cruanes ,  Marcin Zukowski

IPC: G06F16/27 ,  G06F9/50 ,  G06F16/14 ,  G06F16/21 ,  G06F16/22 ,  G06F16/951 ,  G06F16/182 ,  G06F16/23 ,  G06F16/2455 ,  G06F16/2458 ,  G06F16/9535 ,  G06F16/2453 ,  G06F9/48 ,  H04L29/08

Abstract: Example resource management systems and methods are described. In one implementation, a processing device is to: provide a plurality of virtual processors, each virtual processor comprising a cache memory to cache a portion of database data accessed from a storage platform; receive a query referencing the database data stored in the storage platform; determine at least one task to process the received query; and allocate the at least one task to a subset of the plurality of virtual processors to process the at least one task. The allocation may be based at least in part on the respective portions of the database data that are cached within the respective cache memories of the subset of the plurality of virtual processors.

公开(公告)号：US20220035834A1

公开(公告)日：2022-02-03

申请号：US17501426

申请日：2021-10-14

Applicant: SNOWFLAKE INC.

Inventor： Thierry Cruanes ,  Benoit Dageville ,  Marcin Zukowski

IPC: G06F16/27 ,  G06F16/22 ,  G06F9/50 ,  G06F9/48 ,  G06F16/2458 ,  G06F16/21 ,  G06F16/2455 ,  G06F16/14 ,  G06F16/182 ,  G06F16/9535 ,  H04L29/08 ,  G06F16/2453 ,  G06F16/23 ,  G06F16/951

Abstract: A method and apparatus managing a set of processors for a set of queries is described. In an exemplary embodiment, a device receives a set of queries for a data warehouse, the set of queries including one or more queries to be processed by the data warehouse. The device further provisions a set of processors from a first plurality of processors, where the set of processors to process the set of queries, and a set of storage resources to store data for the set of queries. In addition, the device monitors a utilization of the set of processors as the set of processors processes the set of queries. The device additionally updates a number of the processors in the set of processors provisioned based on the utilization/Furthermore, the device processes the set of queries using the updated set of processors.

公开(公告)号：US20210248160A1

公开(公告)日：2021-08-12

申请号：US17244578

申请日：2021-04-29

Applicant: SNOWFLAKE INC.

Inventor： Thierry Cruanes ,  Benoit Dageville ,  Marcin Zukowski

IPC: G06F16/27 ,  G06F9/50 ,  G06F16/14 ,  G06F16/21 ,  G06F16/22 ,  G06F16/951 ,  G06F16/182 ,  G06F16/23 ,  G06F16/2455 ,  G06F16/2458 ,  G06F16/9535 ,  G06F16/2453 ,  G06F9/48 ,  H04L29/08

Abstract: A method and apparatus managing a set of processors for a set of queries is described. In an exemplary embodiment, a device provision a set of computing resources of a database system, the set of computing resources to process a set of queries of the database system and determines a utilization of the set of computing resources during processing of the set of queries. The device further updates the set of computing resources based on the utilization of the set of computing resources by the set of queries. Updating the set of computing resources includes updating a number of processors and a set of storage resources to process the set of queries of the database system, the set of storage resources being shared by each of the processors and processes the set of queries using the set of computing resources as updated.

公开(公告)号：US20210200769A1

公开(公告)日：2021-07-01

申请号：US17249794

申请日：2021-03-12

Applicant: Snowflake Inc.

Inventor： Florian Andreas Funke ,  Thierry Cruanes ,  Benoit Dageville ,  Marcin Zukowski

IPC: G06F16/2453 ,  G06F16/22 ,  G06F16/2455

Abstract: Systems, methods, and devices, for managing data skew during a join operation are disclosed. A method includes computing a hash value for a join operation and detecting data skew on a probe side of the join operation at a runtime of the join operation using a lightweight sketch data structure. The method includes identifying a frequent probe-side join key on the probe side of the join operation during a probe phase of the join operation. The method includes identifying a frequent build-side row having a build-side join key corresponding with the frequent probe-side join key. The method includes asynchronously distributing the frequent build-side row to one or more remote servers.