公开(公告)号：US11080345B2

公开(公告)日：2021-08-03

申请号：US16264462

申请日：2019-01-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Pride ,  Arindam Bhattacharjee ,  Xiaowei Wang ,  James Alasdair Robert Hodge ,  Mustafa Ahamed

IPC: G06F16/00 ,  G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: Disclosed is a technique that can be performed in a distributed computer network. The technique can include a worker node that receives search instructions defined by a search service based on at least a portion of a search scheme defined by a data intake and query system, to cause the worker node to obtain search results from distributed data storage systems communicatively coupled to the worker node over a network. The distributed data storage systems include an external data storage system and/or an internal data storage system of the data intake and query system. The worker node obtains the search results by searching the distributed data storage systems in accordance with the search instructions, and communicating, over the network to the search service, a combination of search results based on the search results to cause an output by the data intake and query system in accordance with the search scheme.

公开(公告)号：US10896182B2

公开(公告)日：2021-01-19

申请号：US15714029

申请日：2017-09-25

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Christopher Pride

IPC: G06F16/00 ,  G06F16/2455 ,  G06F11/30 ,  G06F7/53 ,  G06F16/27 ,  G06F11/34

Abstract: Systems and methods are disclosed for processing and executing queries against one or more dataset. As part of processing the query, the system determines whether the query is susceptible to a significantly imbalanced partition. In the event, the query is susceptible to an imbalanced partition, the system monitors the query and determines whether to perform a multi-partitioning determination to avoid a significantly imbalanced partition.

公开(公告)号：US20200081891A1

公开(公告)日：2020-03-12

申请号：US16687158

申请日：2019-11-18

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Ashish Mathew ,  Xiaowei Wang ,  Christopher Pride

IPC: G06F16/2455 ,  G06F16/951 ,  G06F16/248

Abstract: The disclosed embodiments include a method performed by a data intake and query system. The method includes receiving a search query by a search head, defining a search process for applying the search query to indexers, delegating a first portion of the search process to indexers and a second portion of the search process to intermediary node(s) communicatively coupled to the search head and the indexers. The first portion can define a search scope for obtaining partial search results of the indexers and the second portion can define operations for combining the partial search results by the intermediary node(s) to produce a combination of the partial search results. The search head then receives the combination of the partial search results, and outputs final search results for the search query, where the final search results are based on the combination of the partial search results.

公开(公告)号：US20180089278A1

公开(公告)日：2018-03-29

申请号：US15665197

申请日：2017-07-31

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Alexander Douglas James ,  Christopher Pride

IPC: G06F17/30 ,  H04L12/26 ,  G06F11/20 ,  H04L29/06 ,  H04L29/08

CPC classification number: G06F16/24568 ,  G06F3/0617 ,  G06F3/065 ,  G06F11/20 ,  G06F11/3409 ,  G06F16/2471 ,  G06F16/27 ,  G06F16/278 ,  G06F16/90335 ,  G06F16/951 ,  H04L43/028 ,  H04L43/08 ,  H04L43/12 ,  H04L43/14 ,  H04L67/1097 ,  H04L69/22

Abstract: Systems and methods are disclosed for processing queries against one or more dataset sources utilizing dynamically allocated partitions operating on one or more worker nodes. The results of the processing are stored in a dataset destination. The queries can identify data in the one or more dataset sources for processing and a manner for processing the data. In addition, the queries can identify the dataset destination for storing results of the query. To process the query, a query coordinator can dynamically allocate partitions operating on worker nodes to retrieve data for processing, process the data, and communicate the data to the dataset sources. In addition, the query coordinator can dynamically allocate partitions based on an identification of the dataset destination.

公开(公告)号：US12072891B1

公开(公告)日：2024-08-27

申请号：US18180728

申请日：2023-03-08

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Ashish Mathew ,  Xiaowei Wang ,  Christopher Pride

IPC: G06F16/2455 ,  G06F16/22 ,  G06F16/2453 ,  G06F16/2458 ,  G06F16/248 ,  G06F16/951

CPC classification number: G06F16/24564 ,  G06F16/22 ,  G06F16/24532 ,  G06F16/2471 ,  G06F16/248 ,  G06F16/951

Abstract: The disclosed embodiments include a method performed by a data intake and query system. The method includes receiving a search query by a search head, defining a search process for applying the search query to indexers, delegating a first portion of the search process to indexers and a second portion of the search process to intermediary node(s) communicatively coupled to the search head and the indexers. The first portion can define a search scope for obtaining partial search results of the indexers and the second portion can define operations for combining the partial search results by the intermediary node(s) to produce a combination of the partial search results. The search head then receives the combination of the partial search results, and outputs final search results for the search query, where the final search results are based on the combination of the partial search results.

公开(公告)号：US11151137B2

公开(公告)日：2021-10-19

申请号：US15713976

申请日：2017-09-25

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Christopher Pride

IPC: G06F16/00 ,  G06F16/2455 ,  G06F7/53 ,  G06F16/27 ,  G06F3/06

Abstract: In an environment where multiple datasets are to be combined, systems and methods are disclosed for allocating a group of data entries from at least one dataset into multiple partitions. For a particular partition, the subgroup in the partition can be combined with data entries from the other dataset. In some cases, groups of data entries from each dataset are assigned to different partitions. For a particular partition, a subgroup is duplicated, some of the data entries of the subgroup are reassigned to other partitions, the subgroup is reformed to include data entries from other partitions, and the reformed subgroup is combined with the subgroup from the other dataset(s).

公开(公告)号：US11023539B2

公开(公告)日：2021-06-01

申请号：US16264430

申请日：2019-01-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Pride ,  Arindam Bhattacharjee ,  Xiaowei Wang ,  James Alasdair Robert Hodge ,  Mustafa Ahamed

IPC: G06F16/00 ,  G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: Disclosed is a technique that can be performed in a distributed computer network. The technique can include a data index and query system that receives a search query and defines a search scheme for applying the search query on distributed data storage systems including an internal data storage system of the data intake and query system and an external data storage system communicatively coupled to the data intake and query system over a network. The data index and query system communicates at least a portion of the search scheme to a search service for application on behalf of the data intake and query system, receives from the search service a search result of the search query obtained by application of the search scheme to the distributed data storage systems, and causes the search result or data indicative thereof to be displayed on a display device.

公开(公告)号：US10726009B2

公开(公告)日：2020-07-28

申请号：US15665148

申请日：2017-07-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Arindam Bhattacharjee ,  Christopher Pride

IPC: G06F16/2453 ,  G06F16/25 ,  G06F16/2455

Abstract: Systems and methods are disclosed for processing queries against one or more dataset sources. The system tracks query resource data and resource utilization data. The query-resource usage data can indicate resources used to execute queries. The node resource utilization data can indicate current utilization of nodes in the system. Upon receipt of a query that identifies a set of data to be processed and a manner of processing the set of data, the system can use the query-resource usage data and the resource utilization data to define a query processing scheme. The query can then be executed using the query processing scheme. In some cases, the query coordinator can dynamically allocate partitions operating on worker nodes to execute the query.

公开(公告)号：US10585951B2

公开(公告)日：2020-03-10

申请号：US15339833

申请日：2016-10-31

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Christopher Pride

IPC: G06F16/00 ,  G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903 ,  G06F16/248 ,  G06F16/2458 ,  G06F16/27 ,  G06F16/2455

Abstract: The disclosed embodiments include techniques to obtain ordered search results based on partial search results from across multiple diverse internal and/or external data sources. The ordering of the search results may be with respect to a parameter associated with the partial search results. An example of a parameter includes time. As such, the disclosed technique can provide a time-ordered search result based on partial search results obtained from across multiple internal and/or external data sources. Moreover, the disclosed technique can provide time-ordered search results regardless of whether the partial search results obtained from the diverse data sources are timestamped.

公开(公告)号：US10353965B2

公开(公告)日：2019-07-16

申请号：US15276717

申请日：2016-09-26

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Pride ,  Arindam Bhattacharjee ,  Xiaowei Wang ,  James Alasdair Robert Hodge ,  Mustafa Ahamed

IPC: G06F16/00 ,  G06F16/951 ,  G06F16/21 ,  G06F16/25 ,  G06F16/904 ,  G06F16/901 ,  G06F16/9038 ,  G06F16/903

Abstract: Disclosed is a technique that can be performed in a distributed computer network. The technique can include a data index and query system that receives search query, defines a search scheme for applying the search query on distributed data storage systems including an internal data storage system of the data index and query system and an external data storage system. The internal data storage system stores data as time-indexed events including respective segments of raw machine data. The data index and query system can transfer a portion of the search scheme to a search service, which can return search results obtained by application of the search scheme to the distributed data storage systems including the internal data storage system and the external data storage system. Lastly, the search results or data indicative of the search results can be output on a display device to the user.