Invention Grant
- Patent Title: Indexing data sources using a highly available ETL for managed search
-
Application No.: US14985209Application Date: 2015-12-30
-
Publication No.: US10725993B1Publication Date: 2020-07-28
- Inventor: Swaminathan Sivasubramanian
- Applicant: Amazon Technologies, Inc.
- Applicant Address: US WA Seattle
- Assignee: Amazon Technologies, Inc.
- Current Assignee: Amazon Technologies, Inc.
- Current Assignee Address: US WA Seattle
- Agency: Hogan Lovells US LLP
- Main IPC: G06F16/22
- IPC: G06F16/22 ; G06F16/23 ; G06F16/25 ; G06F40/205

Abstract:
A managed search provider includes a highly available ETL service to index various data sources for searching. The ETL service can interface with various types of data sources associated with a user's account. When the ETL service receives a request to index a data source, the ETL service can extract a portion of data from the data source and analyze the portion of data to generate an index of the data source without requiring additional input from the user. The ETL service can store the index in a target data store identified in the request and determine whether the data source includes additional data to be indexed. As the data is indexed, the ETL service can maintain checkpoints in case of failure during indexing. Once the data source has been indexed, the ETL service can monitor the data source for changes made since the last indexing and can update the index accordingly.
Information query