Invention Grant
- Patent Title: System and method for URL fetching retry mechanism
-
Application No.: US16938991Application Date: 2020-07-26
-
Publication No.: US11657110B2Publication Date: 2023-05-23
- Inventor: Derry Shribman , Ofer Vilenski
- Applicant: BRIGHT DATA LTD
- Applicant Address: IL Netanya
- Assignee: BRIGHT DATA LTD.
- Current Assignee: BRIGHT DATA LTD.
- Current Assignee Address: IL Netanya
- Agency: May Patents Ltd. c/o Dorit Shem-Tov
- Main IPC: G06F16/909
- IPC: G06F16/909 ; G06F16/955 ; G06F9/54 ; H04L67/02 ; H04L67/146 ; H04L1/00 ; H04L69/40 ; H04L67/01 ; H04L67/568 ; H04L61/59 ; H04L67/56

Abstract:
A method for overcoming intermittent, temporary, or other fetching failures by using multiple attempts for retrieving a content from a web server to a client device is disclosed. The URL fetching may use direct or non-direct fetching schemes, or a combination thereof. The non-direct fetching method may use intermediate devices, such as proxy server, Data-Center proxy server, tunnel devices, or any combination thereof. Upon sensing a failure of a fetching action, the action is repeated using the same or different parameters or attributes, such as by using different intermediate devices, selected based on different parameters or attributes, such as different countries. The repetitions are limited to a pre-defined maximum number or attempts. The fetching attempts may be performed by the client device, by an intermediate device in a non-direct fetching scheme, or a combination thereof. Various fetching schemes may be used sequentially until the content is retrieved.
Information query