Invention Grant
- Patent Title: Adaptive crawl rates based on publication frequency
- Patent Title (中): 基于出版频率的自适应爬网率
-
Application No.: US13053772Application Date: 2011-03-22
-
Publication No.: US08255385B1Publication Date: 2012-08-28
- Inventor: Walter Sun , Yipeng Li , Xiao Zhang , Junaid Ahmed
- Applicant: Walter Sun , Yipeng Li , Xiao Zhang , Junaid Ahmed
- Applicant Address: US WA Redmond
- Assignee: Microsoft Corporation
- Current Assignee: Microsoft Corporation
- Current Assignee Address: US WA Redmond
- Agency: Shook Hardy & Bacon L.L.P.
- Main IPC: G06F7/00
- IPC: G06F7/00 ; G06F17/30
Abstract:
Methods and systems for determining an adaptive crawl rate for a Web crawler based on historical publication data from a Web source are provided. A frequency of publication of the Web source is determined over a specified period of time, and an adaptive crawl rate is calculated using the frequency of publication. The Web crawler is then deployed at the calculated adaptive crawl rate.
Information query
