Invention Grant
US08458163B2 System and method for enabling website owner to manage crawl rate in a website indexing system
有权
网站所有者在网站索引系统中管理爬网率的系统和方法
- Patent Title: System and method for enabling website owner to manage crawl rate in a website indexing system
- Patent Title (中): 网站所有者在网站索引系统中管理爬网率的系统和方法
-
Application No.: US13252149Application Date: 2011-10-03
-
Publication No.: US08458163B2Publication Date: 2013-06-04
- Inventor: Vanessa Fox , Amanda Ann Camp , Maximilian Ibel , Patrik Rene Celeste Reali , Jeremy J. Lilley , Katherine Jane Lai , Ted J. Bonkenburg , Neal Douglas Cardwell
- Applicant: Vanessa Fox , Amanda Ann Camp , Maximilian Ibel , Patrik Rene Celeste Reali , Jeremy J. Lilley , Katherine Jane Lai , Ted J. Bonkenburg , Neal Douglas Cardwell
- Applicant Address: US CA Mountain View
- Assignee: Google Inc.
- Current Assignee: Google Inc.
- Current Assignee Address: US CA Mountain View
- Agency: Morgan, Lewis & Bockius LLP
- Main IPC: G06F17/30
- IPC: G06F17/30
Abstract:
Web crawlers crawl websites to access documents of the website for purposes of indexing the documents for search engines. The web crawlers crawl a specified website at a crawl rate that is based on multiple factors. One of the factors is a pre-set crawl rate limit. According to certain embodiments, an owner for a specified website is enabled to modify the crawl rate limit for the specified website when one or more pre-set criteria are met.
Public/Granted literature
- US20120023091A1 System and Method for Enabling Website Owner to Manage Crawl Rate in a Website Indexing System Public/Granted day:2012-01-26
Information query
