Invention Grant
- Patent Title: Hybrid queue system for request throttling
-
Application No.: US15605775Application Date: 2017-05-25
-
Publication No.: US10944683B1Publication Date: 2021-03-09
- Inventor: James Roskind
- Applicant: Amazon Technologies, Inc.
- Applicant Address: US WA Seattle
- Assignee: Amazon Technologies, Inc.
- Current Assignee: Amazon Technologies, Inc.
- Current Assignee Address: US WA Seattle
- Agency: Kowert, Hood, Munyon, Rankin & Goetzel, P.C.
- Agent Robert C. Kowert
- Main IPC: H04L12/873
- IPC: H04L12/873 ; H04L12/825 ; H04L12/851

Abstract:
Systems and related methods are disclosed to store and throttle requests received by a service provider. In embodiments, the system includes two queues, a first-in-first-out (FIFO) queue and an overflow queue. An incoming request is stored in the overflow queue when there is no room in the FIFO queue. The overflow queue stores the requests in some priority order, which determines the order that the requests are promoted onto the FIFO queue and throttled. The FIFO queue may be sized according to a response time requirement provided in a service level agreement (SLA). In some embodiments, the FIFO queue may dynamically adjust its size based on the expected processing time or abandon duration of incoming requests. The hybrid approach allows a system to handle requests in simple FIFO order in normal circumstances, and in a more sophisticated priority order when the system is overloaded.
Information query