Invention Grant
- Patent Title: Distributed method and apparatus for processing streaming data
-
Application No.: US15281273Application Date: 2016-09-30
-
Publication No.: US10313430B2Publication Date: 2019-06-04
- Inventor: Yao Xu , Cong Wang , Yuncong Zhang , Jianwei Zhang , Xin Huang
- Applicant: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.
- Applicant Address: CN Beijing
- Assignee: Baidu Online Network Technology (Beijing) Co., Ltd.
- Current Assignee: Baidu Online Network Technology (Beijing) Co., Ltd.
- Current Assignee Address: CN Beijing
- Agency: Miller, Matthias & Hull LLP
- Priority: CN201610465909 20160623
- Main IPC: H04L29/06
- IPC: H04L29/06 ; H04L29/08

Abstract:
A distributed method and apparatus for processing streaming data are disclosed. A specific implementation of the method includes: encapsulating received streaming data as a first resilient distributed dataset; performing a grouping operation on the first resilient distributed dataset based on time windows, the grouping operation comprising: assigning each data element in the first resilient distributed dataset into a group corresponding to a time window to which a recorded timestamp of the data element belongs, and forming second resilient distributed datasets comprising a limited number of data elements and respectively corresponding to the time windows; encapsulating the second resilient distributed datasets as a nested dataset comprising a plurality of the second resilient distributed datasets; passing, using a predefined traversal operator, each of the second resilient distributed datasets in the nested dataset successively to a batch operator defined in a finite dataset to perform distributed data processing.
Public/Granted literature
- US20170374137A1 DISTRIBUTED METHOD AND APPARATUS FOR PROCESSING STREAMING DATA Public/Granted day:2017-12-28
Information query