Invention Grant
- Patent Title: Certifying events in a streaming pipeline
-
Application No.: US16881751Application Date: 2020-05-22
-
Publication No.: US11947542B2Publication Date: 2024-04-02
- Inventor: Michael Pippin , David Willcox , Allie K. Watfa , George Aleksandrovich
- Applicant: YAHOO ASSETS LLC
- Applicant Address: US VA Dulles
- Assignee: YAHOO ASSETS LLC
- Current Assignee: YAHOO ASSETS LLC
- Current Assignee Address: US NY New York
- Agency: Greenberg Traurig, LLP
- Agent James J. DeCarlo
- Main IPC: G06F16/24
- IPC: G06F16/24 ; G06F9/54 ; G06F16/2455

Abstract:
Disclosed are embodiments for providing batch performance using a stream processor. In one embodiment, a method is disclosed comprising detecting a real close of books (COB) of a data transport, the real COB associated with a set of raw events transmitted over the data transport, flushing a stream processor in response to detecting the real COB, and retrieving a set of processed events from a distributed file system after the flushing is complete. A fact COB computation is then performed on the set of processed events and the set of raw events, the fact COB computation outputting a number of missing events, each missing event representing a raw event that is not present in the set of processed events. The processed events are then certified upon determining that the number of missing events is below a threshold.
Public/Granted literature
- US20210365459A1 CERTIFYING EVENTS IN A STREAMING PIPELINE Public/Granted day:2021-11-25
Information query