Distributed streaming platform for real-time applications
Abstract:
Software for a distributed streaming platform receives a specification for a streaming application and converts the specification into a logical plan that includes a directed acyclic graph (DAG) with operators as nodes and streams as edges. The software translates the logical plan into a physical plan using any stream modes as specified and with one or more instances of the operators per any static partitioning. Then the software obtains a plurality of containers running on a networked cluster of servers. One slave executes in each container. The software assigns the instances to the slaves for execution according to an execution plan that depends at least in part on the physical plan and number of containers obtained. Each slave monitors throughput of the streaming windows through the instances in the container and other statistics related to service level and reports the results of the monitoring, which are used for dynamic adjustment.
Information query
Patent Agency Ranking
0/0