System and method for dynamic log management of stream processing in a distributed environment
Abstract:
A system and method for dynamic log management of stream processing in a distributed computing environment, such as, for example, a streaming application or stream analytics system. A streaming application can be deployed or published to a cluster, to execute as a client application. A cluster manager coordinates with worker nodes, to commit tasks associated with the streaming application. If a need arises to generate lower-level log data associated with the streaming application, for example to diagnose an underlying cause of a warning/error message, a configuration job can be committed to the cluster to execute as a separate log-configuration application. The log-configuration application operates with the cluster manager to determine the set of working nodes currently associated with the streaming application, and modify the logger configuration at those nodes, to record or otherwise provide log data according to a modified logging level, for example to provide lower-level log messages.
Information query
Patent Agency Ranking
0/0