Abstract:
Systems and methods for enabling automated log analysis with controllable resource requirements are provided. A training set for log pattern learning is generated based on heterogeneous logs generated by a computer system. An incremental learning process is implemented to generate a set of log patterns from the training set. The heterogeneous logs are parsed using the set of log patterns. A set of applications is applied to the parsed logs.
Abstract:
A method is provided that includes transforming training data into a neural network based learning model using a set of temporal graphs derived from the training data. The method includes performing model learning on the learning model by automatically adjusting learning model parameters based on the set of the temporal graphs to minimize differences between a predetermined ground-truth ranking list and a learning model output ranking list. The method includes transforming testing data into a neural network based inference model using another set of temporal graphs derived from the testing data. The method includes performing model inference by applying the inference and learning models to test data to extract context features for alerts in the test data and calculate a ranking list for the alerts based on the extracted context features. Top-ranked alerts are identified as critical alerts. Each alert represents an anomaly in the test data.
Abstract:
A heterogeneous log pattern editing recommendation system and computer- implemented method are provided. The system (600) has a processor (605) configured to identify, from heterogeneous logs, patterns including variable fields and constant fields. The processor (605) is also configured to extract a category feature, a cardinality feature, and a before-after n-gram feature by tokenizing the variable fields in the identified patterns. The processor (605) is additionally configured to generate target similarity scores between target fields to be potentially edited and other fields from among the variable fields in the heterogeneous logs using pattern editing operations based on the extracted category feature, the extracted cardinality feature, and the extracted before-after n-gram feature. The processor (605) is further configured to recommend, to a user, log pattern edits for at least one of the target fields based on the target similarity scores between the target fields in the heterogeneous logs.
Abstract:
Systems and methods are disclosed for parsing logs from arbitrary or unknown systems or applications by capturing heterogeneous logs from the arbitrary or unknown systems or applications; generating one pattern for every unique log message; building a pattern hierarchy tree by grouping patterns based on similarity metrics, and for every group it generates one pattern by combing all constituting patterns of that group; and selecting a set of patterns from the pattern hierarchy tree.
Abstract:
Systems and methods for controlling legacy switch routing in one or more hybrid networks of interconnected computers and switches, including generating a network underlay (304) for the one or more hybrid networks by generating a minimum spanning tree (MST) (306) and a forwarding graph (FWG) (308) over a physical network topology of the one or more hybrid networks (400), determining an optimal path between hosts on the FWG by optimizing an initial path with a minimum cost mapping (312), and adjusting the initial path (310) to enforce the optimal path (314) by generating and installing special packets in one or more programmable switches to trigger installation of forwarding rules for one or more legacy switches (516).
Abstract:
Systems and methods for decoupled searching and optimization for one or more data centers, including determining a network topology for one or more networks of interconnected computer systems embedded in the one or more data centers (304), searching for routing candidates based on a network topology determined (310), and updating (314) and applying (316) one or more objective functions to the routing candidates to determine an optimal routing candidate to satisfy embedding goals based on tenant requests, and to embed the optimal routing candidate in the one or more data centers (412).
Abstract:
Systems and methods for implementing content-level anomaly detection for devices having limited memory are provided. At least one log content model is generated (130) based on training log content of training logs obtained from one or more sources associated with the computer system. The at least one log content model is transformed (140) into at least one modified log content model to limit memory usage. Anomaly detection is performed (170) for testing log content of testing logs obtained from one or more sources associated with the computer system based on the at least one modified log content model. In response to the anomaly detection identifying one or more anomalies associated with the testing log content, the one or more anomalies are output (170).
Abstract:
A method for implementing automatic and scalable log pattern learning in security log analysis is provided. The method includes collecting security logs generated by a computer system. An incremental learning process is implemented to generate a set of log patterns from the collected security logs. The collected security logs are parsed using the set of log patterns.
Abstract:
A security system using automatic and scalable log pattern learning in security log analysis is provided. The security system includes one or more management services configured to generate security logs, and a security log analysis service operatively coupled to the one or more management services. The security log analysis service is configured to collect the security logs generated by the one or more management services, implement an incremental learning process to generate a set of log patterns from the collected security logs, parse the collected security logs using the set of log patterns, and analyze the parsed security logs for one or more security applications.
Abstract:
A computer-implemented method, computer program product, and computer processing system are provided. The method includes preprocessing, by a processor, a set of heterogeneous logs by splitting each of the logs into tokens to obtain preprocessed logs. Each of the logs in the set is associated with a timestamp and textual content in one or more fields. The method further includes generating, by the processor, a set of regular expressions from the preprocessed logs. The method also includes performing, by the processor, an unsupervised parsing operation by applying the regular expressions to the preprocessed logs to obtain a set of parsed logs and a set of unparsed logs, if any. The method additionally includes storing, by the processor, the set of parsed logs in a log analytics database and the set of unparsed logs in a debugging database.