Abstract:
Systems and methods for enabling automated log analysis with controllable resource requirements are provided. A training set for log pattern learning is generated based on heterogeneous logs generated by a computer system. An incremental learning process is implemented to generate a set of log patterns from the training set. The heterogeneous logs are parsed using the set of log patterns. A set of applications is applied to the parsed logs.
Abstract:
A heterogeneous log pattern editing recommendation system and computer- implemented method are provided. The system (600) has a processor (605) configured to identify, from heterogeneous logs, patterns including variable fields and constant fields. The processor (605) is also configured to extract a category feature, a cardinality feature, and a before-after n-gram feature by tokenizing the variable fields in the identified patterns. The processor (605) is additionally configured to generate target similarity scores between target fields to be potentially edited and other fields from among the variable fields in the heterogeneous logs using pattern editing operations based on the extracted category feature, the extracted cardinality feature, and the extracted before-after n-gram feature. The processor (605) is further configured to recommend, to a user, log pattern edits for at least one of the target fields based on the target similarity scores between the target fields in the heterogeneous logs.
Abstract:
Systems and methods are disclosed for parsing logs from arbitrary or unknown systems or applications by capturing heterogeneous logs from the arbitrary or unknown systems or applications; generating one pattern for every unique log message; building a pattern hierarchy tree by grouping patterns based on similarity metrics, and for every group it generates one pattern by combing all constituting patterns of that group; and selecting a set of patterns from the pattern hierarchy tree.
Abstract:
A method for implementing automatic and scalable log pattern learning in security log analysis is provided. The method includes collecting security logs generated by a computer system. An incremental learning process is implemented to generate a set of log patterns from the collected security logs. The collected security logs are parsed using the set of log patterns.
Abstract:
A security system using automatic and scalable log pattern learning in security log analysis is provided. The security system includes one or more management services configured to generate security logs, and a security log analysis service operatively coupled to the one or more management services. The security log analysis service is configured to collect the security logs generated by the one or more management services, implement an incremental learning process to generate a set of log patterns from the collected security logs, parse the collected security logs using the set of log patterns, and analyze the parsed security logs for one or more security applications.
Abstract:
Methods and systems for log management include pre-processing heterogeneous logs and performing a log management action (112) on the pre-processed plurality of heterogeneous logs. Pre-processing the logs includes performing a fixed tokenization (104) of the heterogeneous logs based on a predefined set of symbols, performing a flexible tokenization (106) of the heterogeneous logs based on a user-defined set of rules, converting timestamps (108) in the heterogeneous logs to a single target timestamp format, and performing structural log tokenization (110) of the heterogeneous logs based on user-defined structural information.
Abstract:
Mobile phones and methods for mobile phone failure prediction include receiving respective log files from one or more mobile phone components, including at least one user application. The log files have heterogeneous formats. A likelihood of failure of one or more mobile phone components is determined based on the received log files by clustering the plurality of log files according to structural log patterns and determining feature representations of the log files based on the log clusters. A user is alerted to a potential failure if the likelihood of component failure exceeds a first threshold. An automatic system control action is performed if the likelihood of component failure exceeds a second threshold.
Abstract:
A method is provided that is performed in a network having nodes that generate heterogeneous logs including performance logs and text logs. The method includes performing, during a heterogeneous log training stage, (i) a log-to-time sequence conversion process for transforming clustered ones of training logs, from among the heterogeneous logs, into a set of time sequences that are each formed as a plurality of data pairs of a first configuration and a second configuration based on cluster type, (ii) a time series generation process for synchronizing particular ones of the time sequences in the set based on a set of criteria to output a set of fused time series, and (iii) an invariant model generation process for building invariant models for each time series data pair in the set of fused time series. The method includes controlling an anomaly-initiating one of the plurality of nodes based on the invariant models.
Abstract:
An exemplary method for detecting one or more anomalies in a system includes building a temporal causality graph describing functional relationship among local components in normal period; applying the causality graph as a propagation template to predict a system status by iteratively applying current system event signatures; and detecting the one or more anomalies of the system by examining related patterns on the template causality graph that specifies normal system behaviors. The system can aligning event patterns on the causality graph to determine an anomaly score.
Abstract:
Systems and methods are disclosed for analyzing logs generated by a machine by analyzing a log and identifying one or more abstract landmark delimiters (ALDs) representing delimiters for log tokenization; from the log and ALD, tokenizing the log and generating an increasingly tokenized format by separating the patterns with the ALD to form an intermediate tokenized log; iteratively repeating the tokenizing of the logs until a last intermediate tokenized log is processed as a final tokenized log; and applying the tokenized logs in applications.