Mining Invariants from Console Logs for System Problem Detection

Detecting execution anomalies is very important to the maintenance and monitoring of large-scale distributed systems. People often use console logs that are pro-duced by distributed systems for troubleshooting and problem diagnosis. However, manually inspecting con-sole logs for the detection of anomalies is unfeasible due to the increasing scale and complexity of distri-buted systems. Therefore, there is a great demand for automatic anomaly detection techniques based on log analysis. In this paper, we propose an unstructured log analysis technique for anomaly detection. In the tech-nique, we propose a novel algorithm to automatically discover program invariants in logs. At first, we covert the unstructured logs to structured logs through a log parsing step. Then, we group the log messages accord-ing to the relationship among log parameters. After that, we learn the program invariants from the log message groups. The mined invariants can reveal the inherent linear characteristics of program work flows. With these learned invariants, we can automatically detect anomalies in logs. Experiments on Hadoop show that the technique can effectively detect execution anomalies. Compared with the state of art approaches, our approach can not only detect numerous real problems with high accuracy but also provide intuitive insight to the problems.

Mining Invariants from Console Logs.pdf
PDF file

In  Annual Technical Conference (full paper)

Publisher  USENIX
All copyrights reserved by USENIX 2007

Details

TypeProceedings
> Publications > Mining Invariants from Console Logs for System Problem Detection