Yang Song, Anca Sailer, and HIdayatullah Shaikh
Problem determination and resolution PDR is the process of detecting anomalies in a monitored system, locating the problems responsible for the issue, determining the root cause and fixing the cause of the problem. The cost of PDR represents a substantial part of operational costs, and faster, more effective PDR can contribute to a substantial reduction in system administration costs. In this paper, we propose to automate the process of PDR by leveraging machine learning methods. The main focus is to effectively categorize the problem a user experiences by recognizing the problem specificity leveraging all available training data such like the performance data and the logs data. Specifically, we transform the structure of the problem into a hierarchy which can be determined by existing taxonomy in advance. We then propose an efficient hierarchical incremental learning algorithm which is capable of adjusting its internal local classifier parameters in real-time. Comparing to the traditional batch learning algorithms, this online learning framework can significantly decrease the computational complexity of the training process by learning from new instances on an incremental fashion. In the same time this reduces the amount of memory required to store the training instances. We demonstrate the efficiency of our approach by learning hierarchical problem patterns for several issues occurring in distributed web applications. Experimental results show that our approach substantially outperforms previ ous methods.
|Published in||the 11th IFIP/IEEE International Symposium on Integrated Network Management (IM 2009)|
© 2008 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. http://www.ieee.org/