StrobeLight: Lightweight Availability Mapping and Anomaly Detection

Large-scale distributed systems span thousands of independent hosts which come online and go offline at their users’ whim. Such availability flux is ostensibly a key concern when systems are designed, but this flux is rarely measured in a rich way post-deployment, either by the distributed system itself or by a standalone piece of infrastructure. In this paper we introduce StrobeLight, a tool for monitoring per-host availability trends in enterprise settings. Every 30 seconds, StrobeLight probes Microsoft’s entire corporate network, archiving the ping results for use by other networked services. We describe two such services, one offline and the other online. The first service uses longitudinal data collected by our StrobeLight deployment to analyze large-scale trends in our wired and wireless networks. The second service draws live StrobeLight measurements to detect network anomalies like IP hijacking in real time. StrobeLight is easy to deploy, requiring neither modification to end hosts nor changes to the core routing infrastructure. Furthermore, it requires minimal network and CPU resources to probe our network of over 200,000 hosts.

usenix09-camera-ready.pdf
PDF file

In  Proceedings of USENIX Technical

Publisher  USENIX
All copyrights reserved by USENIX 2009

Details

TypeInproceedings
> Publications > StrobeLight: Lightweight Availability Mapping and Anomaly Detection