Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Brief Announcement: Flash-Log -- A High Throughput Log

Mahesh Balakrishnan, Phil Bernstein, Dahlia Malkhi, Vijayan Prabhakaran, and Colin Reid

Abstract

With today’s advanced storage technology, e.g., flash memory devices, it takes many compute nodes accessing storage directly over the network to utilize the device’s high throughput. We devise Flash-Log, a reliable, persistent and totally-ordered log which uses network-attached storage devices directly, and alleviates any bottleneck servers in between. There is also a scale-out benefit: To grow the system, one simply adds more compute nodes to scale up throughput. The log provides compute nodes with the ability to append multi-page records and to read those records. Appended records are totally ordered, and append and read operations are reliable and provide strict atomicity guarantees. Append-record operations are made atomic in the face of failures by committing provisional append-ordering information onto a log. We present two protocols, both of which are novel variants of Paxos. Unlike Paxos, storage segments serve not only to store the log, but also to persistently store information on the total ordering of transactions. The core difficulty is to implement certain functionalities using segments that support only simple operations that are appropriate for implementation in a device controller. This is particularly challenging when dealing with autonomous configuration management: Our design automatically migrates to new storage devices in response to segment fill-up or to a failure.

Details

Publication typeInproceedings
Published in24th International Symposium on Distributed Computing (DISC 2010)
URLhttp://rd.springer.com/chapter/10.1007/978-3-642-15763-9_39
PublisherSpringer Verlag
> Publications > Brief Announcement: Flash-Log -- A High Throughput Log