From Paxos to CORFU: A Flash-Speed Shared Log

In a 2006 talk [11], Jim Gray predicted that “flash is the new disk”; five years later, indeed, flash is making headway into data centers, but it is usually disguised as standard block storage. We posit that new abstractions are required to exploit the potential of large-scale flash clusters. It has been observed that each individual flash drive is best utilized in a log-structured manner due to the intrinsic properties of flash. Further to that, in order to harness the aggregate bandwidth of a cluster of  flash units, we propose to organize the entire cluster as a single shared log. CORFU1 is a new storage system for flash clusters which demonstrates the feasibility of this approach. The key idea in CORFU is to expose a cluster of network-attached flash devices as a single, shared log to clients running within the data center. Applications running on the clients can append data to this log or read entries from its middle.