BitVault: a Highly Reliable Distributed Data Retention Platform

In this paper, we report the design and implementation of the storage layer of BitVault: a content-addressable retention platform for large volume of reference data – seldom-changing information that needs to be retained for a long period of time. BitVault uses “smart brick” as the building block to lower the hardware cost. However, the challenges are to maintain low management cost in a system that needs to scale all the way from one brick to tens of thousands of bricks, to ensure reliability and to deliver with a simple enough design. Our design incorporates P2P technologies for its self-managing and self-healing capabilities and uses massively parallel repair to reduce vulnerability window of data loss. The simplicity of the architecture relies on an eventu-ally reliable membership service provided by a perfect one-hop DHT (distributed hash table), and its object-driven repair model yields last-copy recall guarantee: independent of how many other failures that may occur and their sequences, as long as the last copy of a data object still remains in the system, the data can be retrieved and its replication degree fully restored. A proto-type has been implemented. Theoretical analysis, simulations and experiments are conducted to validate the design of BitVault.

tr-2005-179.pdf
PDF file

Publisher  Association for Computing Machinery, Inc.
Copyright © 2004 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Publications Dept, ACM Inc., fax +1 (212) 869-0481, or permissions@acm.org. The definitive version of this paper can be found at ACM’s Digital Library –http://www.acm.org/dl/.

Details

TypeInproceedings
URLhttp://www.acm.org/
Pages15
NumberMSR-TR-2005-179
InstitutionMicrosoft Research
> Publications > BitVault: a Highly Reliable Distributed Data Retention Platform