[Openais] announcement of the vinzvault project
sdake at redhat.com
Tue Apr 27 16:36:18 PDT 2010
CCing openais and linux-cluster ml since there may be some cluster
developers interested in participating in this project there.
I am pleased to announce we are starting a new project called vinzvault
to help resolve some of the difficulties in deploying virtual machines
There are other projects that use similar technology or have similar
goals as ours. The Ceph filesystem provides a cloud file-system for
large scale machines to use as storage. Hail provides a S3 API for
accessing information. Cassandra provides a distributed database using
techniques similar to what we are planning to provide eventually
consistent replicated bigtable style databases.
Our project is focused around one goal: providing a small footprint
(10kloc) highly available block storage area for virtual machines
optimized for Linux data-centers. Our plans don't depend on SAN
hardware, software, hardware fencing devices, or any other hardware then
is commonly available on commodity hardware. We intend to trade these
lower-scale high cost technologies for higher-scale lower cost
Some of our requirements:
* Easy to use, deploy, and manage.
* 100,000 host count scalability.
* Only depend on commodity hardware systems.
* Migration works seamlessly within a datacenter without SAN hardware.
* VM block images can be replicated to N where N is configurable per VM
* VM block images can be replicated to various data centers.
* Low latency block storage access for all VMs.
* Tuneable block sizes per VM.
* Use standard network mechanisms to transmit blocks to the various
* Avoid multicast.
* Ensure only authorized host machines may connect to the vinzvault
* No central metadata server - everything is 100% distributed.
We plan to execute this project using an overlay DHT hash table called
D1HT(1). The 1 in D1HT indicates there is, in a majority of cases, only
1 network request/response required per block of storage. Like all
solutions that trade performance for scale/cost, our project may not
meet your deployment needs, but we aim to focus on correctness first and
performance second. We hope readers will participate in the development
of this LGPL/GPL open source project.
Our mailing list is vinzvault at fedorahosted.org.
One final note - no code is in our repo yet - that is for developers
interested in this technology to make happen (this is a from scratch
implementation). Lets get cracking!
More information about the Openais