[Openais] announcement of the vinzvault project

[Steven Dake]

CCing openais and linux-cluster ml since there may be some cluster
developers interested in participating in this project there.

I am pleased to announce we are starting a new project called vinzvault
to help resolve some of the difficulties in deploying virtual machines
in data-centers.

There are other projects that use similar technology or have similar
goals as ours.  The Ceph filesystem provides a cloud file-system for
large scale machines to use as storage.  Hail provides a S3 API for
accessing information.  Cassandra provides a distributed database using
techniques similar to what we are planning to provide eventually
consistent replicated bigtable style databases.

Our project is focused around one goal: providing a small footprint
(10kloc) highly available block storage area for virtual machines
optimized for Linux data-centers.  Our plans don't depend on SAN
hardware, software, hardware fencing devices, or any other hardware then
is commonly available on commodity hardware.   We intend to trade these
lower-scale high cost technologies for higher-scale lower cost
techniques.

Some of our requirements:
* Easy to use, deploy, and manage.
* 100,000 host count scalability.
* Only depend on commodity hardware systems.
* Migration works seamlessly within a datacenter without SAN hardware.
* VM block images can be replicated to N where N is configurable per VM
image.
* VM block images can be replicated to various data centers.
* Low latency block storage access for all VMs.
* Tuneable block sizes per VM.
* Use standard network mechanisms to transmit blocks to the various
replicas.
* Avoid multicast.
* Ensure only authorized host machines may connect to the vinzvault
storage areas.
* No central metadata server - everything is 100% distributed.

We plan to execute this project using an overlay DHT hash table called
D1HT(1).  The 1 in D1HT indicates there is, in a majority of cases, only
1 network request/response required per block of storage.  Like all
solutions that trade performance for scale/cost, our project may not
meet your deployment needs, but we aim to focus on correctness first and
performance second.  We hope readers will participate in the development
of this LGPL/GPL open source project.

Our mailing list is vinzvault at fedorahosted.org.  

One final note - no code is in our repo yet - that is for developers
interested in this technology to make happen (this is a from scratch
implementation).  Lets get cracking!

Regards
-steve

(1) http://www.cos.ufrj.br/~monnerat/D1HT_paper.html