Daily Cup of Tech

I once owned a dog that would bury bones everywhere it could think of. And it drove us nuts. Just when we thought we had gotten rid of all its skeletal stash, he would show up with another one!

In a way, this is what Cleversafe intends to do with data. They have published Create your own Dispersed Storage Grid, a guide for creating a dispersal grid where data is duplicated over a number of systems in such a way that if a percentage of the systems are down or unavailable, all of your data is still accessible.

From their website:

The Dispersed Storage Project is the central point of development and idea exchange for developers around the world to contribute to innovative storage solutions leveraging dispersed storage methodology.

The project uses information dispersal algorithms IDAs to separate data into 11 unrecognizable DataSlices™ and distribute them, via secure Internet connections, to 11 storage locations throughout the world, creating a storage grid. With dispersed storage, transmission and storage of data is inherently private and secure. No single entire copy of the data is in one location, and only 6 out of the 11 nodes need to be available in order to perfectly retrieve the data.

Data on the grid remains private and secure in the face of natural catastrophes, or failures of hardware, connection, facility, or IT management. Moreover, the individual data slices do not carry enough information for an unauthorized viewer to determine the original content.

The Cleversafe Dispersed Storage software includes client software with both a comprehensive command line interface (CLI) as well as a complete programming interface (DSAPI) to support any type of storage application. This software further includes grid server software for creating a dispersed storage grid. The Cleversafe software also manages metadata for file systems stored on a Dispersed Storage grid. In addition, this project includes the multi-terabyte Cleversafe Research Storage Grid at eleven separate hosting facilities.

This open source project is the first time information dispersal has been used with a Grid Design to store the world’s data.

This sounds a lot like a RAID configuration but on the system level instead of on the hard drive level. This is something that could be really beneficial where WAN links are unreliable.

Trackback link - http://www.dailycupoftech.com/2006/12/17/keep-data-safe-by-storing-it-everywhere/trackback/