Storage, the eternal quandary

LaCie's Bigger Disk is big alright, in both size (one terabyte unformatted) and form factor.

I was thinking about how far we've come in the last twenty years in terms of storage space and found this rather interesting historical perspective: Cost of Hard Drive Space.

Twenty years ago, a ten megabyte hark disk (which today would be large enough to almost hold a couple pictures from one of the newer digital cameras) cost roughly $1,500 (give or take). It wasn't until the following year that First Class Peripherals screwed everyone up by offering a 10 M drive (the Sider) for $700 (yeah, it was slow, but compared to the options of the day, it was a big win for people who needed *cough* mass storage — oh how things do change).

At a list price of $1,200, the biggest Bigger Disk is a 10,000:1 (plus a bit) improvement in cost per byte over that time which is quite amazing when you think about it.

We still have the same lurking problems (or I do anyway). How do you back it all up?

For a while in the early nineties, external storage had caught up with fixed media and it was relatively inexpensive to backup normal desktop system sized hard disks to magnetic optical and SyQuest cartridges. Tape backup finally seemed to get rolling with DLT in the mid nineties and while the cost was pretty steep for casual users, small offices could reliably back up somewhat large data sets (less than 40 G).

And since then? Not much as far as I can tell. VXA looked like it might useful, but the price point never moved and it's a single vendor technology. AIT drives which can handle about 70 G are available under $1,000, and there does seem to be downward price movement, but compared to hard disk prices, it's glacial.

Setting up home RAID arrays seems to be all the rage these days, because the cost of fixed media keeps plummeting. That works fine until you lose two disks in the array at the same time as happened recently to a friend. You still need a real backup somewhere and I just don't see how to do that effectively. You could just use mirroring I suppose.

Perhaps the answer is to split things up. A small data set that is extremely important would be backed up using more expensive techniques which ensure some sort of longevity. The larger set of temporary data (mostly media, like the 300 M movie animation of the MER landing) would be on big, semi disposable disks. If I lose it, oh well; I got it from the net, I can go find it again.

I still can't quite figure out the right approach, but I know from experience that it's going to get bigger.