AVS › AVS Forum › Video Components › Home Theater Computers › Automatic/unattended backup og 1-2 TB
New Posts  All Forums:Forum Nav:

Automatic/unattended backup og 1-2 TB

post #1 of 33
Thread Starter 
Like many others, I have a large RAID array in the basement, which I use as media server. While RAID ensures high availability, it does not eliminate the need for backups. While it is painful enough to lose ripped DVDs, family photos and videos are irreplaceable and therefore invaluable!

But how on earth do you backup 1-2 Terabytes without hassle??? Certainly, I don't want to spend my weekends juggling DVDs in the basement! After some research, I have come to the conclusion that - if I really want a zero-hassle, wholly unattended solution - I may need a tape autochanger, like an Exabyte Packetloader. It would set me back 2500$, which is painful, but it might give me ultimate peace of mind.

I would be grateful for any input about alternatives. Possibly less expensive ones! Also, my strategy would be to run incremental backups, which would allow for versioning of files, and therefore highest reliability, and yet wouldnt consume huges amounts of space like image files. Am I right?
post #2 of 33
Thread Starter 
I might add that the Exabyte products look good on paper - but I have no first-hand experience with them. They all need to be attached to a server, via SCSI or FireWire. I would prefer a NAS-like, server-independent autoloader that simply requires an Ethernet connection - but I am not even sure whether such a product exists...
post #3 of 33
What's wrong with 500GB hard drives mounted in removable sleds? You could buy 16 of them for $2500 - enough to have four backup copies of your 2TB. What are the chances of all four copies and your originals being unreadable?
post #4 of 33
Thread Starter 
what i am striving to do is a daily incremental backup of those files on the NAS that have changed in the past 24 hrs. I am particularly interested in keeping backups of each version of files that have been changing.

And I suspect that I will not be sufficiently disciplined to go to the basement every week and juggle sled-mounted hard drives...
post #5 of 33
Get an LT0-3-based autoloader with 5 to 24 tapes.
post #6 of 33
Personally, I think you said it best yourself.
The only irreplaceable data are personal videos and pics. Do you have 2TB of those?
What I do is use a large external drive and auto sync to that every night. Then, from time-to-time, I burn DVDs and leave a copy at my parents house.
Really, you could just burn any family videos to DVD when you create the file for an instant offline backup. For pictures, back up as necessary. This leaves you with a very small backup commitment.

$2500 buys lots of new movies
post #7 of 33
Thread Starter 
hmmm... Herzlichen Dank to KSchmit2 and thank you to video321. You both raise valid points - although you point me to opposite directions.

I am still pondering over which might be the best technology.

- I like archival-copy systems. If I wipe out a big directory, and realize my mistake much later, my synced HDD won't help - but an archive will.

- However, I dont wish to burn DVDs. Too much hassle, too many possible errors.

- on the other hand, tape is slow, non-random-access, the drives are very expensive, the tapes themselves appear to be quite expensive as well, and the capacity of the tapes is not at all that huge after all. These are a lot of drawbacks!!! Could it be that tape is becoming totally obsolete?

So, I know that I need to do something, but I am still uncertain as to what...

And yes, I indeed have a real lot of photos and videos...
post #8 of 33
RAID1. No attending required. Just replace drives as they die.

And yes, tape is obselete unless you're a corporation needing to adhere to Sarbanes-Oxley.
post #9 of 33
Hitachi just came out with 1TB drive for $399. Right now seems like only Dell has them but they have to go on sale in normal channels soon. I would do full back up, put the drive away and do incremental backups to some smaller drive, you can schedule this at certain time when nobody is around. Once smaller drive is full I would do another full back up and start over or you could actually do incremental backups to DVD or tape. DVD is not a serious option since it would probably take more than 200 discs to hold 1TB and assuming one disc takes 6 min to burn, about 20 hours time. BTW DVD discs do fail to burn properly, get scratch or have factory defects so I would expect few of those 200 to have at least partial data corruption. I wouldn't use tape even if you pay me.
post #10 of 33
Thread Starter 
Pete: your strategy of doing one big full backup and a lot of incrementals is great - but if the backup drive fails you are out of luck - big time! It would appear to me that tape might be the better medium for that strategy, despite all its obvious disadvantages.

Could a good robotic solution overcome the hassle of tape? The Exabyte VXA-2 PacketLoader 1x10 1U with FireWire can apparently be had for around 1700 $. Now, that's quite in the (uppermost) range of what I'd be prepared to spend. It holds ten 80GB tapes, which I would need for a first full backup. I would then put in another complement of 10 tapes, and set-and-forget it. I figure that it should run automated backups for 1-1.5 years, before I need to manually intervene and buy new tapes.

If that's the case, the hassle factor would certainly appear acceptable to me! But maybe I am neglecting something.

One final argument in favor of tape may be power consumption. While a backup disk wastes away 15-18 W continuously for no good reason, tape consumes virtually nothing. However, this is unlikely to reverse climate change, and whether it offsets all the other disadvantages, is unclear to me.
post #11 of 33
Quote:
Originally Posted by aag View Post

One final argument in favor of tape may be power consumption. While a backup disk wastes away 15-18 W continuously for no good reason, tape consumes virtually nothing. However, this is unlikely to reverse climate change, and whether it offsets all the other disadvantages, is unclear to me.

A disk in standby mode consumes less than one watt of electricity, and if you take it out of the system (the whole point of the removable sled), it consumes no power.

I think you're nuts if you think ten tapes are going to have a lower failure rate than a single hard drive that spends most of its life not connected to a computer.

If your data is that important, you need multiple backups, and the advantage of the hard drives, by virtue of their fewer numbers, is just going to increase.
post #12 of 33
Thread Starter 
But maybe I should explain why I am suddently nervous about disk backups. 15 months ago, I bought a ReadyNAS RAID5 array. I deemed myself safe. 6 months ago, one of the drives failed. I exchanged it no problem, even got a replacement drive under warranty from the vendor!

Then, 15 days ago, the array died while performing a whole volume access-intensive indexing. Apparently one drive failed in a way that has screwed up the operating system. The phone support was very responsive, but wasn't able to telnet into the array.

So, here we are. I relied on the RAID for safety, while I should have known better: RAID is good for maximal up-time, but does not exonerate you from backups! As pointed out by the latest poster, I am clearly very much nuts - and I have none to blame but myself!!! There may be a last stray of hope, but I have to send in the RAID to Infrant...

Hence, I now want to do better than that. I am actually a scientist by profession, but my field of expertise is molecular biology, and that does not help at all in this predicament...
post #13 of 33
Ask these guys how much they like tape for backups:

Oops! Technician's error wipes out data for state fund


Tape backups seem fine until you try to restore data from old media...ouch! Redundant drives look like a better way to go these days.
post #14 of 33
Thread Starter 
OK, Rupert, Jhuei et al.: you win!

But which kind of removable-disk device would you recommend? If I go down this way, I would need an enclosure with the following specs:
  • drive bay should be very easily accessible
  • interface should be USB2
  • alternatively, interface could be Gigabit Ethernet (NAS)
  • in the latter case, there should be a way to effect power-down and up on-demand

I entirely agree that this is simpler than using tape. I could buy one terabyte drive to run the first full backup (compressed), take it out, and insert another terabyte drive for incrementals (should last a full year until it gets filled).

Right?
post #15 of 33
Quote:
Originally Posted by aag View Post


- on the other hand, tape is slow


LTO-3 runs at a sustained read/write spead of 80 MB/s (uncompressed). You'll actually have significant problems attaining that sustained speed from your HDDs. Fortunately LTO drives can fall back to slower speeds though.

As for random-access:
I thought you wanted an archive for disaster recovery. Random access is totally insignificant in that situation.

Also, tape is not becoming obsolete in the professional environment.

LTO-4 (with optional WORM support) is just around the corner, with sustained transfer rates of 160 MB/s and tape libraries of up to 240 PB.
post #16 of 33
Btw, you always have to backup your backups when data is really important.

And for tape that means that you should copy one tape to another tape in at least 6 month or 12 month intervals. That way you make sure that the data remains fine, and corruption can be remedied because you have another set of backups.
post #17 of 33
Hi,

Quote:


RAID1. No attending required. Just replace drives as they die.

This certainly seems to be a simple and reliable option.

Dumb question re. RAID1: If one is using a true hardware card to do RAID1 and the card should die, will the data not still be available on the drives as it is only mirrored and not striped? If this is the case then RAID1 seems a good option here. No worries that loosing the card will also loose all your data and the odds of both drives in an array failing are extremely slim. Of course, you could reduce the risk even further by making an external backup of the RAID1 arrays.

J
post #18 of 33
Quote:
Originally Posted by aag View Post

I could buy one terabyte drive to run the first full backup (compressed), take it out, and insert another terabyte drive for incrementals (should last a full year until it gets filled).

Right?

Not quite.
Depending on the files, they may not be anymore compressable. An example would be MP3 files that are already highly compressed; from a file format stand point.
post #19 of 33
Thread Starter 
Would any typical backup software recognize HDDs as "removable media"? Maybe this is far-fetched, but I am a little worried of the following situation:

I would be trying to backup 1.5 TB to more than one 800GB (or 1TB) HDDs. Would the software understand that, and prompt me to replace the HDD in the enclosure - even if it appears to be a eSATA HDD rather than a bona fide removable medium?

Sorry for being a pedant - I just want to be sure I cover all issues...
post #20 of 33
RAID is NOT BACKUP so those suggesting RAID1 as a full solution are missing the point. RAID1 is just as likely to save you from a disk failure as RAID5, but not a controller failure, user error or "act of god."

Tape is the best way to go if you buy good equipment. Forget what you know about old DDS or QIC tape setups as today's tape drives are far more reliable. DLT, LTO and even Exabyte's VXA drives and associated tapes are fast and last a long time. Don't forget that the cost of the drive is one thing, but tapes for the capacity you need will set you back a hefty amount as well, depending on the technology you choose.

Going tape will allow you to not only backup your stuff, but get it off site. This is just as important as backing up the data at all because in the case of fire or flood RAID or local backups aren't going to do a thing.

Exabyte was bought by Tandberg in 2006 but they still sell VXA equipment. I have used VXA tapes and drives and they work well. They're not as quick as DLT or LTO but if most of your data is static (sounds like it is) then you could use a VXA drive just fine.

If you're looking to do just mistake and disaster recovery, and don't care about snapshots in time, then doing a full backup twice a year and doing differential backups after that will be a nice cost effective method. It'll cut down on the number of tapes you need and honestly, a tapeload wouldn't be needed. I would get enough tapes to do a full backup plus two more. Do a full backup and send those tapes off for off site storage in say a safety deposit box. If you're really paranoid you can send them out of town. On the remaining two tapes, do a differential backup each week or every two weeks rotating between the two tapes each time. You can if you want (though this will get to be alot of work), also send one of the two tapes off site.
post #21 of 33
Thread Starter 
Dear mym6, thank you for your (obviously highly competent) input. I am fascinated by the diversity of opinions! As I wrote above, I have now experienced - the hard way - that RAID5 is not equal backup! (Sigh!)

Since you are the only supporter of VXA thus far, let me pick your brains a little more. The capacity of VXA tapes appears underwhelming. A fully-loaded VXA autoloader has similar capacity as a current HDD. The cost of drive and media is not insignificant either.

Why not, then, adopt the very same full+incremental strategy you suggest, but on HDDs rather than tapes? HDDs may be more expensive than tapes (though not much more so on a $/GByte basis), but I'd save 1.7-2 K$ on the tape autoloader...

Of course, the above is not to negate your arguments, but rather to encourage you to expose more of them.
post #22 of 33
Quote:
Originally Posted by aag View Post

Dear mym6, thank you for your (obviously highly competent) input. I am fascinated by the diversity of opinions! As I wrote above, I have now experienced - the hard way - that RAID5 is not equal backup! (Sigh!)

Since you are the only supporter of VXA thus far, let me pick your brains a little more. The capacity of VXA tapes appears underwhelming. A fully-loaded VXA autoloader has similar capacity as a current HDD. The cost of drive and media is not insignificant either.

Why not, then, adopt the very same full+incremental strategy you suggest, but on HDDs rather than tapes? HDDs may be more expensive than tapes (though not much more so on a $/GByte basis), but I'd save 1.7-2 K$ on the tape autoloader...

Of course, the above is not to negate your arguments, but rather to encourage you to expose more of them.

Well truth be told I had forgotten how slow VXA was. I just went and looked on the tandberg site and it's quoting a 83GB/h transfer rate. Of course that's an "up to rating" meaning real world you'll be lucky to see 3/4 of that. Even if you do see those kinds of speeds it'll take a day just to do a full backup. I would throw out my VXA suggestion all together.

As someone else suggested LTO-3 (LTO-4 is coming as stated) is probably the fastest of the tape technologies right now but you'll spend a bunch too. I have an LTO-2 drive at work and it will backup and verify ~170GB of data across three servers in less than four hours. The backup software always reports a throughput of ~900MB for the whole process. You'd get better performance if the drive is locally attached only (two of my servers are across gigabit)

To actually answer your question though. Hard drives are going to be quick (backup speed for the price) and cheap but if you do want to move them a lot I'd worry some about durability since you're not only moving the medium but also the moving parts. If your tape drive fails you replace it and all of your tapes are fine. If a drive fails you lose stuff. It's rare today that a tape itself would fail unless you never replace it and it becomes excessively worn.

I honestly just image my laptop's 80GB drive to an external hard drive about once a month but I don't have as much data accumulated as you. Since my data needs are currently well within the size of hard drives available today I'll just continue to back up to a single drive. Which brings up another point, you'll have to find some backup software that will backup to multiple hard drives, I'm personally not aware any that will do that, has good scheduling and doesn't cost a fortune.
post #23 of 33
Not sure if this was mentioned yet either but don't look so much as the compressed capacity if you're backing up music, video and pictures. They are all compressed already (unless you store raw images, raw wav files) and you won't see much benefit, if any, with compression.
post #24 of 33
Thread Starter 
Thank you. Things are getting much clearer now!

Four drives with largest-available capacity will cost me ca. a grand, and will cover my backup needs for >1yr. What I need next, is a sled enclosure with a USB-to-SATA interface (would something like this do?).

Finally, I need a good backup software that allows for backup-to-USB-device, scheduling, and ideally e-mails an alert whenever the medium is full or any other trouble arises. Any advice on the latter?
post #25 of 33
The enclosure you found seems good, nice that they also carry the extra trays. Try searching newegg as well. Friend of mine always finds nice external enclosures but I can't find them right now.

I assume you're using Windows so you could try the built in backup software. It's actually very similar to an old version of Backup Exec. What's nice is that it works with VSS or Volume Shadow Services meaning you can backup open files. I had a ton of issues getting scheduled jobs when storing to tape, but no problems with scheduled jobs that go to a backup file. Of course, it lacks the ability to span multiple files or disks and it won't email you on failure (or success for that matter).

You might end up better off just replicating your current RAID setup and use it as the backup destination. Either that or you'll need to start going some research on google. I did some quick searching but I'm not getting anything that looks promising.
post #26 of 33
Quote:
Originally Posted by aag View Post

Pete: your strategy of doing one big full backup and a lot of incrementals is great - but if the backup drive fails you are out of luck - big time! It would appear to me that tape might be the better medium for that strategy, despite all its obvious disadvantages.

Could a good robotic solution overcome the hassle of tape? The Exabyte VXA-2 PacketLoader 1x10 1U with FireWire can apparently be had for around 1700 $. Now, that's quite in the (uppermost) range of what I'd be prepared to spend. It holds ten 80GB tapes, which I would need for a first full backup. I would then put in another complement of 10 tapes, and set-and-forget it. I figure that it should run automated backups for 1-1.5 years, before I need to manually intervene and buy new tapes.

If that's the case, the hassle factor would certainly appear acceptable to me! But maybe I am neglecting something.

One final argument in favor of tape may be power consumption. While a backup disk wastes away 15-18 W continuously for no good reason, tape consumes virtually nothing. However, this is unlikely to reverse climate change, and whether it offsets all the other disadvantages, is unclear to me.


I think you're paranoid, how can 2 hard drives fail at the same time? You've original and then you have back up, they would have to fail pretty much at the same time, which is highly unlikely. In last 6 years of using computers daily at work and at home I have one hard drive fail and it was brand new, started giving me problems from the moment I got it so I didn't even loose any data . Once I get data fully backed up I would take it off line to avoid other problems, like power surge knocking everything out. Tapes are least reliable, slowest, most expensive data storage there is. Your power argument is off the wall since in standby mode both, HDD and tape drive will use couple watts at most, besides I suggested to do main backup and then disconnect the hard drive all together, there are 51/4 bay trays, you can install your HDD into and have it disconnected from system until needed. I got Mobile Rack for $20, I can power down the hard drive with the press of the button or take the whole tray with drive out by pulling on the handle. But if you insist on having the most expensive, slowest and least reliable backup solution there is what else do you want me to say? Tape is your best bet here.
post #27 of 33
I wouldn't write off tape backup as unreliable. Since switching away from the low end setups like DDS to VXA and then LTO-2 I have never once had a backup or restore fail in four years. I can't say the same about drives themselves during that time. Tape drives are not all that slow today unless you're pulling from a fast raid set. In this case the disks have an unfair advantage in that you're reading from multiple spindles at once where as you can only write to a single tape at a time.

You do bring up a good point about the failure of your backup source drive and backup destination drive. How likely is it that they will both fail? Probably just as likely that a tape backup will fail.
post #28 of 33
Thread Starter 
Whoa, Pete4, I apologize! I was just trying to compare technologies! It seems that there are very diverse opinions around - and that's quite interesting, isn't it?
post #29 of 33
Quote:
Originally Posted by aag View Post

Finally, I need a good backup software that allows for backup-to-USB-device, scheduling, and ideally e-mails an alert whenever the medium is full or any other trouble arises. Any advice on the latter?

Lots of good backup software out there, but Cobian Backup always gets my vote for being full featured freeware. It runs as a service, does e-mail notification, compression, scheduling, some encryption....nice package.


http://www.educ.umu.se/~cobian/cobianbackup.htm
post #30 of 33
Quote:
Originally Posted by mym6 View Post


I assume you're using Windows so you could try the built in backup software. It's actually very similar to an old version of Backup Exec. What's nice is that it works with VSS or Volume Shadow Services meaning you can backup open files.

Doesn't VSS require that the data be stored on Windows Server 2003?
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Home Theater Computers
AVS › AVS Forum › Video Components › Home Theater Computers › Automatic/unattended backup og 1-2 TB