the best media for backups - AVS Forum
Forum Jump: 
 
Thread Tools
Old 05-12-2010, 05:07 AM - Thread Starter
Rgb
AVS Special Member
 
Rgb's Avatar
 
Join Date: Apr 2000
Location: SE Michigan
Posts: 6,893
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 10 Post(s)
Liked: 19
Quote:
Originally Posted by tux99 View Post

Around 20GB could be less or more, depends on the length of the movie and the bitrate of the original.

Don't know, never tried that, why? Do you want back up your MKVs to DVD-R/BD-R?
What's the point in that, buying a 2TB hard disk as back-up is much cheaper per GB.

Not so- the lowest cost 2TB drive on newegg.com this morning is $130, so about $0.06/GB. I just purchased a couple hundred Verbatim DVD+R retail in sealed cakeboxes (regarded as some of the best quality media) for about $0.03/GB, or half the cost per GB.

I like having optical backups, with the plan being to use them on low cost BluRay set tops once more of them can play *.mkv 1080p files directly from DVD/BD-R, or simply build low cost ATOM/ARM/low power CPU+VDPAU or equivalent media PC's with optical drives for every TV.

Hard drives ALWAYS die- not IF but WHEN, whether due to mechanical failure, malware munging the data, or filesystem/MBR/partition table corruption due to OS and/or driver/filesystem bugs.

OTOH, my CD's and DVD's burned from 1995 and the early 2000's respectively play fine in everything.

My motto is, if it ain't burned, you must not want to keep it...
Rgb is offline  
Sponsored Links
Advertisement
 
Old 05-12-2010, 05:23 AM
 
mythmaster's Avatar
 
Join Date: Mar 2008
Location: 255.255.255.255
Posts: 2,142
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by Rgb View Post

Not so- the lowest cost 2TB drive on newegg.com this morning is $130, so about $0.06/GB. I just purchased a couple hundred Verbatim DVD+R retail in sealed cakeboxes (regarded as some of the best quality media) for about $0.03/GB, or half the cost per GB.

I like having optical backups, with the plan being to use them on low cost BluRay set tops once more of them can play *.mkv 1080p files directly from DVD/BD-R, or simply build low cost ATOM/ARM/low power CPU+VDPAU or equivalent media PC's with optical drives for every TV.

Hard drives ALWAYS die- not IF but WHEN, whether due to mechanical failure, malware munging the data, or filesystem/MBR/partition table corruption due to OS and/or driver/filesystem bugs.

OTOH, my CD's and DVD's burned from 1995 and the early 2000's respectively that play fine in everything.

My motto is, if it ain't burned, you must not want to keep it...

I have to agree with this because hard drives always die, but if you have your stuff backed up on a DVD then you are good to go.

You know it's sad but true.
mythmaster is offline  
Old 05-12-2010, 07:31 AM - Thread Starter
Rgb
AVS Special Member
 
Rgb's Avatar
 
Join Date: Apr 2000
Location: SE Michigan
Posts: 6,893
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 10 Post(s)
Liked: 19
Quote:
Originally Posted by mythmaster View Post

I have to agree with this because hard drives always die, but if you have your stuff backed up on a DVD then you are good to go.

I always laugh when some HT/HTPC enthusiast proclaims "optical media is dead- streaming and downloads are the Way" or "I rip all my discs to hard disc (or RAID) and throw away the optical discs!".

Both sentiments are naive, short sighted and lack wisdom from years of HT/HTPC/IP issues & experience.

If printed optical media goes away, we lose all control or power in deciding how to use/buy/sell media. Streams and downloads will be region controlled, DRM'd to the hilt, will come and go at the whim of the IP holder, be edited at the whim of the IP holder, with no resale rights.

Printed optical media will last decades if not your (human) useful lifetime, can be resold/loaned/traded at your discretion, cannot be edited once in your posession (scenes or episodes deemed politically incorrect in the future- this has happended FAR too much already- non-PC Warner/Disney/Popeye toons from the 30's-40's-50's-60's, many films from that time and later- guns for walkie-talkies in Speilberg's ET, anyone?), cannot be "turned off" or made unavailable for rent/streaming/download at any time, you can buy used optical media on ebay for hard to find/OOP films or shows, printed disc DRM will always be broken eventually, and can be played anywhere in the world (PC's or region free set tops).

Ripping/backing up your optical media to hard discs as a short term *convenience* is certainly OK, assuming you have all the optical media somewhere in your possesion as the Master backups, but hard drives/streaming/downloads as the *only* sources for audio/video- no freakin' way...and RAID won't save you from multiple hard disk failures, malware or filesystem/MBR corruptions.

...the locked down Kindle and other ebook readers have demonstrated how important retaining printed media is- and there is ZERO logical difference among printed words (books), printed audio (CD's, DVD-A) and printed video (DVD, BluRay).
Rgb is offline  
Old 05-12-2010, 08:03 AM - Thread Starter
Rgb
AVS Special Member
 
Rgb's Avatar
 
Join Date: Apr 2000
Location: SE Michigan
Posts: 6,893
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 10 Post(s)
Liked: 19
Quote:
Originally Posted by tux99 View Post

What's the point in that, buying a 2TB hard disk as back-up is much cheaper per GB.

With the advent of the LTH BD-R's

http://www.newegg.com/Product/Produc...82E16817130070

BD-R media costs appear to be dropping. I would be happy once the quality (Verbatim) BD-R LTH media is at the same cost/GB as a quality hard drive, or about $0.06/GB, or about $1.50 per 25GB BD-R disc. I have recently seen sales of the Verbatim LTH BD-R 25GB media for about $1.60 per disc, so it's getting close.

Ideally, LTH BD-R media should be the same cost/GB as the quality DVD+R media available, or about $0.75 USD per 25GB disc.

When LTH BD-R's get reliably below $1.50, and LTH compatible BD burners, BD-ROM's and LTH compatible BluRay set tops that can play 1080p full bit rate .mkv's are all commonly below $100 USD, I may start experimenting with BD...
Rgb is offline  
Old 05-12-2010, 09:21 AM
AVS Special Member
 
tux99's Avatar
 
Join Date: Jan 2005
Location: Europe
Posts: 1,523
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Rgb, I agree with your sentiments about printed optical media versus streaming but that has nothing to do with this thread and even with your original question or my reply (or even less so the subject of this thread).

We were purely talking about back-ups, not the source of our media.

I have a wast collection of original DVDs and some BDs, which I know out of personal experience don't last forever. I have already about 3-5% of my 10+ years old original DVDs that are unreadable or skip badly, despite I kept them safely out of sunlight or high humidity or high temperatures (I don't live in a hot or tropical country).
I have also already found several optical writable media from 5-10 years ago that I burned and that are already partially unreadable (a couple of CD-Rs from 13 years ago are completely unreadable).

Because of that and also for play-back convenience I back up all my original media to hard disks (raid5). Since backing up more than 1000 DVDs takes a huge amount of time I don't want to have to do that all over again, so I also back up my raid5 array to plain 2TB external hard disks.

I can't see the use for DVD-R or BD-R, ok they might still be cheaper than hard disks (per GB) but it takes wastly more time to burn to optical media than to copy data to an external hard disk (not to mention the hassle of splitting up the data first and archiving hundreds if not thousands of optical discs).

Yes hard disk die sooner or later, but so far by doing the initial "badblocks -svw" burn-in check of each new disk that I buy (which I mentioned a few times already on this forum), and by RMA'ing those that fail this check, my disks have always lasted at least 5 years (some are even older and still ok) and after 5 years I will want to migrate the data to newer bigger disk anyway.

So I'm curious to know what your strategy is, that involves the use of optical writable discs.

My Linux news / reviews / tips+tricks / downloads web site: http://www.linuxtech.net/
tux99 is offline  
Old 05-12-2010, 12:14 PM - Thread Starter
Rgb
AVS Special Member
 
Rgb's Avatar
 
Join Date: Apr 2000
Location: SE Michigan
Posts: 6,893
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 10 Post(s)
Liked: 19
Quote:
Originally Posted by tux99 View Post

Rgb, I agree with your sentiments about printed optical media versus streaming but that has nothing to do with this thread and even with your original question or my reply (or even less so the subject of this thread).

We were purely talking about back-ups, not the source of our media.

I have a wast collection of original DVDs and some BDs, which I know out of personal experience don't last forever. I have already about 3-5% of my 10+ years old original DVDs that are unreadable or skip badly, despite I kept them safely out of sunlight or high humidity or high temperatures (I don't live in a hot or tropical country).
I have also already found several optical writable media from 5-10 years ago that I burned and that are already partially unreadable (a couple of CD-Rs from 13 years ago are completely unreadable).

Because of that and also for play-back convenience I back up all my original media to hard disks (raid5). Since backing up more than 1000 DVDs takes a huge amount of time I don't want to have to do that all over again, so I also back up my raid5 array to plain 2TB external hard disks.

I can't see the use for DVD-R or BD-R, ok they might still be cheaper than hard disks (per GB) but it takes wastly more time to burn to optical media than to copy data to an external hard disk (not to mention the hassle of splitting up the data first and archiving hundreds if not thousands of optical discs).

Yes hard disk die sooner or later, but so far by doing the initial "badblocks -svw" burn-in check of each new disk that I buy (which I mentioned a few times already on this forum), and by RMA'ing those that fail this check, my disks have always lasted at least 5 years (some are even older and still ok) and after 5 years I will want to migrate the data to newer bigger disk anyway.

So I'm curious to know what your strategy is, that involves the use of optical writable discs.

re: "wast collection"- you and Mr. Chekov, I presume


re: backup strategy

3-5% of bad original DVD's sounds like a lot. I've been ripping a large number of my ~8-12 year old original commercial DVD's, and only last week had an unreadable one with a bad spot, which I suspect was manufactured with the defect. So, I'm not experiencing anywhere near 3-5% failure of original pressed DVD's. If you count that one, then maybe .01% (or less) "failure" for me.

I sometimes do random read tests from CD/DVD burns that are 5-15 years old and can't recall any burned discs I couldn't read in a recent vintage drive. Some older optical drives may just be worn out or unable to read burned discs correctly (by design, before burned DVD's became standardized and the burn process debugged, which really wasn't until at least 2004), if the optical drive is >8 years old or so.

Also, the mfg id of blank media makes the BIGGEST difference in disc quality, burn quality and longevity. I only use 1st class media at

http://www.digitalfaq.com/reviews/dvd-media.htm

burned no faster than 4x, though I may start experimenting at 8x.

NEVER, EVER use adhesive labels on a disc of any kind, either before or after burning- they will make the disc unbalanced and drives won't be able to read them reliably due to the wobble, however slight, no matter how precise a label is applied.

For archival backups of stuff important to me, I might burn two backups on two brands of 1st class media, like Verbatim and Sony, +R only, which purportedly has better burn control and track control info on the disc.

DVD's are ripped to hard disk with the latest Fab then either shrunk with DVDShrink or DVDFab, or split into two using the Split funtion of Fab if I want to preserve 100% video quality without reencoding. I use the shrunk/re-encodes for travel, vacation home, wifes' classroom use (teacher) loans, etc.

The DVDFab split function (which separates a dual layer DVD into two single layers DVD's without touching the video or audio) is EXTREMELY fast about 3-4 (or faster) minutes to split a 7.5GB DVD rip from one hard disk to another, as fast as copying the 7.5GB files straight up from one hard disk to another, so this effectively adds no time to the process assuming you rip to one hard disk and move them later anyways.

I do not Verify every burn- just randomly verify every 20-30th burn to ensure the process and burner are still good.

Yes, burning takes time, but I do other things while burning/shrinking, like email, reading /. or digg, making HOWTO's and posts like this . All machines in the house have a DVD burner (3-4 functional at any given time, not always burning ). Burning over a 100Mbit wired network at 4x has been no issue, so you can burn from the same hard drive the rips are on to several different machines at once over a network- burning two discs at once at 4x is the same as burning one at 8x, but with higher burn quality.


Discs are stored either with the original in a Tyvek sleeve on the side with the inserts of the DVD case, or the original's case is replaced with a 6 disc stackable version like-

http://www.shop4tech.com/item5896.html

with the original(s) on top and backups underneath, plus another two discs can fit in a sleeve on the booklet side.

Originals are on shelves in alphabetical order, with special collections separated, i.e. Star Trek, Star Wars, Indiana Jones (in addition to all film release originals, I have all the Young Indiana Jones boxsets ).

Overflows go into the cakeboxes the blanks came in, sorted by either genre (westerns, horror, etc), actor (Ben Stiller films, Paul Newman films, etc), or director (Speilberg, Wolfgang Petersen, etc).

A 100 disc cakebox might have several actors grouped in the box, separated by one of those clear plastic false discs at the bottom (or top) of a stack of blanks, with a tag made from tape attached with the actor's name on it readable through the cakebox container, identifying all discs beneath the separator.

Cakeboxes effectively become film "canisters" in your "vault"

This media library is kept in a cool, dry, dark basement with relatively constant temp/humidity.

When I feel like a Newman film, I peruse the Newman "stacks", or a Weir film, and so on...

...I'm a Librarian at heart, I guess- call me Mr. Atoz
Rgb is offline  
Old 05-12-2010, 01:43 PM
 
mythmaster's Avatar
 
Join Date: Mar 2008
Location: 255.255.255.255
Posts: 2,142
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
One more off-topic post here:

Once your optical media becomes too scratched to read, it is still salvageable. I've seen machines that fill in the scratches with a liquid plastic that later becomes solid. Here are some examples at Amazon: http://www.amazon.com/s/ref=nb_sb_no...repair&x=0&y=0
mythmaster is offline  
Old 05-12-2010, 02:03 PM
AVS Special Member
 
tux99's Avatar
 
Join Date: Jan 2005
Location: Europe
Posts: 1,523
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by Rgb View Post

re: "wast collection"- you and Mr. Chekov, I presume

Sorry, but I don't get the joke, maybe US only?
Anyway maybe "wast" wasn't the right term but more than 1000 original DVDs is more than most people I know have. I'm also sure there are plenty people on this forum that have much bigger collections but that's beside the point, I didn't intend to make this a "mine is bigger than yours thread"!

Quote:
Originally Posted by Rgb View Post

if optical discs go away or are marginalized as a consumer format in lieu of streaming/downloads stored on hard disks, there would be no point to DVDFab (or AnyDVD) nor this thread

Like I said I perfectly agree with you on that, I'd never buy any DRM crippled downloads or streaming media, but the subject in this thread was about making back-up copies on recordable optical discs, not about original media.

Quote:
Originally Posted by Rgb View Post

3-5% of bad original DVD's sounds like a lot.

I have approximately 150 pre year 2000 DVDs (many US region 1 imports since at that time the choice was better in the US) and I have found already 5 that are unreadable with any DVD player/PC drive I have. They all have no scratches or other visible signs of wear and have been watched only once or twice.
I haven't checked all 150 yet, since I still have to finish backing them up to hard disk.

They all have in common that they have a golden/brownish data surface like many early pressed DVDs had, rather than silver.

If you do a search on Internet forums you'll find that I'm not alone with this.

Quote:
Originally Posted by Rgb View Post

For archival backups of stuff important to me, I might burn two backups on two brands of 1st class media, like Verbatim and Sony, +R only, which purportedly has better burn control and track control info on the disc.

I researched long-term data archival quite a bit (mainly to find good archival media for my personal data like documents and photos, that unlike movies, are irreplaceable) so I do know about good and bad media, I actually even wrote an article about it:
http://www.linuxtech.net/tips+tricks...a_storage.html
But like I said, I make a big distinction between personal irreplaceable data and entertainment media like music and movies.

Your movie archival system sounds very well thought out and if you consistently follow your own procedures as detailed then it should keep your data well safe and organized.
I have full respect for it, but it sounds like way too much work for me and probably most people, especially since with hard disk backups I can achieve the same with much less hassle and time and not much more money (and hard disks are getting cheaper per GB much faster than optical media).

My Linux news / reviews / tips+tricks / downloads web site: http://www.linuxtech.net/
tux99 is offline  
Old 05-12-2010, 02:06 PM
AVS Special Member
 
tux99's Avatar
 
Join Date: Jan 2005
Location: Europe
Posts: 1,523
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by mythmaster View Post

One more off-topic post here:

Once your optical media becomes too scratched to read, it is still salvageable. I've seen machines that fill in the scratches with a liquid plastic that later becomes solid. Here are some examples at Amazon: http://www.amazon.com/s/ref=nb_sb_no...repair&x=0&y=0

Yes I know about that, but scratches aren't the main/only problem, oxidation and chemical decay of the data layer is.
Also the glue that holds the two plastic layers together can get unstuck over time which means the laser can't focus on the data pits anymore since the distance changes.

My Linux news / reviews / tips+tricks / downloads web site: http://www.linuxtech.net/
tux99 is offline  
Old 05-12-2010, 02:10 PM - Thread Starter
Rgb
AVS Special Member
 
Rgb's Avatar
 
Join Date: Apr 2000
Location: SE Michigan
Posts: 6,893
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 10 Post(s)
Liked: 19
Quote:
Originally Posted by mythmaster View Post

One more off-topic post here:

Once your optical media becomes too scratched to read, it is still salvageable. I've seen machines that fill in the scratches with a liquid plastic that later becomes solid. Here are some examples at Amazon: http://www.amazon.com/s/ref=nb_sb_no...repair&x=0&y=0

I have been quite impressed with the Venmill 3500-

http://www.venmill.com/products/vmi3500.html

only used to repair rentals or library discs I have seen, none of my own (yet)
Rgb is offline  
Old 05-12-2010, 02:23 PM - Thread Starter
Rgb
AVS Special Member
 
Rgb's Avatar
 
Join Date: Apr 2000
Location: SE Michigan
Posts: 6,893
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 10 Post(s)
Liked: 19
Quote:
Originally Posted by tux99 View Post

Sorry, but I don't get the joke, maybe US only?
Anyway maybe "wast" wasn't the right term but more than 1000 original DVDs is more than most people I know have. I'm also sure there are plenty people on this forum that have much bigger collections but that's beside the point, I didn't intend to make this a "mine is bigger than yours thread"!

Like I said I perfectly agree with you on that, I'd never buy any DRM crippled downloads or streaming media, but the subject in this thread was about making back-up copies on recordable optical discs, not about original media.

I have approximately 150 pre year 2000 DVDs (many US region 1 imports since at that time the choice was better in the US) and I have found already 5 that are unreadable with any DVD player/PC drive I have. They all have no scratches or other visible signs of wear and have been watched only once or twice.
I haven't checked all 150 yet, since I still have to finish backing them up to hard disk.

They all have in common that they have a golden/brownish data surface like many early pressed DVDs had, rather than silver.

If you do a search on Internet forums you'll find that I'm not alone with this.

I researched long-term data archival quite a bit (mainly to find good archival media for my personal data like documents and photos, that unlike movies, are irreplaceable) so I do know about good and bad media, I actually even wrote an article about it:
http://www.linuxtech.net/tips+tricks...a_storage.html
But like I said, I make a big distinction between personal irreplaceable data and entertainment media like music and movies.

Your movie archival system sounds very well thought out and if you consistently follow your own procedures as detailed then it should keep your data well safe and organized.
I have full respect for it, but it sounds like way too much work for me and probably most people, especially since with hard disk backups I can achieve the same with much less hassle and time and not much more money (and hard disks are getting cheaper per GB much faster than optical media).

re: "wast"- Star Trek's Chekov speaks with a Russian accent, pronouncing v's as w's

re: gold/brown commercial DVD pressings- I wasn't aware of specific issues with these- thanks for the heads up! I will double check the balance of my oldest commercial DVD's...I believe the one bad DVD I mentioned *was* gold/brown...

re: your archiving article

Nice writeup, though I personally would *never* trust any flash media (especially without a hardware write protect switch) over a good quality DVD+R burned properly (even the non-gold Verbatims ) for long term storage.

I really wish most hard drives had a hardware write protect switch like good ole 3.5" floppies...

Given the HUGE amount of CD and DVD burns I've done over 15 years, and the amount of use the burns have gotten, I am HIGHLY confident in them...

Will they be readable in 30 years? Who knows. I don't think a hard disk simply placed on a shelf would work 30 years from now due to spindle motor, actuator arm or bearing stiction or other issues, like availability of compatible interfaces at the time.

However, one way around this is to move all data from one hard drive to a new higher capacity drive every 3-5 years as you move forward, which appears to be your method, which I do for some data.

But reading quality DVD+R in 10 years? Definitely. 20 years? Probably. After that, I'm certain I would move the stuff important to me to holographic or similar higher capacity optical media with multi-TB capacity for a buck or so by that time. Much beyond that? We're all dead .

For the personal photos/videos issue, DVD+R is a total no brainer. Most people's entire photo library would fit on one or two (or 3-5) DVD+R's- simple to burn 5 backups on 5 brands of media and spread the burns among several locations, dirt cheap...

(always assuming the DVD+R burns are kept in a cool, dry, dark, stable environment most of the time)
Rgb is offline  
Old 05-12-2010, 02:39 PM
 
mythmaster's Avatar
 
Join Date: Mar 2008
Location: 255.255.255.255
Posts: 2,142
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Since we were arguing about this in another thread real off-topic-like, I figured that it would be a good idea to discuss all of the options for backup media here in one thread.

Some people say that optical media is the best while others say that hard drives are the best. Each group has their own reasons, and I'd be interested to hear what you think.
mythmaster is offline  
Old 05-12-2010, 02:42 PM
 
mythmaster's Avatar
 
Join Date: Mar 2008
Location: 255.255.255.255
Posts: 2,142
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
I'm also wondering what the failure-rate of SSD's is. Sure, it's WAY more expensive to backup stuff to a SSD, but I'm just considering all options.
mythmaster is offline  
Old 05-12-2010, 03:15 PM
 
mythmaster's Avatar
 
Join Date: Mar 2008
Location: 255.255.255.255
Posts: 2,142
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Personally, I think that optical discs provide the most longevity for off-line backup storage...especially now that we can get 25GB+ on a BD. Despite their problems, I just can't see how storing backups on hard drives could possibly outlast storing them on optical media. If I go a year without a hard drive failing then I feel lucky, but I have DVD's that I burned 5 years ago that still work fine.
mythmaster is offline  
Old 05-12-2010, 05:43 PM
AVS Special Member
 
Phantom Gremlin's Avatar
 
Join Date: Aug 2002
Location: Tualatin Oregon
Posts: 1,407
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Optical media is just too low density for me. Storing 200 movies on 200 separate disks is much much harder to handle than putting those bits on a single hard drive.

Both solutions need some sort of redundancy. There are programs that add additional checksum blocks to files, but that means even more processing steps.

That's why I think that longer term something like the ZFS file system is the way to go. All data is checksummed. It will also recover automatically from "bit rot", assuming that you have stored using one of the RAID-like options.

You need:
  • a ZFS file server with some sort of RAID/mirroring
  • one or more additional ZFS hard drives for off-site backup
  • a strategy to migrate to newer generation drives perhaps every 4 or 5 years. And if not drives, then you need to migrate optical media. The stuff you write at home won't last nearly as long as pressed discs.
  • lots of cash. Drives aren't cheap once you start buying enough for redundancy and off-site backup. Alternatively, lots of time. Optical disks are a little cheaper but require a lot of handling steps.
Phantom Gremlin is offline  
Old 05-12-2010, 06:27 PM - Thread Starter
Rgb
AVS Special Member
 
Rgb's Avatar
 
Join Date: Apr 2000
Location: SE Michigan
Posts: 6,893
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 10 Post(s)
Liked: 19
The relevant posts from

http://www.avsforum.com/avs-vb/showthread.php?t=1082925

should be moved here in a day or so.

Needless to say, I advocate optical disc backup

By the time you spend the extra cash in redundant hard disks and the requisite complexity in managing RAID, the power consumption, managing the software issues with RAID and ZFS, it's MUCH simpler and less costly to just burn everything you want to keep long term.

The ripping step is the same.

Assuming you don't need to re-encode/transcode, there is no extra time there.

THe only extra step is reaching over and placing a blank in your DVD drive and burning, while you're doing something else. It becomes second nature after a short while and you don't notice it.

I suspect most of us do something else while discs are ripping, too- no difference when burning.

No cost to store- use the same cake boxes they came in and sort by actor/director/genre, or year.

I am most interested in BD because of the big boost in capacity per disc, even if simply backing up DVD's to BD! I am hoping future BD set tops will play VIDEO_TS folders burned as data to BD's, and/or play BD's authored as BAD's (Big A$$ DVD's)

(Remember- I coined the term BAD)

...it would be cool to turn a 3-4 disc DVD set into one physical disc, complete and untouched audio/video wise- or a complete 5-7 DVD disc TV season, first DVD-Shrunk (or transcoded to 480p BD compatible x264 or VC1) then reauthored for one BD-R...
Rgb is offline  
Old 05-12-2010, 08:57 PM
 
mythmaster's Avatar
 
Join Date: Mar 2008
Location: 255.255.255.255
Posts: 2,142
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Cool, thanks Rgb.

I'm liking the off-site ZFS storage for backups, too. It's easy enough to run OpenSolaris or FreeBSD in a VM just to manage ZFS drives/partitions or run it as a NAS server.

We should also make it a habit to install to LVM since it is easy to create and restore a LVM snapshot.
mythmaster is offline  
Old 05-13-2010, 05:19 AM
Member
 
KCLC's Avatar
 
Join Date: Jun 2005
Posts: 20
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
USB drives are a good backup medium, they're fast, online and relatively cheap. I back up servers with them now instead of tape. One thing you can be sure of whatever you use will someday be obsolete and you might have to transfer your media.
KCLC is offline  
Old 05-13-2010, 05:56 AM - Thread Starter
Rgb
AVS Special Member
 
Rgb's Avatar
 
Join Date: Apr 2000
Location: SE Michigan
Posts: 6,893
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 10 Post(s)
Liked: 19
Quote:
Originally Posted by KCLC View Post

USB drives are a good backup medium, they're fast, online and relatively cheap. I back up servers with them now instead of tape. One thing you can be sure of whatever you use will someday be obsolete and you might have to transfer your media.

I assume you mean USB *hard drives* and not flash drives?

Most "USB" hard drives are just SATA drives in a case with SATA-USB conversion.
Rgb is offline  
Old 05-13-2010, 08:34 AM - Thread Starter
Rgb
AVS Special Member
 
Rgb's Avatar
 
Join Date: Apr 2000
Location: SE Michigan
Posts: 6,893
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 10 Post(s)
Liked: 19
Quote:
Originally Posted by Phantom Gremlin View Post

Optical media is just too low density for me. Storing 200 movies on 200 separate disks is much much harder to handle than putting those bits on a single hard drive.

The problem with hard disks for long term/archival storage of a lot of movies/video is the number of failure modes, and the severity of the failure if it does occur, looking at the whole reliability/failure equation from a Systems or Reliability Engineering perspective, in light of an FMEA-

http://en.wikipedia.org/wiki/Failure...fects_analysis

The number of failure modes of hard drives is much larger than optical discs- disc surface degradation, actuator arm failure, read/write head failure, head crash, spindle motor failure, and other mechanical or electronics failure sources.

On top of all the mechanical and electrical failures, you have to heap on top *software* failures- corrupted FAT/MBR/partition tables, general filesystems corruptions of many types, PLUS bugs in filesystem drivers/device drivers/OS kernal bugs that write to or manage the filesystem or hard drive!

That failure space is HUGE.

A properly burned optical disc is REMOVED from these potential failure modes once the disc is properly burned and verified and stored in a reasonable fashion.

Since your archive is spread over 200 discs, preferably burned to a couple quality disc brands and separate containers, the liklihood of losing ALL the material with a given failure mode is greatly reduced or eliminated. SOME discs might degrade faster than others, some discs might be lost if separated from the others, but you'll be able to read/use MOST of them.

A whole host of the failures mentioned would render ALL 200 movies unusable on a single hard disk.

Adding enough redundency using hard disks (JBOD, RAID) shoots up the costs and maintenance complexity to 4-6x the cost/GB of quality optical discs.

The only remaining issues for the optical disc are aging of the recorded dye layer and adhesives used to bind the layers (mainly DVD blanks).

While accelerated tests are can be useful, they aren't foolproof or the whole equation.

Simply elevating temps/humidity and other factors doesn't linearly extrapolate failure modes in all cases.

It could be that the adhesive or dye aging characteristics don't scale linearly with temperature/time curves, i.e. just because accelerated aging at 2x room temp yields a life of 3-5 years doesn't necessarily mean at room temp the disc will last only 2x (3-5) or 6-10 years. The aging characteristic could very well be non-linear with respect to temps and other factors, so that under reasonable conditions, the dye and adhesives may last 25-30 years minimum or significantly longer, for example.

Plus, even if some degradation may occur, sporadic hiccups in playback may be tolerable in a 25 year old disc to recover 95%-99% of the "signal" or video.

Mild degradation could be protected against by making a PAR set on the files in the VIDEO_TS folder...
Rgb is offline  
Old 05-13-2010, 07:31 PM
AVS Special Member
 
tux99's Avatar
 
Join Date: Jan 2005
Location: Europe
Posts: 1,523
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Discussing backup media doesn't make sense without specifying first what you want to back up, since different kinds of data have different requirements.

As far as I can see there are 3 main groups of data that a home user might want to back up:

- Personal data (personal text documents, spreadsheets, tax-returns etc, photos taken with digital cameras or phones, home-videos, personal audio recordings, basically any data produced by yourself).
This is likely the most valuable data for a home user since it's irreplaceable and therefore needs extra precautions.
I have written a whole article about best reliable long-term data storage media for this purpose which can be read at the following link:
http://www.linuxtech.net/tips+tricks...a_storage.html

- System backups
By system backups I mean backups of the OS partition which normally doesn't include data, other than maybe the home folder with config files, bookmarks, etc.
For system backups external hard disks or a file server (NAS) are ideal since they allow automated or semi-automated backups and system backups normally don't have a long term retention period anyway (who cares about keeping a copy of their system install from 5 year ago?)

- Commercial media
This would be primarily DVD/BD movie backups and music files (backed up Audio CDs or downloaded music files). These files can be valuable since they can be costly to replace but unlike personal data, they are replaceable so don't require the same extreme caution as personal data.
Also the collection of media files tends to require much more storage space than personal files so personally I use a file server with raid5 to store the files and external off-line hard disks to back up the file server.
Optical media as Rgb suggests can obviously be used too, but personally I find that way too much hassle for no real advantage (other than maybe lower cost).

My Linux news / reviews / tips+tricks / downloads web site: http://www.linuxtech.net/
tux99 is offline  
Old 05-14-2010, 06:00 AM
Member
 
KCLC's Avatar
 
Join Date: Jun 2005
Posts: 20
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
RGB and Tux, you both are absolutely right. Large USB drives are usually sata drives and subject to all the failures of any hard drive. I just threw those out there as a possibility. All storage media is subject to failure, including optical. I work at a tv station where we are in the process of changing news archive storage from tape to optical, Sony XDCam disks. The very first disk sports used had 30 stories on it and none of them could be retrieved at a critical time, the firing of the Royals manager.
The purpose and use of the backup is a very important condsideration. We use, hard drive arrays, 4 different tape formats, external drives, blue ray and the afore mentioned XDCam disks, depending on what we are backing up or archiving. Each choice has it's strengths and weaknesses. The most critical part, as the sports incident illustrates, is retrieval. It seems obvious but if you can't restore your backup, it's not really a backup.
KCLC is offline  
Old 05-14-2010, 08:33 AM - Thread Starter
Rgb
AVS Special Member
 
Rgb's Avatar
 
Join Date: Apr 2000
Location: SE Michigan
Posts: 6,893
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 10 Post(s)
Liked: 19
Quote:
Originally Posted by KCLC View Post

RGB and Tux, you both are absolutely right. Large USB drives are usually sata drives and subject to all the failures of any hard drive. I just threw those out there as a possibility. All storage media is subject to failure, including optical. I work at a tv station where we are in the process of changing news archive storage from tape to optical, Sony XDCam disks. The very first disk sports used had 30 stories on it and none of them could be retrieved at a critical time, the firing of the Royals manager.
.

I wonder if those XDCam optical discs were authored/burned properly to begin with, or experienced some abnormal storage environment issues- left if a hot vehicle in sunlight, etc. Or maybe the playback equipment had issues, or they were recorded/handled by an inexperienced user?

In my experience, when people complain about optical media, their backup and/or storage processes weren't right- authored improperly, wrong software settings when burning, burned too fast, poor quality media, mishandled in use after burning, stored in a poor environment, etc. All simple issues to address- it's about establishing a good *process* and procedures and sticking with them once they are known "good".
Rgb is offline  
Old 05-14-2010, 09:04 AM
AVS Special Member
 
tux99's Avatar
 
Join Date: Jan 2005
Location: Europe
Posts: 1,523
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by Rgb View Post

In my experience, when people complain about optical media, their backup and/or storage processes weren't right- authored improperly, wrong software settings when burning, burned too fast, poor quality media, mishandled in use after burning, stored in a poor environment, etc. All simple issues to address- it's about establishing a good *process* and procedures and sticking with them once they are known "good".

You summarized it very well, the factors that could cause optical media to fail or become unreadable are so many that only experienced people can get it 100% right, most CD-R/DVD-Rs burnt by normal non-technical home users will be unreadable quite quickly.
That's why I wouldn't recommend any home user to rely mainly or exclusively on optical media.

Like I mentioned earlier, optical media can be useful but they aren't the be all end all solution for every backup problem.

My Linux news / reviews / tips+tricks / downloads web site: http://www.linuxtech.net/
tux99 is offline  
Old 05-15-2010, 11:51 PM
AVS Special Member
 
Phantom Gremlin's Avatar
 
Join Date: Aug 2002
Location: Tualatin Oregon
Posts: 1,407
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by tux99 View Post

most CD-R/DVD-Rs burnt by normal non-technical home users will be unreadable quite quickly.
That's why I wouldn't recommend any home user to rely mainly or exclusively on optical media.

That's what I've seen reported many times, though fortunately my personal experience is much better.

I'll throw out the following other memes I have about optical, pick them apart if you want to:
  • DVD+R is better than DVD-R (for some reason related to how it's stored on the media)
  • write-once is better than rewritable media, but that shouldn't be much of an issue for archival since you would probably be using the write-once stuff anyway
  • DVD-RAM is much more reliable than DVD-R or DVD+R
Phantom Gremlin is offline  
Old 05-16-2010, 06:28 AM - Thread Starter
Rgb
AVS Special Member
 
Rgb's Avatar
 
Join Date: Apr 2000
Location: SE Michigan
Posts: 6,893
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 10 Post(s)
Liked: 19
Quote:
Originally Posted by Phantom Gremlin View Post

That's what I've seen reported many times, though fortunately my personal experience is much better.

I'll throw out the following other memes I have about optical, pick them apart if you want to:
  • DVD+R is better than DVD-R (for some reason related to how it's stored on the media)
  • write-once is better than rewritable media, but that shouldn't be much of an issue for archival since you would probably be using the write-once stuff anyway
  • DVD-RAM is much more reliable than DVD-R or DVD+R

I think the benefit of +R is primarily with set top recorders- the ability to edit after writing and play without closing the disc and removing teh disc from the player without closing the session.

The -/+R debate has raged for years and I've never really heard a convincing reason why +R is better than -R for burning in PC drives, other than hints there is better track and burn control info/metadata on +R's. Since there is no cost difference, I've standardized on +R for the past 5 years.

I admit that the low capacity of DVD blank media makes this archive method more labor intensive, but once the process is good and you are experienced, it becomes second nature.

We *really* need 100GB+ optical media at $0.06/GB or less in the near future. Single layer BD-R 25GB discs are a big improvement over DVD in the short term, and maybe dual layer BD-R 50GB will become cost effective, hopefully sooner than later.

I have never even burned a dual layer DVD yet, though I want to experiment with it. I've found it difficult to justify the extra cost/GB of dual layer media, plus adding in compatibility issues with set top DVD players, although this may not be an issue any longer with recent vintage set tops.
Rgb is offline  
Old 05-16-2010, 10:59 AM
AVS Special Member
 
tux99's Avatar
 
Join Date: Jan 2005
Location: Europe
Posts: 1,523
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by Phantom Gremlin View Post

  • DVD-RAM is much more reliable than DVD-R or DVD+R

Agreed (maybe not 'much more' but certainly more), but when considering media for long term archives you need to take into account the availability of reader devices in 10-20 years.
I fear DVD-RAM is too uncommon and therefore it might be hard to get readers for it in the long term.

My Linux news / reviews / tips+tricks / downloads web site: http://www.linuxtech.net/
tux99 is offline  
Old 05-16-2010, 11:02 AM
AVS Special Member
 
tux99's Avatar
 
Join Date: Jan 2005
Location: Europe
Posts: 1,523
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by Rgb View Post

We *really* need 100GB+ optical media at $0.06/GB or less in the near future. Single layer BD-R 25GB discs are a big improvement over DVD in the short term, and maybe dual layer BD-R 50GB will become cost effective, hopefully sooner than later.

BD-R media is completely unproven with regards to durability for now, I wouldn't currently recommend it for long term archiving at all.

Quote:
Originally Posted by Rgb View Post

I have never even burned a dual layer DVD yet, though I want to experiment with it. I've found it difficult to justify the extra cost/GB of dual layer media, plus adding in compatibility issues with set top DVD players, although this may not be an issue any longer with recent vintage set tops.

I have experimented with dual layer DVDs, they are far less reliable then single layer DVD-R (and I only tried the best ones, Verbatim, not the cheaper brands), I wouldn't use them to store anything I can't afford to lose.

My Linux news / reviews / tips+tricks / downloads web site: http://www.linuxtech.net/
tux99 is offline  
Old 05-16-2010, 11:32 AM - Thread Starter
Rgb
AVS Special Member
 
Rgb's Avatar
 
Join Date: Apr 2000
Location: SE Michigan
Posts: 6,893
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 10 Post(s)
Liked: 19
Quote:
Originally Posted by tux99 View Post

BD-R media is completely unproven with regards to durability for now, I wouldn't currently recommend it for long term archiving at all.
.

Agreed- it takes years to reveal all the basic issues and get a good feel for the fundamental reliability/longevity, despite what all the BluRay fanboi's say
Rgb is offline  
Old 05-16-2010, 11:42 AM - Thread Starter
Rgb
AVS Special Member
 
Rgb's Avatar
 
Join Date: Apr 2000
Location: SE Michigan
Posts: 6,893
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 10 Post(s)
Liked: 19
Quote:
Originally Posted by tux99 View Post

You summarized it very well, the factors that could cause optical media to fail or become unreadable are so many that only experienced people can get it 100% right, most CD-R/DVD-Rs burnt by normal non-technical home users will be unreadable quite quickly.
That's why I wouldn't recommend any home user to rely mainly or exclusively on optical media.

Like I mentioned earlier, optical media can be useful but they aren't the be all end all solution for every backup problem.


...and home (i.e. non-technical or mildly saavy) users could build and/or maintain a RAID with sufficient redundancy?

After explaining the basic rules of burning DVD's:

Only use first class media per

http://www.digitalfaq.com/reviews/dvd-media.htm

Only buy discs in cakeboxes or jewel cases- never in bare shrink wrap.

Use IMGBurn (free beer, Windows or Wine)
Burn at no more than 4x
Keep blanks in cakebox until burned in a cool dry place out of sunlight.
Do not touch the recording surface before burning- after burning just clean with soap/water.
Store long term in any hub mounted/supported container- cakebox, dvd case, jewel case, etc in a cool dry environment out of sunlight.

Every non-techie I've advised has been able to burn what they need and play it where they need- any PC, set tops, etc.

Again, I recognise that for mass, multi-TB storage, individual 4.3GB discs are a hassle in handling. But that's a separate issue vs. long term viability/reliability/simplicity/cost per GB issues.
Rgb is offline  
 
Thread Tools


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off