AVS › AVS Forum › Gaming & Content Streaming › Networking, Media Servers & Content Streaming › Ripping large Blu Ray library (1000+), Recommendations needed for ripping method and playback device
New Posts  All Forums:Forum Nav:

Ripping large Blu Ray library (1000+), Recommendations needed for ripping method and playback device

post #1 of 55
9/26/12 at 7:36am
Thread Starter 
I've been reading through a lot of the threads in this forum trying to get advice for this, but it seems like every person's situation calls for something different.

Here is what I am currently trying to do. I have a movie library of over 1000 blu rays that I would like to put on hard drives so that I can have instant access to the movies, as well as whole house streaming.

The setup:
Everything will be wired.
Dedicated 7.1 theatre, along with several other tvs throughout the house

What I'm looking for:
I only care about the movies. I do not watch special features.
Subtitles will be needed. I have a large collection of Criterion/Foreign/Silent titles.
Lossless audio
Video Quality dependent on cost/benefit. I'm ok with compressing video as long as it's not overly noticeable, to save HDD space. (I realize this will probably open up some debate)

I'm doing this in preparation of moving into a house in about a year. I did a little bit of research prior to finding this forum, and have been using Handbrake. It looks like this may not have been the best method, so I might need to go back and start over. There's a lot more options than I had initially found. Currently, I have an old 5 HDD bay computer that I have filled with 3 TB drives for ~14TB of Media Storage space. I plan on building a RAID server for the new house and moving everything over at that time, but I don't have the time or space to currently do that.

So, my questions are:

What is the best method of ripping the movies from the disc to the HDD. I really like the simplicity of ripping the movie to mkv. I was under the impression that Handbrake compressed the video to make the file size manageable, but still gave me uncompressed audio but it looks like that may not have been the case. (Is that not what Auto Passthru did?) As mentioned earlier, I don't care at all about extras, so I don't see why it's important to me to use a disc image, but feel free to convince me otherwise. Chapter stops are also not of importance to me.

What is the best playback option. I built a HTPC about 5-7 years ago, but it looks like they're pretty outdated compared to all the new media streamers that are available now. I don't really use Netflix, Hulu, Amazon, or any of that. All I really want is something that will stream the movies and allow me to play subtitles. Would like something with a very nice GUI. Given the volume of my movie library, something with decent organization is also important. I currently use a WDTV to view the files I have stored. It serves it's purpose, but I've seen HTPC programs with much nicer functionality.
Reply
Reply

AVS Top Picks

post #2 of 55
9/26/12 at 8:58am
Use MakeMKV to rip the main movie and remux into a MKV file. No conversion is performed, include whatever language tracks and subtitles you want.

Significantly smaller container overhead.

If you want to compress the video you can do it at a later stage. Rip first.
Reply
Reply
post #3 of 55
9/26/12 at 9:42am
Even with movie-only ripping you're still going to need 30-40 TB of storage space. So that's probably where you should start. An NAS system is the best route for the storage, and playback devices at each location with wired network. Any media box that supports HD audio and video in MKV or M2TS files will suffice for each location. SMB/NFS support for connecting to the NAS is standard in these media boxes.

Obviously you also need a networked PC with a BD drive or two for ripping, and appropriate software. Plan on 30-45 min per disc for ripping.
Reply
Reply
post #4 of 55
9/26/12 at 11:47am
Quote:
Originally Posted by rdgrimes View Post

Even with movie-only ripping you're still going to need 30-40 TB of storage space. So that's probably where you should start. An NAS system is the best route for the storage, and playback devices at each location with wired network. Any media box that supports HD audio and video in MKV or M2TS files will suffice for each location. SMB/NFS support for connecting to the NAS is standard in these media boxes.
Obviously you also need a networked PC with a BD drive or two for ripping, and appropriate software. Plan on 30-45 min per disc for ripping.

I agree with everything above. I will also add to put 4 BD-Roms in your PC along with 2 HD's (once they are filled, remove and replace with empty ones), this way you can easily do 4 movies at a time which will cut down considerably on the amount of time this is going to take. Also having 4 ROM's will help because there will be a few BR's along the line that may not read in one ROM but will read in another one, even an identical model believe it or not.

The process is going to be a LONG one but will be WELL worth the time when it is complete. smile.gif

Bill
Reply
Reply
post #5 of 55
9/26/12 at 1:03pm
Quote:
Originally Posted by mrfattbill View Post

I agree with everything above. I will also add to put 4 BD-Roms in your PC along with 2 HD's (once they are filled, remove and replace with empty ones), this way you can easily do 4 movies at a time which will cut down considerably on the amount of time this is going to take.

I second this. Buy as many BD-roms as you can fit in the PC, and even consider getting a second PC, filled with BD-roms, for ripping. You'll save weeks, if not months, by doing this, and the cost is relatively small compared to the storage system and the original cost of the 1000 BDs.
Reply
Reply
post #6 of 55
9/26/12 at 1:48pm
wouldn't it take about a MONTH straight at 24 hours a day to rip 1000 blu rays at 40 minutes each? or am i off?
Reply
Reply
post #7 of 55
9/26/12 at 2:06pm
Quote:
Originally Posted by comicguy View Post

wouldn't it take about a MONTH straight at 24 hours a day to rip 1000 blu rays at 40 minutes each? or am i off?

You might be right. If we round it to 45 minutes, you'll rip 4 discs in 3 hours, or 32 discs in 24 hours.

1000 / 32 = 31days 6 hours

Obviously, you won't be ripping 24h a day (unless you build a ripping robot), so it'll take considerably longer. And that's why you should buy as many drives as you can, and rip in parallell. If you have 8 drives, and you rip 8 discs when you wake up, change the discs and rip 8 more when you get back from work, and finally put in 8 discs before you go to bed, you'll rip 24 discs a day without really putting any effort into it. You can easily add three more ripping sessions in the evening, increasing the total to 48 discs in a weekday. In the weekend, stop by the ripping computers once an hour or so, and you'll rip 16x8x2 (=256) discs during the weekend.

This way you'll have ripped almost 500 discs in a week.
Reply
Reply
post #8 of 55
9/26/12 at 2:18pm
I rip my retail BD as MainMovie to a folder structure, using DVDFab to remove the junk.
On Prince of Persia, this was the only way to get an accurate MainMovie. Clown 0.79 couldn't dig it out.

After that, I use the Clown_BD (package) to rebuild that folder structure back to a single .M2TS file.
Note the .M2TS is named for the output folder used by the eac3to output program. In my case, this is "..\Movie".

This operation consumes 3x the disk space of the BD disk by itself.
After the final .M2TS is created, the folders for DVDFab and ..\Movie can be cleared of the temp files.
This deletes 66% of the temp consumed space and returns it to the disk free pool.
The Movie.m2ts file is renamed, and moved to my server share.
Moving the movie file clears the remaining 33% of local disk space consumed by this operation.

The process is entirely manual, so I kick it off as I do my other work (I work from home).
On an i7/870 Lynnfield, the DVDFab takes 30~45 minutes.
Another 60 to 90 minutes is needed for the Clown package to complete.

If one BD takes 2 hours, a thousand BD will take 2,000 hours or 333 days if you do 3x daily.

I do hope your server is ZFS, RAID or backed up, because a disk crash would thrash many hours of work.
At 35gb per BD, a thousand BD is 35,000 gb or 35 TB or 18 drives at 2TB each.
The upper recommended limit for ZFS is about 40 drives, so 18 drives + parity drives is well within reach.

I really enjoy my subtitles, no menus, no previews, no spam and fast loading of the movie file.
This is what makes all the effort worthwhile to me.
Reply
Reply
post #9 of 55
9/26/12 at 3:54pm
Quote:
Originally Posted by bgavin View Post

The upper recommended limit for ZFS is about 40 drives, so 18 drives + parity drives is well within reach.

Max 40 drives on ZFS? Do you mean for a vdev, or a pool? I have 21 mirrored vdevs in my pool now (i.e. 42 drives), and I'm pretty satisfied so far. I'd hate it if I'm on the edge of what's considered safe.
Reply
Reply
post #10 of 55
9/26/12 at 4:49pm
The 40 drive limit is in their documentation.
ZFS is an Oracle inspiration, and the docs are there.

From the ZFS Best Practices.pdf:
Quote:
Avoid creating a RAIDZ, RAIDZ-2, RAIDZ-3, or a mirrored configuration with one logical device of 40+ devices.
Reply
Reply
post #11 of 55
9/26/12 at 7:08pm
Thread Starter 
Thanks for all the input so far. So is MakeMKV the consensus for the best ripping program to use? I've been using that for my criterions since I tend to care more about the PQ/AQ on those titles, but have been using Handbrake for the rest of them due wanting a smaller file size. Can someone tell me exactly what handbrake does when compressing the files? As I said earlier, I thought that by choosing Auto Passthru, I was getting the exact audio tracks from the blu ray. Is this not the case? I really liked that Handbrake ripped the files to about 1/3 the size of the lossless files that MakeMKV would put out, but it sounds like I'm losing more than I thought I was.

As for the time concerns, I'm still about a year out from having the setup complete, which is why I am starting the project now. I have two computers with BD drives and am able to get about 8-10 files ripped per day. I also bring my laptop to work with me two days a week and rip continuously while there. It's not a problem for me if it takes several months to get them all ripped, especially since it sounds like I might need to start over.
Reply
Reply
post #12 of 55
9/26/12 at 8:37pm
Quote:
Originally Posted by John_Galt View Post

Thanks for all the input so far. So is MakeMKV the consensus for the best ripping program to use? I've been using that for my criterions since I tend to care more about the PQ/AQ on those titles, but have been using Handbrake for the rest of them due wanting a smaller file size. Can someone tell me exactly what handbrake does when compressing the files? As I said earlier, I thought that by choosing Auto Passthru, I was getting the exact audio tracks from the blu ray. Is this not the case? I really liked that Handbrake ripped the files to about 1/3 the size of the lossless files that MakeMKV would put out, but it sounds like I'm losing more than I thought I was.

After Handbrake, does the movie still have lossless audio (DTS HDMA or Dolby TrueHD)? If so then Handbrake hasn't touched it. If not, you've lost something.
Reply
Reply
post #13 of 55
9/26/12 at 9:24pm
I have a tiny fraction of that number of movies but I want the very best quality regardless of how many so I use MakeMKV. I want video and audio untouched but the crap gone and it does a great job doing that.

If you are going to spend that amount of time ripping disks why would you not want the best quality? IN a few years you may have a 4K display, will you be happy with the compressed video then? Do uncompressed and buy a few more Hard Drives...
Reply
Reply
post #14 of 55
9/26/12 at 9:56pm
I personally would recommend main movie only with uncompressed (lossless) video and audio with your preferred subtitled. I would also recommend using Blu-ray folder structure instead of MKV.

Why Blu-ray folder structure? Because it will probably work 99.9% of the time on most devices.
Reply
Reply
post #15 of 55
9/26/12 at 10:19pm
Guys, one little question, what about 3D movies? How can I maintain video/audio quality and reduce size? Thank you.
Reply
Reply
post #16 of 55
9/27/12 at 7:18am
For lossless 3D, options are still limited. The latest DVDFab lets you do a movie-only rip (to ISO I believe), but haven't tried it myself. You can also use the latest MakeMKV to create a MVC MKV rip. From what I've read, it retains the base image and the right-eye information, but playback is tricky. For example, some Realtek 1186-based streamers will play it if you force playback as side-by-side.
Reply
Reply
post #17 of 55
9/27/12 at 7:39am
Quote:
Originally Posted by John_Galt View Post

Thanks for all the input so far. So is MakeMKV the consensus for the best ripping program to use? I've been using that for my criterions since I tend to care more about the PQ/AQ on those titles, but have been using Handbrake for the rest of them due wanting a smaller file size. Can someone tell me exactly what handbrake does when compressing the files? As I said earlier, I thought that by choosing Auto Passthru, I was getting the exact audio tracks from the blu ray. Is this not the case? I really liked that Handbrake ripped the files to about 1/3 the size of the lossless files that MakeMKV would put out, but it sounds like I'm losing more than I thought I was.

Handbrake re-encodes the video, and you can choose whether to re-encode or passthrough the audio. I can't remember if there is some restriction on which audio formats it can passthrough.

Re-encoding video is never lossless, even with the most expertly-chosen encoding settings. Since you are a beginner, your settings are likely very far from expert and so you are losing quite a lot of video quality.

If you are happy with that, then by all means continue. It is up to the individual to choose what level of video quality he is satisfied with.

If you want to make the audio tracks smaller, the simplest thing to do is to choose a smaller track at the ripping stage. For example choose the DTS core rather than the full DTS-HD track. Or if a 640kbps AC3 track is present, choose that over the alternatives. Again, it is up to the individual to choose what audio quality suffices.

Although personally I don't see the point in spending so much on a Bluray collection and then reducing the quality of your rips.
Reply
Reply
post #18 of 55
9/27/12 at 8:06am
Thread Starter 
You guys bring up some good points (which, I guess serves the whole purpose of asking for advice). My reasons for compressing the video silly comes from the 720 v 1080 debate from several years ago, where I never believed that there was a strong perceptible difference between the two from a standard set at standard viewing distance. There's also the possibility that with new format monitors coming out in the future, that there will be new format source material and I would need to repurchase my library at that time anyways. In an effort to future proof as much as possible, I probably should lean more towards the highest quality encoding, so I will rethink that decision. I was really trying to do this project as economically as possible. I recently quit my FT job to return to school for a year, so my disposable income has gone down dramatically compared to six months ago. Alternatively, I have plenty of free time to devote to this.

As for handbrake, when I play the files through my audio system, it does show up as the lossless format.

Someone mentioned 3D titles. I have a handful of them, but for the time being, I'm just playing those through my blu ray player and am not worried about ripping them. I'll wait until there are better ripping options before attempting those.

Someone mentioned ripping to a blu ray folder structure. What is the best method for that and how does the file size compare to a mkv from makemkv? I figure I'll play around with the two and see what I like better.
Reply
Reply
post #19 of 55
9/27/12 at 8:32am
This is what I've been using with all of my BD and DVD movies.

Encryption removal: AnyDVD HD. In my opinion the best.

BD ripping: ClownBD as main movie only, HD sound track and secondary AC3 sound track, force subs, Spanish / English subs as BR Folder structure. I do BR folder structure now to keep chapter support.

DVD ripping: MakeMKV, main movie, 5.1 sound track, force subs, Spanish/English subs.

DVD TV SHOWS (all my TV Shows are DVD only): I remove all the extras and do an ISO file with AnyDVD HD. However, I am thinking of going with single episodes file with MakeMKV for easier jukebox maintenance for Dune and PCH media players.
Reply
Reply
post #20 of 55
9/27/12 at 8:45am
I also like to use ClownBD to rip to M2TS or ISO format (movie only). I'd recommend trying several different tools and use what gives you the results you want. DVDFab has a free trial, ClownBD is free. I find MakeMKV to be too limited for everyday use. I prefer to do a full disc rip to HDD first, then process to a second HDD.
Reply
Reply
post #21 of 55
9/27/12 at 8:49am
Quote:
Originally Posted by rdgrimes View Post

I also like to use ClownBD to rip to M2TS or ISO format (movie only). I'd recommend trying several different tools and use what gives you the results you want. DVDFab has a free trial, ClownBD is free. I find MakeMKV to be too limited for everyday use. I prefer to do a full disc rip to HDD first, then process to a second HDD.

The cool thing about doing BR Folder structure with ClownBD is that it makes a single M2TS file with a mpls file, which keeps the chapter information. So, I can just take the M2TS file out of the BR folder, and make it into a single file M2TS movie wink.gif.
Reply
Reply
post #22 of 55
9/27/12 at 10:31am
I also use AnyDVD HD and ClownBD to rip to blu-ray folder structure. Just think of it as the main movie only blu-ray disk which if needed can be re-encoded latter. Another thing about blu-ray folder structure is that it is native with minimum BD coding and so most BD capable device should not have problem handling the video and audio correctly.
Reply
Reply
post #23 of 55
9/27/12 at 12:39pm
Quote:
Originally Posted by John_Galt View Post

Thanks for all the input so far. So is MakeMKV the consensus for the best ripping program to use? I've been using that for my criterions since I tend to care more about the PQ/AQ on those titles, but have been using Handbrake for the rest of them due wanting a smaller file size. Can someone tell me exactly what handbrake does when compressing the files? As I said earlier, I thought that by choosing Auto Passthru, I was getting the exact audio tracks from the blu ray. Is this not the case? I really liked that Handbrake ripped the files to about 1/3 the size of the lossless files that MakeMKV would put out, but it sounds like I'm losing more than I thought I was.
As for the time concerns, I'm still about a year out from having the setup complete, which is why I am starting the project now. I have two computers with BD drives and am able to get about 8-10 files ripped per day. I also bring my laptop to work with me two days a week and rip continuously while there. It's not a problem for me if it takes several months to get them all ripped, especially since it sounds like I might need to start over.

Best is a subjective term. MakeMKV is a very good choice, but isn't the best for defeating DRM. Compressed versus Uncompressed will be the other issue to consider. Ripping the movie only should take about 30-40 minutes. Compressing with Handbrake can take from a couple of hours to a day or more depending on quality settings and available computer power. Really, that's a whole different discussion that would open up a huge can of worms.

With a project that large, I would invest in AnyDVDHD to make sure I don't run into tany BR discs that I just can't rip. Personally, I use Tsmuxer to rip them, following this guide:

http://www.dvd-guides.com/guides/blu-ray-rip/256-remux-blu-ray-to-mkv

Except that instead of demuxing from Tsmuxer, I just select <.m2ts> as a destination and I end up with a 13GB-30GB movie only file that is not compressed any further than what's on the original BR. Don't have to worry about the remuxing steps with mkvtoolnix.

At that point you could decide whether to take the time to compress it further or not. If you have an i5-2500k or better, you could probably do a decent compression down to 4GB or so if you're willing to sacrifice some resolution and do it in less than 2 hours. By that time, though, you're into one BR for 3 hours or so. That's 3000 man hours for the library (120 days@24 hours a day). If quality is really important, plan on 6-8 hours to properly compress the file, but plan on spending the better part of a year setting up your library.
Reply
Reply
post #24 of 55
9/27/12 at 1:08pm
Quote:
Originally Posted by GusGus748s View Post

The cool thing about doing BR Folder structure with ClownBD is that it makes a single M2TS file with a mpls file, which keeps the chapter information. .

Roger that, and you can also use the option to make an ISO of the same structure. Lots of options there.
Reply
Reply
post #25 of 55
9/27/12 at 1:25pm
Quote:
Originally Posted by smitbret View Post

Best is a subjective term. MakeMKV is a very good choice, but isn't the best for defeating DRM. Compressed versus Uncompressed will be the other issue to consider. Ripping the movie only should take about 30-40 minutes. Compressing with Handbrake can take from a couple of hours to a day or more depending on quality settings and available computer power. Really, that's a whole different discussion that would open up a huge can of worms.
With a project that large, I would invest in AnyDVDHD to make sure I don't run into tany BR discs that I just can't rip. Personally, I use Tsmuxer to rip them, following this guide:
http://www.dvd-guides.com/guides/blu-ray-rip/256-remux-blu-ray-to-mkv
Except that instead of demuxing from Tsmuxer, I just select <.m2ts> as a destination and I end up with a 13GB-30GB movie only file that is not compressed any further than what's on the original BR. Don't have to worry about the remuxing steps with mkvtoolnix.
At that point you could decide whether to take the time to compress it further or not. If you have an i5-2500k or better, you could probably do a decent compression down to 4GB or so if you're willing to sacrifice some resolution and do it in less than 2 hours. By that time, though, you're into one BR for 3 hours or so. That's 3000 man hours for the library (120 days@24 hours a day). If quality is really important, plan on 6-8 hours to properly compress the file, but plan on spending the better part of a year setting up your library.

AHHHHHHHHHH, you guys that want to compress blurays down to 4GB KILL ME!!! Just go buy the damn dvd! Why oh WHY would you want to waste the time and energy to do THIS. Strip out the HD audio too while you are at it to save evn more space! AHHHHHHHHH
Reply
Reply
post #26 of 55
9/27/12 at 1:54pm
Quote:
Originally Posted by comicguy View Post

AHHHHHHHHHH, you guys that want to compress blurays down to 4GB KILL ME!!! Just go buy the damn dvd! Why oh WHY would you want to waste the time and energy to do THIS. Strip out the HD audio too while you are at it to save evn more space! AHHHHHHHHH

I agree with this. I say, store uncompressed disc images or just play off the discs themselves. An expertly encoded 1080p MKV (x264), of say ~20gb for a 2h movie will look very close to the source, but the effort required to save 15GB per title is not worth it to me (unless you have a massively powerful encoding rig and enjoy the process). HDD prices are dropping again, and WD is coming out with Hitatchi tech 3TB and 4TB drives this fall which will further help.
Reply
Reply
post #27 of 55
9/28/12 at 12:43am
Quote:
Originally Posted by rdgrimes View Post

Roger that, and you can also use the option to make an ISO of the same structure. Lots of options there.

+1 That's the main reason why I use the format because you have options later.
Quote:
Originally Posted by grambo View Post

I agree with this. I say, store uncompressed disc images or just play off the discs themselves. An expertly encoded 1080p MKV (x264), of say ~20gb for a 2h movie will look very close to the source, but the effort required to save 15GB per title is not worth it to me (unless you have a massively powerful encoding rig and enjoy the process). HDD prices are dropping again, and WD is coming out with Hitatchi tech 3TB and 4TB drives this fall which will further help.

I agree with this as well. I don't understand why people spend so much time to go from 1080p to 480p or lower when they can just buy the damn DVD, which is cheaper.
Reply
Reply
post #28 of 55
9/28/12 at 7:03am
Quote:
Originally Posted by GusGus748s View Post

The cool thing about doing BR Folder structure with ClownBD is that it makes a single M2TS file with a mpls file, which keeps the chapter information. So, I can just take the M2TS file out of the BR folder, and make it into a single file M2TS movie wink.gif.

For me, I'm spoiled that my WD Live player has a skip-to-time function so I have no use for the native chapter skip points. So, I just use Clown_BD to make the single BD.m2ts file that I store on my NAS. If I ever wanted to make it into a BDMV folder structure again I could just run it through TSMUXER and give it default 5 min chapter points. I don't ever use MKV. I see no reason to take the native .m2ts file off the BD disk and recode it into something else.
Reply
Reply
post #29 of 55
9/28/12 at 7:09am
Quote:
Originally Posted by comicguy View Post

AHHHHHHHHHH, you guys that want to compress blurays down to 4GB KILL ME!!! . . . WHY would you want to waste the time and energy to do THIS.

Yes the notion of compressing a BD to fit on a DVD-R is kind of comical for us. But over in the DVD Fab forum they are very concerned about compression -- usually down to 23GB -- so they can fit their Netflix rentals on a single BD-R.
Reply
Reply
post #30 of 55
9/28/12 at 7:22am
Quote:
Originally Posted by Kelson View Post

For me, I'm spoiled that my WD Live player has a skip-to-time function so I have no use for the native chapter skip points. So, I just use Clown_BD to make the single BD.m2ts file that I store on my NAS. If I ever wanted to make it into a BDMV folder structure again I could just run it through TSMUXER and give it default 5 min chapter points. I don't ever use MKV. I see no reason to take the native .m2ts file off the BD disk and recode it into something else.

I am not encoding anything. It is a single m2ts fiel with a mpls file, which contains the chapter information.
Reply
Reply
New Posts  All Forums:Forum Nav:
  Return Home

AVS Top Picks

AVS › AVS Forum › Gaming & Content Streaming › Networking, Media Servers & Content Streaming › Ripping large Blu Ray library (1000+), Recommendations needed for ripping method and playback device