Are MKV's really 1:1 copies? - AVS Forum
Forum Jump: 
Reply
 
Thread Tools
post #1 of 64 Old 06-06-2011, 04:15 AM - Thread Starter
AVS Special Member
 
wnorris's Avatar
 
Join Date: Jul 2003
Posts: 2,578
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
I've started to question whether any MKV program actually makes 1:1 copies of BD's or DVD's. Because of another issue, which I will probably start a thread about later, I've really been exploring MKV ripping options.

One thing I noticed is that with pretty much any MKV ripping option out there, it doesn't appear that a true 1:1 copy is being made.

For example, lets look at the Blu-ray for 30 Days of Night. The H264 stream on the disc has 162695 frames, according to eac3to.

If I use DVDFab to make a movie only copy in a m2ts, eac3to reports that file's H264 stream has 162695 frames.

However, if I used DVDFab to rip a MKV, eac3to reports that the H264 stream on that file has 162683 frames. That is 12 fewer frames than on the disc itself.

Now, if I use MakeMKV to rip the MKV, eac3to reports the H264 stream on that file has 325389 frames.

So in my mind, if the number of frames in the H264 differs from the source, then you are not getting a true 1:1 copy. Is my logic wrong here?
wnorris is offline  
Sponsored Links
Advertisement
 
post #2 of 64 Old 06-06-2011, 05:20 AM
Senior Member
 
drcos's Avatar
 
Join Date: Jul 2001
Location: right next to the Gulf coast of FLA
Posts: 315
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
MKVs are containers. There is no remuxing being done to put the information into the file.
I think something in your process is skewing your results.

I'll stop ripping my BDs when I can put them in and watch the movie without trailers, warnings, cutesy menus...
drcos is offline  
post #3 of 64 Old 06-06-2011, 06:39 AM - Thread Starter
AVS Special Member
 
wnorris's Avatar
 
Join Date: Jul 2003
Posts: 2,578
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
I don't see how there can be anything in my process to skew the results. I mean, with MakeMKV its simply drop the disc in, click a button, and you have your MKV.

eac3to is straightforward to use and I simply extract the H264 stream from the final output of these programs. I use the same command line in each case, only changing the source file portion of the command line.

It would be nice if someone else could repeat this test to verify. But at this point, I don't believe that any of these MKV generating programs are actually giving us 1:1 rips. The video files are being altered. I don't know what is going on with MakeMKV either. It almost appears as if it is doubling the number of frames. Yet the total file size of extracted H264 streams is nearly identical (identical for the source BD and DVD Fab movie only m2ts, slightly different for the rest, but only a few hundred k). I thought the size difference might be attributed to some sort of header compression or something of that nature, but header compression shouldn't alter the actual number of frames in the stream.

I have now performed this test on multiple BD discs (have yet to try DVD's) and get similar results regardless of the title. MKV generating programs appear to be altering the source.

My next step is to use eac3to to extract all the individual streams from the disc, and then remux them together with MKVMerge. Then use eac3to to re-extract the individual streams to see if they are still identical to the source, or if the MKV container alters the streams.
wnorris is offline  
post #4 of 64 Old 06-06-2011, 06:42 AM - Thread Starter
AVS Special Member
 
wnorris's Avatar
 
Join Date: Jul 2003
Posts: 2,578
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by drcos View Post
MKVs are containers. There is no remuxing being done to put the information into the file.
I think something in your process is skewing your results.
Also, remuxing is being done by both programs (DVDFabs settings for ripping mkv is even called mkv.remux). They however are not supposed to re-encode any of the streams.
wnorris is offline  
post #5 of 64 Old 06-06-2011, 07:03 AM
Senior Member
 
drcos's Avatar
 
Join Date: Jul 2001
Location: right next to the Gulf coast of FLA
Posts: 315
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
How are you getting frame information from eac3to? Here is what I get from eac3to for Tron (the original):
BD Movie-only rip-
M2TS, 1 video track, 1 audio track, 1 subtitle track, 1:35:47, 24p /1.001
1: Chapters, 19 chapters
2: h264/AVC, 1080p24 /1.001 (16:9)
3: DTS Master Audio, English, 5.1 channels, 24 bits, 48kHz
(core: DTS, 5.1 channels, 24 bits, 1509kbps, 48kHz)
4: Subtitle (PGS), English
Converted to MKV (MakeMKV)
MKV, 1 video track, 1 audio track, 1 subtitle track, 1:35:47, 24p /1.001
1: h264/AVC, 1080p24 /1.001 (16:9)
2: DTS Master Audio, 5.1 channels, 24 bits, 48kHz
(core: DTS, 5.1 channels, 24 bits, 1509kbps, 48kHz)
3: Subtitle (PGS)

The BD folder is 23.5GB. The MKV is 22.2GB.

I'll stop ripping my BDs when I can put them in and watch the movie without trailers, warnings, cutesy menus...
drcos is offline  
post #6 of 64 Old 06-06-2011, 07:18 AM - Thread Starter
AVS Special Member
 
wnorris's Avatar
 
Join Date: Jul 2003
Posts: 2,578
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by drcos View Post
How are you getting frame information from eac3to? Here is what I get from eac3to for Tron (the original):
BD Movie-only rip-
M2TS, 1 video track, 1 audio track, 1 subtitle track, 1:35:47, 24p /1.001
1: Chapters, 19 chapters
2: h264/AVC, 1080p24 /1.001 (16:9)
3: DTS Master Audio, English, 5.1 channels, 24 bits, 48kHz
(core: DTS, 5.1 channels, 24 bits, 1509kbps, 48kHz)
4: Subtitle (PGS), English
Converted to MKV (MakeMKV)
MKV, 1 video track, 1 audio track, 1 subtitle track, 1:35:47, 24p /1.001
1: h264/AVC, 1080p24 /1.001 (16:9)
2: DTS Master Audio, 5.1 channels, 24 bits, 48kHz
(core: DTS, 5.1 channels, 24 bits, 1509kbps, 48kHz)
3: Subtitle (PGS)

The BD folder is 23.5GB. The MKV is 22.2GB.
You have to demux to get the frame count. Demux track 2.

eac3to Filename.m2ts 2: Tron.h264

Just insert the file name for your m2ts. This will give you a file called Tron.h264 in the same folder as the source (or you can specify a path if you are ripping from a disc). This is the demuxed video stream. eac2to will generate a txt file that includes a frame count.

Or if you want to avoid all that command line typing, you can download the HDBRStreamExtractor, which is a GUI for the eac3to command line.

http://code.google.com/p/hdbrstreamextractor/

Then do the same again, but substitute your mkv filename and on the MKV it will be

eac3to Filename.mkv 1: Tron.h264

because the h264 stream is the first item in your MKV. This assumes your mkv and m2ts are in different folders. If they are in the same folder, call it TronMKV.h264 or some other name so it doesn't rewrite the stream. Then you will also have to extracted streams. If you believe eac3to's frame count is wrong, there are other programs you can use to get frame counts for these streams.
wnorris is offline  
post #7 of 64 Old 06-06-2011, 09:22 AM
AVS Addicted Member
 
dbone1026's Avatar
 
Join Date: Aug 2008
Location: Long Island, New York
Posts: 11,314
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 16 Post(s)
Liked: 71
I have heard nothing but issues with DVDfab created mkvs and honestly do not find it an adequate solution for mkvs.

As far as MakeMKV, I would be curious if you get the same results with both h264 and vc1. I know with eac3to if cannot determine the correct fps for h264 and instead defaults to 25fps (so for BluRay you have to manually set it back to 23.976). This is not an issue with VC-1. I am not sure what MakeMKV does to address this, but if you get inconsistent results between VC1 and H264 that could be an answer

Cheers,
Damian

MSS.net blog (contributing editor) -
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

Windows Entertainment and Connected Home MVP 2010-2012

dbone1026 is offline  
post #8 of 64 Old 06-06-2011, 09:33 AM - Thread Starter
AVS Special Member
 
wnorris's Avatar
 
Join Date: Jul 2003
Posts: 2,578
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by dbone1026 View Post

I have heard nothing but issues with DVDfab created mkvs and honestly do not find it an adequate solution for mkvs.

As far as MakeMKV, I would be curious if you get the same results with both h264 and vc1. I know with eac3to if cannot determine the correct fps for h264 and instead defaults to 25fps (so for BluRay you have to manually set it back to 23.976). This is not an issue with VC-1. I am not sure what MakeMKV does to address this, but if you get inconsistent results between VC1 and H264 that could be an answer

Well, incorrect frame rate detection shouldn't effect the total number of frames. Total number of frames x frame rate= runtime. Even if it were back calculating total number of frames based on runtime and frame rate, it would result in a 4% difference in total number of frames if the framerate was incorrectly assumed to be 25 fps.

That would be a difference of 6500 frames, not the 12 you see with DVDFab or the 160,000+ difference with MakeMKV.

However, I don't think the bug you are referring to is with eac3to. I think that is with MKV Merge. eac3to correctly detects H264 frame rates (look at drocs post, both his m2ts and mkv are properly detected by eac3to).
wnorris is offline  
post #9 of 64 Old 06-06-2011, 09:38 AM
AVS Addicted Member
 
dbone1026's Avatar
 
Join Date: Aug 2008
Location: Long Island, New York
Posts: 11,314
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 16 Post(s)
Liked: 71
Quote:
Originally Posted by wnorris View Post

Well, incorrect frame rate detection shouldn't effect the total number of frames. Total number of frames x frame rate= runtime. Even if it were back calculating total number of frames based on runtime and frame rate, it would result in a 4% difference in total number of frames if the framerate was incorrectly assumed to be 25 fps.

That would be a difference of 6500 frames, not the 12 you see with DVDFab or the 160,000+ difference with MakeMKV.

However, I don't think the bug you are referring to is with eac3to. I think that is with MKV Merge. eac3to correctly detects H264 frame rates (look at drocs post, both his m2ts and mkv are properly detected by eac3to).

Duh, sorry about that, meant mkv merge. SO that still begs the question why mkvmerge has the issue and what does MakeMKV do to circumvent this

Cheers,
Damian

MSS.net blog (contributing editor) -
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

Windows Entertainment and Connected Home MVP 2010-2012

dbone1026 is offline  
post #10 of 64 Old 06-06-2011, 10:00 AM - Thread Starter
AVS Special Member
 
wnorris's Avatar
 
Join Date: Jul 2003
Posts: 2,578
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by dbone1026 View Post

Duh, sorry about that, meant mkv merge. SO that still begs the question why mkvmerge has the issue and what does MakeMKV do to circumvent this

Yes, that also leads me to the conclusion that their is no such thing as a lossless MKV rip with the current crop of software. MakeMKV appears to do frame interpolation/ frame doubling. DVDFab drops at least a few frames compared to the source file. MKV Merge has a few glitches, like not being able to detect H264 frame rate (which eac3to can detect it, so why not MKV Merge), plus it it really slow to demux all the streams and then remux into MKV.

It's like MKV can be 99.9% lossless, but it can't be 100% lossless like ISO, m2ts, etc.
wnorris is offline  
post #11 of 64 Old 06-06-2011, 10:29 AM
Advanced Member
 
mbryanr's Avatar
 
Join Date: Nov 2009
Posts: 539
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Note another reference to frame doubling (with no response)

http://www.makemkv.com/forum2/viewto...ing+mkv#p13719
mbryanr is offline  
post #12 of 64 Old 06-06-2011, 10:36 AM - Thread Starter
AVS Special Member
 
wnorris's Avatar
 
Join Date: Jul 2003
Posts: 2,578
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by mbryanr View Post

Note another reference to frame doubling (with no response)

http://www.makemkv.com/forum2/viewto...ing+mkv#p13719

Thats me. I'm on the MakeMKV forums trying to get some insight. If know that with ffdshow there was a frame doubling filter that was supposed to improve output and not require much, if any, additional space. I don't know how it worked though. I also found a reference to frame interpolation in MKV files, basically the same as motion smoothing on TV's. I couldn't find much in depth information on it, just that you could create MKV's that have interpolated frames. Not sure how this would increase the file size or effect playback, just that it is possible. So MakeMKV could also be doing something like that too. Which if it is frame doubling of doing interpolation, then again, you aren't getting a 1:1 rip.
wnorris is offline  
post #13 of 64 Old 06-06-2011, 10:45 AM
AVS Addicted Member
 
dbone1026's Avatar
 
Join Date: Aug 2008
Location: Long Island, New York
Posts: 11,314
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 16 Post(s)
Liked: 71
Quote:
Originally Posted by wnorris View Post

Thats me. I'm on the MakeMKV forums trying to get some insight. If know that with ffdshow there was a frame doubling filter that was supposed to improve output and not require much, if any, additional space. I don't know how it worked though. I also found a reference to frame interpolation in MKV files, basically the same as motion smoothing on TV's. I couldn't find much in depth information on it, just that you could create MKV's that have interpolated frames. Not sure how this would increase the file size or effect playback, just that it is possible. So MakeMKV could also be doing something like that too. Which if it is frame doubling of doing interpolation, then again, you aren't getting a 1:1 rip.

Are you a KY rep?

Cheers,
Damian

MSS.net blog (contributing editor) -
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

Windows Entertainment and Connected Home MVP 2010-2012

dbone1026 is offline  
post #14 of 64 Old 06-06-2011, 12:53 PM
Senior Member
 
peterjcat's Avatar
 
Join Date: Jul 2007
Posts: 288
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
There's not going to be any smoothing-type interpolation etc going on -- that would involve encoding frames and would take forever. If the frames really were doubled the file size would be double too. I would guess that something is being either mislabelled or misreported, or perhaps there are junk frames at the beginning or end that are being added or removed. It should be a 1:1 copy in that every actual video frame is reproduced without alteration. Why don't you step through the various versions frame by frame and see if anything's different?
peterjcat is offline  
post #15 of 64 Old 06-06-2011, 04:41 PM
Senior Member
 
drcos's Avatar
 
Join Date: Jul 2001
Location: right next to the Gulf coast of FLA
Posts: 315
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by dbone1026 View Post

As far as MakeMKV...I know with eac3to if cannot determine the correct fps for h264 and instead defaults to 25fps (so for BluRay you have to manually set it back to 23.976).

I have yet to do an h264 conversion with MakeMKV that it doesn't miss the fps. At least it reminds me to set the fps (to 24/10001).

I will look more into this frame thing.

I'll stop ripping my BDs when I can put them in and watch the movie without trailers, warnings, cutesy menus...
drcos is offline  
post #16 of 64 Old 06-06-2011, 07:13 PM
AVS Special Member
 
MarkHotchkiss's Avatar
 
Join Date: Mar 2007
Location: Long Beach, California
Posts: 1,476
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 82
Quote:
Originally Posted by peterjcat View Post

. . . Why don't you step through the various versions frame by frame and see if anything's different?

God, that could be awfully tedious for 162 thousand frames. However, it is worth a try at the beginning and ending, and maybe at a chapter boundary. It could something as simple as the last 12 frames of the end-credits were chopped-off (a buffering issue).
MarkHotchkiss is offline  
post #17 of 64 Old 06-07-2011, 03:29 AM
Senior Member
 
peterjcat's Avatar
 
Join Date: Jul 2007
Posts: 288
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by MarkHotchkiss View Post

God, that could be awfully tedious for 162 thousand frames. However, it is worth a try at the beginning and ending, and maybe at a chapter boundary. It could something as simple as the last 12 frames of the end-credits were chopped-off (a buffering issue).

That's more what I had in mind
peterjcat is offline  
post #18 of 64 Old 06-11-2011, 12:51 PM - Thread Starter
AVS Special Member
 
wnorris's Avatar
 
Join Date: Jul 2003
Posts: 2,578
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
I've been working with Mike more on the MKV problem.

http://www.makemkv.com/forum2/viewto...p=13865#p13865

http://www.makemkv.com/forum2/viewtopic.php?f=1&t=3660

It is becoming increasingly more obvious that there is a bug in MakeMKV.

I strongly recommend that if you are using MakeMKV 1.6.8 or 1.6.10 with Blu-rays that you stop doing so immediately. The evidence strongly suggest that you are making flawed and corrupted MKV's. Even earlier versions do not appear to be giving you true 1:1 copies, but the files are not corrupted.
wnorris is offline  
post #19 of 64 Old 06-12-2011, 05:44 AM
Member
 
lmaolmao's Avatar
 
Join Date: Oct 2008
Posts: 179
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
i wouldn't worry about it unless you see some corruption.

mkv just strips out the SPS and PPS from each frame in the .h264 and muxes it into the main header.

it basically makes your file non-BD compatible, but that's only a problem if you want to put it back in .m2ts and burn it on a BD25/50.
lmaolmao is offline  
post #20 of 64 Old 06-13-2011, 08:03 AM - Thread Starter
AVS Special Member
 
wnorris's Avatar
 
Join Date: Jul 2003
Posts: 2,578
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by lmaolmao View Post

i wouldn't worry about it unless you see some corruption.

mkv just strips out the SPS and PPS from each frame in the .h264 and muxes it into the main header.

it basically makes your file non-BD compatible, but that's only a problem if you want to put it back in .m2ts and burn it on a BD25/50.

According to the creator of MakeMKV, the streams extracted back out of the MKV with eac3to should be identical to the streams extracted from the physical disc, all the way down to identical MD5 checksums. MakeMKV is supposed to just be putting the streams in a container, the same as MKV Merge.

Further, after investigating the output from MakeMKV for DVD's (MPEG2), MakeMKV appears to actually be altering the duration of the title. The reported duration of a title actually appears to decrease a couple of seconds after MakeMKV handles it. It looks like this may be a frame rate issue. 29.97 fps material being processed as 30 fps, and likewise 23.97 fps material processed as 24 fps. This minor speedup causes the film duration to be shorter by ~ .1%. This also means you are altering the pitch of the audio slightly. Nothing like PAL at 4%, but still not a 1:1 copy. Not 100% sure if this is the problem, but the symptoms seem to match the theory.

The extracted m2v files from the MakeMKV output do not match the extracted streams made from the disc with DVD Decrypter or PGC Demuxer.

Having repeated these tests with MKV Merge, he streams remain identical. The only current method I have found of making true 1:1 MKV's is for a Blu-ray workflow of:

Eac3to to extract streams from the physical disc.
BDSup2Sub to check subs for forced subtitles (optional).
Remux with MKV Merge GUI.

and DVD workflow of:

PGC Demuxer (or DVD Decrypter) to extract streams.
SubRip to check subs for forced subtitles (optional).
ChapterXtractor to generate MKV compatible chapter txt file.
Remux with MKV Merge GUI.

So far using this workflow results in MKVs maintain a true 1:1 copy to the original source material, assuming you are aware of some of the pitfalls (like needing to specify frame rate form AVC, etc.). It is slightly less convenient than a one-for-all MakeMKV software, but not much more time consuming.

I know eac3to, BDSub2Sub and the MKV toolnix all are command line, so someone could (Or maybe already have) create a GUI that uses all three and simplifies the workflow. But on the DVD side, I don't know if those programs have a command line alternative. It looks like ChapterXtractor, PGC Demux have command line operation. SubRip does not. So if there was a command line replacement program that could check for forced subs and extract only forced subs, then a GUI could also be made to automate the process for DVDs.

So it is possible to have 1:1 conversions to MKV. You just can't get there with two of the biggest one-for-all MKV rippers on the market, MakeMKV and DVDFab.
wnorris is offline  
post #21 of 64 Old 06-16-2011, 06:02 AM
AVS Addicted Member
 
jmpage2's Avatar
 
Join Date: Feb 2005
Posts: 10,053
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 77
Have you pointed out the problem with MakeMKV to the author?
jmpage2 is offline  
post #22 of 64 Old 06-16-2011, 06:36 AM - Thread Starter
AVS Special Member
 
wnorris's Avatar
 
Join Date: Jul 2003
Posts: 2,578
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by jmpage2 View Post

Have you pointed out the problem with MakeMKV to the author?

I am still troubleshooting the problems with the developer. So far we have confirmed one bug that will be fixed in the next release. H264 streams have a header for each frame. MakeMKV was actually inserting two headers for each frame in its conversion. eac3to counts the number of headers to determine the number of frames. Since the MakeMKV files had 2X the number of headers, eac3to reported 2X the number of frames. This also explained why MakeMKV H264 streams were also larger than the original extracted streams.

There also appears to be a bug when ripping DTS-HD MA audio tracks. We have confirmed that the extracted stream is altered from the source, but at this point I don't think they have discovered why yet or what is actually being altered. I haven't done any testing yet to see if it effects other audio formats too.

I have also discovered that if you use MakeMKV and make multiple rips of the same material, no two files will have the same checksum. If you extract the individual streams, the separate streams will have matching checksums, but the file as a whole will never match. I passed this on to them, but haven't received any explanation yet as to why this occurs. It could be a bug, or it could be that some type of random seeding occurs as part of the decryption process and somehow this random seeding makes its way into some headers or something, preventing two files from ever being the same. In a way, I could see this as a negative because it basically means the every rip has its own unique fingerprint. This could make them easier to track and trace, depending on how the seeding was done. If you have 10,000 people using DVD FAB to make a MKV, and all 10,000 files have an identical checksum, it is much harder to trace a copy back to the person who actually ripped it. But if you have 10,000 people using MakeMKV, and all 10,000 files have a unique checksum, it makes it easier to try to trace to the source. Probably not a feature that a person sharing torrents of ripped discs would want. And depending on how the "random" seed is determined, could make it even worse. I have seen random seeding used where the seed value is determined by things like processor serial number, video card serial number, etc. (usually more than one of these values are used in the process). If someone reverse engineers how the seed is determined, its even possible they could link a certain MKV rip back to the processor or MB that created it. I'm not saying that this is what happens, but until the issue is explored, the potential for it is there I guess. Hopefully they explain it to me, because other programs do not behave this way.

We are still trying to work through all the Blu-ray issues and haven't gotten to troubleshooting anything on the DVD side yet. They did say that sometimes streams get padded with zeros for disc data rate purposes (controls where data physically ends up on the disc) and MakeMKV removes this zero padding when its found. From my research into it, this was more commonly done on DVD and isn't typically done on Blu-ray authoring.

This could potentially be the source of the differences I see on DVD, but I don't understand how the final output could be a few seconds different than the padded original. The padding should not effect the runtime of a feature. But we really haven't went into troubleshooting DVD output yet.

Its fairly time consuming for me to keep making all these rips and then extracting streams, etc. Other demands have forced me to step away from it for the past few days.
wnorris is offline  
post #23 of 64 Old 06-16-2011, 09:24 AM
AVS Special Member
 
sean_w_smith's Avatar
 
Join Date: Jan 2005
Location: Los Altos, CA and Huntsville, UT
Posts: 4,292
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
Quote:
Originally Posted by wnorris View Post

I am still troubleshooting the problems with the developer. So far we have confirmed one bug that will be fixed in the next release. H264 streams have a header for each frame. MakeMKV was actually inserting two headers for each frame in its conversion. eac3to counts the number of headers to determine the number of frames. Since the MakeMKV files had 2X the number of headers, eac3to reported 2X the number of frames. This also explained why MakeMKV H264 streams were also larger than the original extracted streams.

There also appears to be a bug when ripping DTS-HD MA audio tracks. We have confirmed that the extracted stream is altered from the source, but at this point I don't think they have discovered why yet or what is actually being altered. I haven't done any testing yet to see if it effects other audio formats too.

I have also discovered that if you use MakeMKV and make multiple rips of the same material, no two files will have the same checksum. If you extract the individual streams, the separate streams will have matching checksums, but the file as a whole will never match. I passed this on to them, but haven't received any explanation yet as to why this occurs. It could be a bug, or it could be that some type of random seeding occurs as part of the decryption process and somehow this random seeding makes its way into some headers or something, preventing two files from ever being the same. In a way, I could see this as a negative because it basically means the every rip has its own unique fingerprint. This could make them easier to track and trace, depending on how the seeding was done. If you have 10,000 people using DVD FAB to make a MKV, and all 10,000 files have an identical checksum, it is much harder to trace a copy back to the person who actually ripped it. But if you have 10,000 people using MakeMKV, and all 10,000 files have a unique checksum, it makes it easier to try to trace to the source. Probably not a feature that a person sharing torrents of ripped discs would want. And depending on how the "random" seed is determined, could make it even worse. I have seen random seeding used where the seed value is determined by things like processor serial number, video card serial number, etc. (usually more than one of these values are used in the process). If someone reverse engineers how the seed is determined, its even possible they could link a certain MKV rip back to the processor or MB that created it. I'm not saying that this is what happens, but until the issue is explored, the potential for it is there I guess. Hopefully they explain it to me, because other programs do not behave this way.

We are still trying to work through all the Blu-ray issues and haven't gotten to troubleshooting anything on the DVD side yet. They did say that sometimes streams get padded with zeros for disc data rate purposes (controls where data physically ends up on the disc) and MakeMKV removes this zero padding when its found. From my research into it, this was more commonly done on DVD and isn't typically done on Blu-ray authoring.

This could potentially be the source of the differences I see on DVD, but I don't understand how the final output could be a few seconds different than the padded original. The padding should not effect the runtime of a feature. But we really haven't went into troubleshooting DVD output yet.

Its fairly time consuming for me to keep making all these rips and then extracting streams, etc. Other demands have forced me to step away from it for the past few days.

cool stuff.....

Sean
sean_w_smith is offline  
post #24 of 64 Old 06-16-2011, 09:27 AM
AVS Addicted Member
 
jmpage2's Avatar
 
Join Date: Feb 2005
Posts: 10,053
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 77
Unfortunately this leaves me concerned as I have already ripped a ton of my stuff with MakeMKV. Rather surprising that these kind of issues exist on a product that is supposed to be out of beta.
jmpage2 is offline  
post #25 of 64 Old 06-16-2011, 10:40 AM - Thread Starter
AVS Special Member
 
wnorris's Avatar
 
Join Date: Jul 2003
Posts: 2,578
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by jmpage2 View Post

Unfortunately this leaves me concerned as I have already ripped a ton of my stuff with MakeMKV. Rather surprising that these kind of issues exist on a product that is supposed to be out of beta.

MakeMKV is still in beta. Visit their website and it even states, Welcome to MakeMKV Beta. Run the software and the banner at top says MakeMKV Beta. I don't think it will be out of Beta until 2.0.

But yeah, it does suck. They did say that the double headers in H264 shouldn't effect playback (partly why it has gone undetected for so long). My experience is that while it might not normally effect playback on most devices, there are probably some out there that it might cause problems for.

And since they haven't figured out the DTS-HD bug, they don't know yet if it has been there all along, or something recently introduced. There is another DTS bug where if you have two DTS tracks in one MKV it doesn't play correctly. That bug was new to 1.6.10, so obviously there have been some recent changes on the audio side.

The part that really sucks about all this is that I am sort of particular about everything being a 1:1 copy (otherwise I would just re-encode everything on DVD at least). Also, once scared away, it isn't easy for me to go back. So at this point I don't trust the MakeMKV software. Even if I go through this very lengthy debugging with them (I've already spent a ton of hours doing this debug) and we get H264 and DTS-HD working perfectly, I still wont trust the software until I've ran through similar testing with MPEG2, VC-1, AC3, TrueHD, PCM, etc. And then go through it for all the flavors of DVD streams.

So I'm looking at hundreds of hours of testing ahead of me to feel comfortable going back to the software, assuming the bugs are fixed.

And to go back and fix everything, I would have thousands of MKVs to re-encode (unless they provide a tool to remove the double headers).

Up until version 1.6.8, I had experienced zero problems with any of my MKVs. So I am assuming at this point, they are all good (even with double headers with H264). But I've identified every MKV I made using 1.6.8 and newer and I am in the process of going back to disc and re-encoding with my new method for all those titles. So I still have hundreds of MKV's that have to be recreated. Time I could have spent watching MKVs instead of making them.
wnorris is offline  
post #26 of 64 Old 06-16-2011, 10:43 AM
AVS Addicted Member
 
jmpage2's Avatar
 
Join Date: Feb 2005
Posts: 10,053
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 77
Beta or not, MakeMKV gave me fits when my "test" license expired. At the time there was not another beta key available so I ended up registering it for $50 (and got myself on some nasty spam lists in the process).

Oh, well, that's life.
jmpage2 is offline  
post #27 of 64 Old 06-16-2011, 11:53 AM
AVS Addicted Member
 
dbone1026's Avatar
 
Join Date: Aug 2008
Location: Long Island, New York
Posts: 11,314
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 16 Post(s)
Liked: 71
Quote:
Originally Posted by wnorris View Post
MakeMKV is still in beta. Visit their website and it even states, Welcome to MakeMKV Beta. Run the software and the banner at top says MakeMKV Beta. I don't think it will be out of Beta until 2.0.

But yeah, it does suck. They did say that the double headers in H264 shouldn't effect playback (partly why it has gone undetected for so long). My experience is that while it might not normally effect playback on most devices, there are probably some out there that it might cause problems for.

And since they haven't figured out the DTS-HD bug, they don't know yet if it has been there all along, or something recently introduced. There is another DTS bug where if you have two DTS tracks in one MKV it doesn't play correctly. That bug was new to 1.6.10, so obviously there have been some recent changes on the audio side.

The part that really sucks about all this is that I am sort of particular about everything being a 1:1 copy (otherwise I would just re-encode everything on DVD at least). Also, once scared away, it isn't easy for me to go back. So at this point I don't trust the MakeMKV software. Even if I go through this very lengthy debugging with them (I've already spent a ton of hours doing this debug) and we get H264 and DTS-HD working perfectly, I still wont trust the software until I've ran through similar testing with MPEG2, VC-1, AC3, TrueHD, PCM, etc. And then go through it for all the flavors of DVD streams.

So I'm looking at hundreds of hours of testing ahead of me to feel comfortable going back to the software, assuming the bugs are fixed.

And to go back and fix everything, I would have thousands of MKVs to re-encode (unless they provide a tool to remove the double headers).

Up until version 1.6.8, I had experienced zero problems with any of my MKVs. So I am assuming at this point, they are all good (even with double headers with H264). But I've identified every MKV I made using 1.6.8 and newer and I am in the process of going back to disc and re-encoding with my new method for all those titles. So I still have hundreds of MKV's that have to be recreated. Time I could have spent watching MKVs instead of making them.
That is a shame. I never could quite get comfortable with MakeMKV to use (I just stuck with Clown_BD + mkvmerge). Just some of the mkv issues (handling of forced subs, header compression, etc...) led me to move away from mkvs and instead just do folder rips with Clown_BD (and tbh since I started I have not had one issue). The only downside is that there is very little support for transcoding folder rips to mobile devices, but right now I rarely ever do this (and if I plan on playing a movie via mobile I just take the extra hour to create an m4v mobile version and copy over)

Cheers,
Damian

MSS.net blog (contributing editor) -
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

Windows Entertainment and Connected Home MVP 2010-2012

dbone1026 is offline  
post #28 of 64 Old 06-16-2011, 12:50 PM
AVS Special Member
 
sean_w_smith's Avatar
 
Join Date: Jan 2005
Location: Los Altos, CA and Huntsville, UT
Posts: 4,292
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
Quote:
Originally Posted by dbone1026 View Post
That is a shame. I never could quite get comfortable with MakeMKV to use (I just stuck with Clown_BD + mkvmerge). Just some of the mkv issues (handling of forced subs, header compression, etc...) led me to move away from mkvs and instead just do folder rips with Clown_BD (and tbh since I started I have not had one issue). The only downside is that there is very little support for transcoding folder rips to mobile devices, but right now I rarely ever do this (and if I plan on playing a movie via mobile I just take the extra hour to create an m4v mobile version and copy over)
They got those issues worked out, BTW:

I would not be surprised if there were bugs in these other solutions too. Most of us are not going to microanalyze all the possibilties and do the work to really see if most of the programs are indeed doing what they say they are. I hope they get this worked out. I like there software.

I really enjoy having chapter support verus m2ts and the 1 step process....

Sean
sean_w_smith is offline  
post #29 of 64 Old 06-16-2011, 12:54 PM
AVS Addicted Member
 
dbone1026's Avatar
 
Join Date: Aug 2008
Location: Long Island, New York
Posts: 11,314
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 16 Post(s)
Liked: 71
Quote:
Originally Posted by sean_w_smith View Post
They got those issues worked out, BTW:

I would not be surprised if there were bugs in these other solutions too. Most of us are not going to microanalyze all the possibilties and do the work to really see if most of the programs are indeed doing what they say they are. I hope they get this worked out. I like there software.

I really enjoy having chapter support verus m2ts and the 1 step process....

Sean
Who is they? forced subtitles don't work 100%, depends on what hardware/software player you are using.

Also, I am ripping to BR Folder which maintains chapters and with Clown_BD is just as much a 1 step process as MakeMKV

Cheers,
Damian

MSS.net blog (contributing editor) -
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

Windows Entertainment and Connected Home MVP 2010-2012

dbone1026 is offline  
post #30 of 64 Old 06-16-2011, 02:21 PM
AVS Special Member
 
sean_w_smith's Avatar
 
Join Date: Jan 2005
Location: Los Altos, CA and Huntsville, UT
Posts: 4,292
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
Quote:
Originally Posted by dbone1026 View Post

Who is they? forced subtitles don't work 100%, depends on what hardware/software player you are using.

Also, I am ripping to BR Folder which maintains chapters and with Clown_BD is just as much a 1 step process as MakeMKV

They = MKV authors

Forced subtitles work fine for me on the few titles I tested on the PCH C-200.

Sean
sean_w_smith is offline  
Reply Networking, Media Servers & Content Streaming

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off