From the Superbit discussion:

THe 138 minutes is based on a full muxed BD50. It will run at its peak value the entire time (48mbs)

Of course no movie needs that so it really have plenty of space over.

IIRC, 40mbs for vid, 8mbs for audio? But that really doesn't matter to my question.

So, for a 5.1 48khz PCM audio track, if my math is correct -

16bit: (6ch*16bits*48000hz) = (4.608mbs)*60sec/min*60min/hr = 16.58 Gb/hr or 2.073GB/hr (about 4.15GB for a two hr movie)

24bit: (6ch*24bits*48000hz) = (6.912mbs)*60sec/min*60min/hr = 24.88 Gb/hr or 3.110GB/hr (about 6.22GB for a two hr movie)

Now, if you compress it with Dolby TrueHD, DTS-HDMA or some other lossless codec, it will take up even less space.

But, here is my question: If PCM is the "raw" audio in its uncompressed state (and we have an example of its size above), what is the bitrate/size of "raw" video? There are different codecs for video (mpeg2/mpeg4, VC-1, etc.), and I have yet to see a BD/HDDVD without compression. How much "quality" are we missing by using compression on the video? How much uncompressed video can a BD/HDDVD hold?

Just throwing some A/V geek questions out there.

Jim

Video is usually compressed 100:1 or more. 200GB per hour?

So, after a little thinking, how about this:

1920X1080 bits = 2,073,600 bits
2,073,600bits*24frames/sec = 49.766Mbs?

Now, if we extrapolate it:

49.766Mbs*60sec/min*60min/hr = 179.159 Gb/hr = 22.39 GB/hr

Does that make sense? If so (and I think there is an error somewhere), for a typical 90 minute or less movie, you shouldn't any compression for video or audio. Something must be off.

You can compute the source data rate pretty easily.

We have 1920x1080 pixels = 2073600 pixels (about 2 "megapixels")
For progressive movie content, we have 24 frames per second which means we have 49766400 pixels per second.

Now the question is, how many bits each pixel take. To save space, we use the "YUV" color space for compressing video. The sampling rate for each component is specified relative to each other. In case of HD optical, all codecs in both formats are forced to use 4:2:0. This means for every luminance (black & white Y component) sample, we have half as much data for "chroma"/color (UV component).

The resolution for these formats is also fixed at 8 bits. This means that the Y component is 8 bits and we have 4 bits for color for a total of 8+4 = 12 bits/pixel. Quite a saving over 24-bits used in your computer desktop to read this text!

So let's convert the above pixels/sec to bits/sec by multiplying it by 12 which gets us 597,196,800 bits/sec. In other words, the data rate is nearly 600 megabits/sec if my math is right.

So if you encode said video at say, 30 megabits/sec, you are compressing 20:1. Or put another way, 95% is thrown away and the resulting 5% is what you are looking at :).

You can compute the source data rate pretty easily.

We have 1920x1080 pixels = 2073600 pixels (about 2 "megapixels")
For progressive movie content, we have 24 frames per second which means we have 49766400 pixels per second.

Now the question is, how many bits each pixel take. To save space, we use the "YUV" color space for compressing video. The sampling rate for each component is specified relative to each other. In case of HD optical, all codecs in both formats are forced to use 4:2:0. This means for every luminance (black & white Y component) sample, we have half as much data for "chroma"/color (UV component).

The resolution for these formats is also fixed at 8 bits. This means that the Y component is 8 bits and we have 4 bits for color for a total of 8+4 = 12 bits/pixel. Quite a saving over 24-bits used in your computer desktop to read this text!

So let's convert the above pixels/sec to bits/sec by multiplying it by 12 which gets us 597,196,800 bits/sec. In other words, the data rate is nearly 600 megabits/sec if my math is right.

So if you encode said video at say, 30 megabits/sec, you are compressing 20:1. Or put another way, 95% is thrown away and the resulting 5% is what you are looking at :).

In your opinion, what does "perceptually lossless to the master" mean? This question isn't necessarily related to your above post.

By the way, the real source is 4:2:2 and hopefully in 10-bits. This increases the data rate to almost 800 mbit/sec. We get it down to 600 mbit/sec using decimation filtering (i.e. throwing away) some of the data before encoding.

http://www.aja.com/ajashare/AJA_dataratecalculator_win_11-7.zip

You can quickly get an idea about bitrate for uncompressed HD.

Andrew

In your opinion, what does "perceptually lossless to the master" mean? This question isn't necessarily related to your above post.

Wow, full swing from objective measures to subjective :).

Perceptual lossless means that if you take a group of people not schooled in the science of video compression, and show them the original and the compressed video, they can't reliably tell one apart from the other. And in double blind tests, would likely give the same score to the original (if presented as a potential version of the compressed material) that they would to the actual compressed content.

At the risk of stating the obvious, no movie encode goes through a double blind test before being released :). So a judgement call is made by the compressionist that this point, or close approxmiation of the same is achieved and content is then published.

By the way, the real source is 4:2:2 and hopefully in 10-bits. This increases the data rate to almost 800 mbit/sec. We get it down to 600 mbit/sec using decimation filtering (i.e. throwing away) some of the data before encoding.

I really appreciate the quick lesson in video compression. I have learned a lot in the last hour!


So......movies are shot in 10 bits, 4:2:2? What is the last 2?

If so, that is 10+5+5 = 20 bits?

So......
1920X1080 bits = 2,073,600 bits
2,073,600bits*24frames/sec*20bits = 995 Mbs or almost a GB/sec?

Does this 10/4:2:2 sound right?

Thanks again to everyone.

Jim

The resolution for these formats is also fixed at 8 bits. This means that the Y component is 8 bits and we have 4 bits for color for a total of 8+4 = 12 bits/pixel. Quite a saving over 24-bits used in your computer desktop to read this text!


Shouldn't it be 8 + 4 + 4 = 16 bits per pixel?

In any case, iirc, the rough numbers for uncompressed 1080i is 1.25 Gbps for 4:2:2 reduced colour space and 1.5 Gbps for 4:4:4.

I guess 4:2:0 would be roughly 1Gbps.

24 fps would of course be 80% of that, or 800 Mbps.

This is a pretty good doc as well. See page 8, it breaks down the bandwidth/storage required for video by width, height, frame rate, and type.

http://www.adobe.com/designcenter/premiere/articles/prem7_hdprimer_w_ue/prem7_hdprimer_w_ue.pdf

1920x1080 24f, 10-bit 4:2:2 = 994 Mbps / 437 GiB per hour :eek:

So......movies are shot in 10 bits, 4:2:2? What is the last 2?


Wiki is your friend :D

http://en.wikipedia.org/wiki/Chroma_subsampling

So let's see...

For 4:2:2 we effectively have Y at 1920x1080 and U/V at 960x1080.

So 2 MPixels worth of Y and 2 MPixels worth of U and V combined.

For 4:2:0 we effectively have Y at 1920x1080 and U/V at 960x540.

4:2:0 would be 2 Mpixels of Y and 1 MPixel for U and V combined.

So it looks like Amir was right and I was wrong for 4:2:0. I guess I am getting old and musty :D

using D-Cinema v1.1 as a measuring stick....

displaying a 4k source in an uncompressed TIFF format (not using jpeg2000) needs roughly about 10 Gb/s as it sits today. jpeg2000 reduces that to somewhere above 250 Mb/s. This is full depth color (12 bits).

As an HD editor, when I capture video onto a Final Cut Pro System, it averages about 7 gigs/minute at 1080i @ 59.94fps. That means for 30 frame per second video at 1080i, it would require 420 gigs per hour. For 1080p at 24 fps, that would translate to about 336 gigs per hour.

In otherwords,

It will be a LOOOOOOOOOOOOOOOOONNNNG time before we ever see lossless video in our homes.

using D-Cinema v1.1 as a measuring stick....

displaying a 4k source in an uncompressed TIFF format (not using jpeg2000) needs roughly about 10 Gb/s as it sits today. jpeg2000 reduces that to somewhere above 250 Mb/s. This is full depth color (12 bits).

Figgie,

4K source? I have heard this before, but don't know what it means. Any help would be appreciated.

Jim

In otherwords,

It will be a LOOOOOOOOOOOOOOOOONNNNG time before we ever see lossless video in our homes.

And more importantly, is it even necessary? I'm sure it will happen one day. The average person can see the difference between 480p and 1080p and we're still having trouble convincing people to upgrade. Is uncompressed video quality a bigger jump in actual viewing situation?

Figgie,

4K source? I have heard this before, but don't know what it means. Any help would be appreciated.

Jim4K is an industry standard digital resolution: ~4096 x 2160. 4K is often used for scanning/restoring high-profile film content. In some scenarios, more than 4K may be used.

2K is ~2048 x 1080. 2K DLP is what you generally see when visiting your local digital cinema theater.

How much "quality" are we missing by using compression on the video?

Theoretically, it depends. There is both compression and chroma subsampling going on. In addition, farther back in the chain you have more bit-depth, and also more resolution. The quality that HD-DVD/BRD provide is pretty darn impressive.

The take-home point that SOOO many people miss completely out of largely mistaken technical understanding and a legacy of misunderstanding compression artifacts as synonomous with compression is bad, is that compression is positive. Compression is EXTREMELY good. Many people completely fail to understand this fact. Compression allows dramatically increased quality when implemented properly. This is simply because there are still today, major bandwidth and storage limitations for data, which video uses a LOT of. Given a limited pipeline and limited data storage, we have to make certain compromises. There simply is no way to get uncompressed HD video to the home in any economically feasible way. You could do it if you wanted to use scores of high-def discs and change discs many times through the movie. But who wants to go back to the days of laserdisc players that would flip the 2-sided disc for you, and you'd STILL need to put in another disc to finish the movie? Who wants to have a giant disc carousel in their house just to watch ONE movie. Who can afford the scores of terabytes of storage and all the electricity and rack space it takes in servers to have a reasonable library of HD media in their house?

How much uncompressed video can a BD/HDDVD hold?

NONE. Again, lets consider how much the bitrate of uncompressed 4:4:4 content, just in 8-bit would be for standard definition, DVD-quality 480p. It would be nearly 200megabits per second. You can't achieve those kinds of bitrates form the disc AT ALL. So it could HOLD that data, you just couldn't PLAY it from the disc. So just pretend it's only a storage media and you will transfer that to a hard drive where you could play from the drive at that kind of bitrate. At that kind of bitrate, on a 50gb disc, you could get basically 2050 seconds of standard definition video. That's just 34 minutes of standard definition content.

Oh that's right, you actually want to watch a whole movie at "glorious" 480p? You're looking at FOUR 50gb bluray discs JUST for standard definition.

Oh that's right, you want to listen to your movie too? Extras?

So sure, you COULD have uncompressed video. But such a desire is not just misinformed, it's plain stupid. By using heavy compression intelligently, you can achieve extremely high video quality and save VAST amounts of data and bandwidth. This gives you more space for things like higher resolution.

So that's your choice: glorious high-definition 1080p video delivery at full movie length, or literally MINUTES of standard definition video that would be MAYBE perceptually slightly better than a good DVD.

So to the people who mistakenly think that compression=bad, they can feel free to advocate a high-def 4:4:4 10-bit format that delivers a whopping 1,500mbits per second. And they can drool over their marginally if at all better picture for the wonderful 4.6 minutes of video delivered on a 50gbdisc.

Anybody excited to use twenty five 50gb bluray discs just to watch a SINGLE movie? Anybody?

Still railing against the "evils" of compression are we...?

What would the video bit rate be with the best loss-less compression? I mean typical, since there wil be lots of variation.

Maybe like 60%? 50%? It's still astronomical and unworkable. And with all of our visual weaknesses it's simply stupid not to take advantage of the data-saving possibilities to gain BETTER quality content delivery. You're still talking more than the Alien quadriology pack of discs just to watch ONE movie. And for a quality improvement that would be at MOST barely perceptible to even the most fanatical of fanatics(of which I am one). And if you had a delivery system that could ACTUALLY deliver content like this(which is completely unrealistic, because all the hardware would need to be able to handle the fully uncompressed bitrates for certain passages), it would be completely stupid to do so when you could turn around and use that same space and bandwidth to deliver, oh, I don't know, 4k content...

It's almost as ridiculous as people who worry about the newer compression methods because they are more aggressive. Absolutely hilarious, that one, when people think that h.264 or VC1 is worse because it's more heavily compressed. How absurd.

The compression is really the last thing we should be worrying about at this point, if we really want to increase quality there's a lot that needs to be done first before going "uncompressed" will be beneficial:

More care taken in the digital transfer of the original film stock.
More care taken in the cleanup of that original transfer.
Avoidance of "enhancements" (like EE, DNR, etc).
Increase the bit depth for home video to 10-bit (no more banding).
Increase the chroma sampling to 4:2:2 or 4:4:4.
Expand the color gamut to DCinema gamut (which I believe is much closer to the original film than SMPTE-C primaries that are being used today).

Any or all of those would make a much bigger difference than going "lossless" for video.

And more importantly, is it even necessary? I'm sure it will happen one day. The average person can see the difference between 480p and 1080p and we're still having trouble convincing people to upgrade. Is uncompressed video quality a bigger jump in actual viewing situation?

It's actually not that big a jump at all. I've viewed movies from their HDCam masters, and then mastered to Blu-ray and HD DVD, and the disc versions are completely transparent to the master. I work in HD exclusively every day and can easily spot weak material. I believe the codecs being used are extremely efficient and effective in bringing the best picture quality to the home. Of course there are exceptions (think of the earliest Blu-ray releases), but for the most part the weaker looking titles are a result of a weaker looking master.

Maybe like 60%? 50%? It's still astronomical and unworkable. And with all of our visual weaknesses it's simply stupid not to take advantage of the data-saving possibilities to gain BETTER quality content delivery. You're still talking more than the Alien quadriology pack of discs just to watch ONE movie. And for a quality improvement that would be at MOST barely perceptible to even the most fanatical of fanatics(of which I am one). And if you had a delivery system that could ACTUALLY deliver content like this(which is completely unrealistic, because all the hardware would need to be able to handle the fully uncompressed bitrates for certain passages), it would be completely stupid to do so when you could turn around and use that same space and bandwidth to deliver, oh, I don't know, 4k content...

It's almost as ridiculous as people who worry about the newer compression methods because they are more aggressive. Absolutely hilarious, that one, when people think that h.264 or VC1 is worse because it's more heavily compressed. How absurd.


Funny thing is that this same principal applies to audio........Use of lossless there is a good thing but some people insisit that it is compressed.. wah wah boohoo!

But I digress. Video is compressed heavily. It has to be. But look how good it looks! I come from the IT world, where if the storage can be used more EFFICENTLY than we will use it more EFFICENTLY (compression etc).

It's actually not that big a jump at all. I've viewed movies from their HDCam masters, and then mastered to Blu-ray and HD DVD, and the disc versions are completely transparent to the master. I work in HD exclusively every day and can easily spot weak material. I believe the codecs being used are extremely efficient and effective in bringing the best picture quality to the home. Of course there are exceptions (think of the earliest Blu-ray releases), but for the most part the weaker looking titles are a result of a weaker looking master.
If the codecs be used were extremely efficient and effective, why is it that "tweaking" of the encodes are necessary? Why do they say that it's how good the person doing the encoding is? If the codecs were amazing there'd be no need for people to do tweaking of the encodes.

Some compression may be good for the end user delivery format (ie. the consumer) where the players are not capable of higher bitrates/larger storage capacity.

However, when the player is capable of high bitrates and can store high amounts of data, we should be using as much of that bitrate as possible especially when the content needs it. And with the current codecs blurring the video etc. when there's not enough bitrate (which may make the person doing the encode not notice the artefacts as much) or macroblocking etc./crushed blacks/banding etc. it shows that a higher bitrate is better when the media is capable of it. This applies to both optical media formats - if the bandwidth is there - it should be used for the highest PQ/AQ possible.

And with the current codecs blurring the video etc. when there's not enough bitrate (which may make the person doing the encode not notice the artefacts as much) or macroblocking etc./crushed blacks/banding etc. it shows that a higher bitrate is better when the media is capable of it. This applies to both optical media formats - if the bandwidth is there - it should be used for the highest PQ/AQ possible.

How would they not notice if they have to compare it against the MASTER they are using? That is an absolutely absurd postulation!

Technology has ALWAYS been about doing more with LESS ( One admin / 60 servers up from 1 Person / 40 servers). Video, audio etc is using that very technology to acheive that. Otherwise as Chris Mentioned, 25 BD disc's to watch one movie!

How would they not notice if they have to compare it against the MASTER they are using? That is an absolutely absurd postulation!

Technology has ALWAYS been about doing more with LESS ( One admin / 60 servers up from 1 Person / 40 servers). Video, audio etc is using that very technology to acheive that. Otherwise as Chris Mentioned, 25 BD disc's to watch one movie!

Perhaps they're using the wrong type of monitor to watch the compressed version or perhaps their not doing a side by side comparison with a monitor (say LCD?) with enough resolution and size etc.? Blurred compression artefacts (which blur the entire picture a bit) would be harder to see than non-blurred ones. Obviously compression by itself isn't amazing or there'd be no need for tweaking etc. as I said above, and it would'nt matter what the content contained or how complex it was.

Perhaps they're using the wrong type of monitor to watch the compressed version or perhaps their not doing a site by side comparison with a monitor (say LCD?) with enough resolution and size?

It does not work that way. Being the IT of those compressionist. I had to supply said technology to the compressionist to do thier job CORRECTLY.
I can tell you that those monitors are calibrated at the start of the day for 6500K color. They have a choice of 5 -6 monitors (CRT, Rear, proj, LCD etc) Even to this day they usually prefer CRT monitors as thier primary display as they can get to well beyond the "1080p" resolution that is used for digital based encodes.

It does not work that way. Being the IT of those compressionist. I had to supply said technology to the compressionist to do thier job CORRECTLY.
I can tell you that those monitors are calibrated at the start of the day for 6500K color. They have a choice of 5 -6 monitors (CRT, Rear, proj, LCD etc) Even to this day they usually prefer CRT monitors as thier primary display as they can get to well beyond the "1080p" resolution that is used for digital based encodes.

I thought one of the insiders (Ben?) said they like to use their LCDs at a "too bright" setting to make it easier to see artefacts in the darker areas?

I thought one of the insiders (Ben?) said they like to use their LCDs at a "too bright" setting to make it easier to see artefacts in the darker areas?

You see the 5-6 monitors I mentioned? The too bright setting (high brightness, mid contrast) is to bring out the really really dark scenes enough to see the macro etc. That is why they have NUMEROUS monitors in the suite with them.

If the codecs be used were extremely efficient and effective, why is it that "tweaking" of the encodes are necessary? Why do they say that it's how good the person doing the encoding is? If the codecs were amazing there'd be no need for people to do tweaking of the encodes.

Some compression may be good for the end user delivery format (ie. the consumer) where the players are not capable of higher bitrates/larger storage capacity.

However, when the player is capable of high bitrates and can store high amounts of data, we should be using as much of that bitrate as possible especially when the content needs it. And with the current codecs blurring the video etc. when there's not enough bitrate (which may make the person doing the encode not notice the artefacts as much) or macroblocking etc./crushed blacks/banding etc. it shows that a higher bitrate is better when the media is capable of it. This applies to both optical media formats - if the bandwidth is there - it should be used for the highest PQ/AQ possible.

Throwing away 95-97% of the data and looking that good and you are questioning the need for hand tweaking?

As to your bitrate/bandwidth rant, it's all for naught if you have no access to the source to be able to tell if the image is supposed to be sharper or if that was just the way it was filmed (no such thing as perfect focus 100% of the time for 100% of the movies).

The compression is really the last thing we should be worrying about at this point, if we really want to increase quality there's a lot that needs to be done first before going "uncompressed" will be beneficial:

More care taken in the digital transfer of the original film stock.
More care taken in the cleanup of that original transfer.
Avoidance of "enhancements" (like EE, DNR, etc).
Increase the bit depth for home video to 10-bit (no more banding).
Increase the chroma sampling to 4:2:2 or 4:4:4.
Expand the color gamut to DCinema gamut (which I believe is much closer to the original film than SMPTE-C primaries that are being used today).

Any or all of those would make a much bigger difference than going "lossless" for video.

Preach on brother man! Can I get an amen? Right now, it's the "garbage in" part of the equation that minimizes the quality we are getting out of HDM.

If the codecs be used were extremely efficient and effective, why is it that "tweaking" of the encodes are necessary? Why do they say that it's how good the person doing the encoding is? If the codecs were amazing there'd be no need for people to do tweaking of the encodes.


Because the advanced codecs, in terms of implementation is still pretty immature, meaning they can have better versions down the road. The need for hand tweaking is a reflection of inability to have good heuristics done to automate much of these ROI work.

There are many frame and even inter-frame analysis tools along with multi-pass encodes that can be thrown in to try to squeeze bitrates and minimize perceived loss of quality to hit a bitrate target, but such IP secrets are still deeply kept within the encoder community.

In the broadcast space, there is no room for tweaking since often these streams have to go out live (eg : sporting event) so your heuristics in rate control have to be really good to beat the competition. Each family of encoders will have their tricks that they won't reveal to the world, but because the tricks do work quite well. :D Noone involved in these things will discuss them in detail, except in general terms.

There's a lot of patents done in these area, but not all the ideas in the patents work well, and people won't say which ones work really well. ;)

But all these trickery are done when you have to hit a bitrate target, they 'work' because they reduce the size of areas of the screen where supposedly your eye won't pay attention to, while keeping regions that people are most sensitive to in higher detail.

In reality, those areas that are ignored often still have visible artifacts, so ideally, over relying on these tweaking tools is a matter of economics, and not of getting the best picture you really can.

Digital camera that capture jpeg can in theory play these tricks to reduce the size of the jpegs, but anyone who really wants the best pictures would ignore the normal setting and go to Fine or RAW, which gives them either high quality or lossless.

As long as the movie can take higher bitrates and preserve more detail, and still fit within one disk, there is no advantage for us as collectors to ask for lower bitrate encodes. Not unless someone wants to stream these disks to a HDD or to some IPTV thing over a bandwidth constrained network. Keeping high bitrates is HDM's strongest defense against Managed Copy or IPTV as you can simply have a clear difference between these other versions of the movie.

Thes top 3 high bit rate encoded titles that I own are

1) 28 Weeks Layer (Blu-ray) - 37 Mbps AVC - FOX
2) Rescue Dawn (Blu-ray) - 33 Mbps AVC - FOX
3) Pan's Labyrinth (U.K Blu-ray) - Avg 28-30 Mbps VC-1 - OPTIMUM

But there are other low bit rate titles like Seabiscuit, King Kong, Casino, POTC etc that look better than few of the above mentioned titles.