[AnonymousCoward]

Hi,


I'm trying to learn this stuff inside and out, and am hoping someone can help me with mpeg2 packet structure? Actually this has to do with mpeg2 packet structure for satellite & cable systems, but maybe you Replay gurus can help.


- It seems the mpeg stream, once authorized, goes through a demux and then decoder. And it seems the mpeg packets are (almost?) headerless at this point? Is there no timestamping to keep them in order? Meaning they're buffered FIFO?


- There is some difference between HD mpeg2, SD mpeg2, audio mpeg2, and OTA mpeg2. What distinguishes these? I think it's PID. If so, is there a standard assignment?


- I know DirecTV has 130 byte and 140 byte data packets. What is the difference? Is 140 byte HD? Does 140 byte have headers? Is bandwidth class in one of these headers?


- I know Dish HD is 1920x1080p, and that DirecTV is much less. What is the difference in native mpeg2 packet structure between DirecTV and Dish?


- For the HD units I presume that there is only one mpeg2 decoder, and not a special one for OTA. Correct? (Dish and DTV) And OTA does come compressed in mpeg2, correct? Packet structure?


- What headers are required by most PC mpeg2-playing apps? (cough) Can some apps be configured to capture from various sources?


Please help if you can.

 

[richardtallent]

I would also be interested in this--either the format of a captured RTV MPEG-2 program stream or a demux-ed video stream (say, from Womble) of the same.


I've been contemplating writing a "MPEGTRAN" program to crop MPEG2 video streams for a while, but have been too lazy to look up the format. My main reason: the green stripe and VBI info in the Replay captures and (most of all) cropping my Enterprise collection to 16:9 (thus cutting file size).


Also, has anyone written a program to convert Replay 2020 video files (edited with Womble) to work with the 4000 series? This requires creating the metadata files (guide snips and index file) as well, but I haven't seen anything to do this yet.

 

[jtl]

Getting a 2020 video stream to work with the 4000 doesn't look any easier than converting any other mpeg2 stream.


There are several layers involved; the MPEG-2 video stream is defined by ISO/IEC 13818-2 (AKA ITU-T H.262). The two different multiplexing formats (program stream and transport stream) are defined by ISO/IEC 13818-1 (ITU-T H.222.0). There's *lots* of variability allowed by the specs, but the Replay decoder is (seems to be) much much pickier.


DirecTV presumably uses transport streams -- that's the kind of thing it's intended for. The Replay uses program streams.


There are several different kinds of timestamps involved, but I can't remember if any of them are in the PES (packetized elementary stream), or if they're all at the transport level.


ITU will allow you to download three documents for free, if you can wade their their combersome confusing convoluted registration/download system. The ISO versions of the docs are available more easily, but more expensively.


While I'm listing document numbers - MPEG-2 Audio is ISO 13818-3; I haven't found the equivalent ITU doc # (and would like it, if someone else finds it).

 

[AnonymousCoward]

Some new info. It seems PID is used to delineate streams, but PID for video and audio VARIES! Only PAT & CAT PIDs are stationary, at 0 & 1 resp.

"Upon first receiving a transport stream, the demultiplexer must look for PIDs 0 and 1 in the packet headers. All PID 0 packets contain the Program Association Table (PAT). All PID 1 packets contain Conditional Access Table (CAT) data.


"By reading the PAT, the demux can find the PIDs of the Network Information Table (NIT) and of each Program Map Table (PMT). By finding the PMTs, the demux can find the PIDs of each elementary stream.


"Consequently, if the decoding of a particular program is required, reference to the PAT and then the PMT is all that is needed to find the PIDs of all of the elementary streams in the program. If the program is encrypted, then access to the CAT will also be necessary. As demultiplexing is impossible without a PAT, the lockup speed is a function of how often the PAT packets are sent. MPEG specifies a maximum of 0.5 seconds between the PAT packets and the PMT packets that are referred to in those PAT packets. In DVB and ATSC, the NIT may reside in packets that have a specific PID. "


But I am also reading that DVB transport packets are timestamped! Which conflicts with other credible info I have.


I find tht DirecTV came out before mpeg2 was standardized, so they made early boxen do mpeg1 AND mpeg2, in anticipation of mpeg2 coming online. Also DirecTV audio is not DVB standard, but proprietary, whereas Dish is DVB/mpeg2-compliant all the way.

 

[AnonymousCoward]

Up?

 

[klaatu]

Why are we talking about transport streams?


I don't know about earlier models, but the RTV4K does not use transport streams, it uses program streams. Transport streams are based on a fixed packet size (188 bytes) and are used for broadcast/network applications. Program streams are based on variable sized packets (packs) and are used in applications where the data doesn't need to travel over a lossy network.


It appears that the 4K does the audio decode in software, as a result, I would imagine that it would also have to do the demux in software. This might explain why the demux is overly strict. I tried a handful of tests and didn't have any luck playing back remuxed streams to my 4K. If the Replay folks had control over both the mux and demux, then I'd imagine they could cheat quite a bit, making it farily hard to generate streams that will playback on the 4K. If they used an of the shelf demux or an integrated demux+audio/video hardware decoder, then it would be much easier to generate streams that it could decode.

 

[richardtallent]

Sort of on this side topic, I've downloaded the docs for the MPEG2 file format standard and perused through them a bit. I've always hated dealing with binary formats that don't have the decency to sit on byte boundaries, but I suppose I'll have to deal with it...


In particular, I want to create an application to crop ReplayTV-created files (both from my 2020 and my new 4040) without recompression. This is mostly to reduce the file size of shows captured in letterbox format (e.g., Enterprise and West Wing). For now, I've decided on a few rules and assumptions:


1. I think I'll go ahead and demux before doing this just to keep the program simple for now. Muxing afterwards shouldn't be a problem since the number of frames remains unchanged and I'm not stuck on compatibility of the stream back to the Replay.


2. My understanding is that the files are 720x480 and are interlaced, resulting in frames made up of two 720x240 fields. From what I gather, the 16x16 macroblocks are defined at the field level for interlaced video, so I can only crop in 32x32 increments, which should still be fine.


3. I *think* all I need to do is work through the file looking at and keeping up with the location of the slices of macroblocks. Since slices seem to not wrap around vertically, doing vertical cropping should only require identifying the vertical location of the slice and filtering it out of the output stream if it is outside the cropping boundary. Horizontal cropping to remove the green bar will require a little more work.


4. It'd be spiffy if someone reduced the file stream to a stream of hierarchal objects (say, with a frame object at the top level) with class methods to manage their contents and properties and the ability to serialize them back to a file stream. The code wouldn't be as efficient as raw functional code, but it would be very well-documented, and the abstraction would allow many other applications to be created, from editors to still-capture export functions to statistical analysis, with future changes easily added to support other file format variations and/or errors. I'm a .NET fan, so I may go that route with the built-in support for file streams, reflection and serialization, and robust collections.


5. The programmers responsible for the previous paragraph have been sacked.

 

[AnonymousCoward]

Surely demux is done in hardware. Software could never keep up.


Yes, I'm talking about transport streams. I Was hoping someone here would know this, but maybe not.

 

[markchev]

There is a lot of very technical information on the Tektronix web site. www.tektronix.com.


i work for tektronix. this is not a solicitation to purchase our products (although you can if you want).


There's a cool MPEG standards poster you can order. http://www.tek.com/Measurement/cgi-b...Set=television


if this link doesn't work, go to tek.com ...then go to Products...Video Test...Technical Papers.


have fun

 

[klaatu]

Quote:

Originally posted by AnonymousCoward Surely demux is done in hardware. Software could never keep up. Yes, I'm talking about transport streams. I Was hoping someone here would know this, but maybe not.

On the 4K I think the demux is done in software (or maybe in the fpga). It looks like the mpeg decoder only has a transport demux, but the encoder outputs program streams... Doing the demux in software wouldn't be that bad, the stream is only 10mbps peak (max VBR rate) and the demux isn't that complex.


My previous comment was just pointing out that the initial post was a bit off topic.


Now to answer your questions:


With transport streams, tuning a program is pretty straight forward once you figure out what PIDs you want to listen to, you just program the demux to filter the packets of each type (for unencrypted content, there are typically 2 PIDs, you need to listen to an audio PID, a video PID (sometimes there is a 3rd systems PID) and route the packets to the correct decoder. For most cable set-tops, you don't need a transport packet fifo because the packets arrive just in time. For a network based packet delivery, you need to buffer incomming packets to compensate for jitter.


As to PID assignment, only a few PIDs are hard assigned. To find a program, first you listen for PID 0, the PAT (program association table). The PAT locates the PMT (program mapping table) PID for each program. Once you know the PMT PID, you program the mux to listen for it. The PMT contains (among other things) a mapping from stream type to PID. Once you find the PMT, you then know what PIDs to listen to for each stream component (eg, audio & video).


It is actually a bit more complex, because on a cable or sat system, you have multiple possible transports, one on each 6mhz channel. Each 6mhz channel can store 27-36mbps (depending on the type of modulation) and would have 8-12 programs. So in that case, you have to listen for another program that contains the channel map that maps programs to channels (frequency ranges). If the content is encrypted, you also have to listen for various CA PIDs...


HD & SD MPEG-2 are pretty much the same, only the resolution is different. The video bitstream contains things like resolution, frame rate, compression type, etc.


mpeg transport packets are 188 bytes. Wrapped around the transport packet, you would also have some sort of forward error correction. Each packet has a 4 byte header and a 184 byte payload. The weird packet size was designed to fit in a small number (3) of ATM frames...


The core MPEG decoder is the same for OTA or satelllite or cable HD. The primary differences would be in the tuner and demodulator hardware.


Very few PC software or hardware decoders can handle transport streams. At my previous company we had to jump through all kinds of hoops to get a PC decoder card that could properly demux and play our transport streams. Most PC decoders will only play program streams.

 

[jconawa]

Here is a good site which even breaks down the packet into precise fields/bits.

http://www.erg.abdn.ac.uk/public_htm...eg2-trans.html

 

[AnonymousCoward]

Thanks peoples.


Emmm, Klaatu you've helped alot, but I wish I could ask you my real questions.

[email protected]

 

[AnonymousCoward]

Does anyone know of a utility to convert a transport stream into PES's?