AVS Forum banner

1 - 20 of 2239 Posts

·
Registered
Joined
·
3,325 Posts
Discussion Starter #1
I've posted the latest version of HDTVtoMPEG2:

http://www-personal.umich.edu/~balazer/HDTVtoMPEG2/


I've re-united the readme and attempted to update the change log. This version includes the fixes in 1.10b and adds additional fixes by Cris Moore.


I can continue to host HDTVtoMPEG2 here for the time being, so if there isn't a more appropriate person or place to host it, please send your bug fixes to me.



Some basic info about HDTVtoMPEG2:


HDTVtoMPEG2 is primarily an MPEG-2 transport stream editor. Editing is performed as a simple transport packet level cut & splice, which means the output has a transport stream discontinuity at the splice points. Different decoders handle discontinuities with different amounts of grace. You may experience audio and/or video break-ups at the discontinuities. If you need editing with no discontinuities in the output, check out VideoReDo.


HDTVtoMPEG2 can break the output into multiple file segments. It can strip unwanted programs and PIDs from the input, and it regenerates tables. It can optionally preserve the transport stream bit rate (to keep a constant bit rate stream as such).


HDTVtoMPEG2 can scan for commercials by detecting black pillarbox bars.


I think the transport stream to program stream conversion was broken with newer versions of HDTVtoMPEG2. There are probably better programs to use for that. (X-Muxer or VideoReDo?)
 

·
Registered
Joined
·
1,151 Posts
HDTVtoMPEG2 (often referred to as "H2" or "H2M")
Current version: v1.11.94

The current version is available here:
http://www.eecs.umich.edu/~balazer/HDTVtoMPEG2/

What's it for?

Despite its name, HDTVtoMPEG2 main purpose is to edit commercials out of HDTV transport streams and write out another transport stream. It still has the capbability to convert to program streams (mpeg2) but this area has received little to no work since v1.10b. If you need this capability you should use another tool like VideoRedo.

HDTVtoMPEG2 Feature\\Bug fix history

Feature\\Bug fix history (v1.11.nn)

.94 - Fixed improper Output file name and crash when deleting from Input box introduced with tooltip change.

.93 - Input list box now only shows the file name. Tooltip shows the path.

.92 - Recognize .rec and .mpg as valid transport stream extensions

.92 - Fixed a bug that could cause the dreaded "Could not find a Channel" bug on valid TS files.

.91 - Clips button state is now saved in the ini file

.91 - Added grey color bar support during commercial scan.

.90 - Relaxed the color requirement for pure black on 4x3 color bars.

.90 - Fixed "The parameter is incorrect" dialog message when you scroll the mouse around in an empty Process list box.

.90 - Fixed code so it could handle TS streams with section encapsulated data, i.e. pointer_field != 0.

.89 - Fixed crash caused when no channel is found

.89 - Fixed crash during loading of some TS file when calculating avg TS rate

.89 - Updated to Visual Studio 2005

.88 - Fixed Max Size dialog popup bug when entering a size larger than 99999

.88 - Fixed invalid file name problem related to very large filenames when using Add

.88 - Fixed crash when using Preserve Bit Rate on some transport streams

.88 - Fixed bug that incorrectly extracted the PCR during PMT validation

.87 - Removed the transport stream extension case sensitivity.

.87 - Option to change the vertical_size_value in the sequence_header from 1088 to 1080

.87 - Added a Configuration menu

.87 - New Cut region marker. Forces a new file to be created at a specified point

.87 - Ability to re-order the input file list by dragging items with mouse

.86 - Region markers are now saved when closing H2M.

.86 - Fixed bug where shortcut keys would not work immediately after Deleting a file from the process list box

.86 - MaxSize, VideoPID, and AudioPID boxes now handled properly when shortcut keys are being used

.85 - Added Alt+D\\d to delete region marker

.85 - Default Channel is now auto checked

.85 - Added an RUSure dialog when the Clear All region markers button is clicked

.85 - Fixed bug where a .TRI file was not created if a .TRI file was selected as the input file

.85 - Region info is no longer saved when Clear All is selected, only when Process\\Include\\Exclude\\Delete\\Move

.84 - Fixed missing TS info (Resolution, FPS, etc.) for some streams

.83 - Fixed possible crash when editing low bitrate SD channels

.83 - Added message box indicating SD channels are not supported with Commercial Scan

.82 - Attempt to include black frames before and after program. Disable with IncludeBlackFrames=0

.81 - UI indicates Avg. TS Rate now instead of bogus Video rate

.81 - Weighted average TS Rate is now calculated instead of using a fixed 19.2Mbps rate

.81 - Option to select 10 second Commercial Scan rate instead of the default 60 seconds

.80 - Re-work of Commercial Scan code to fix troublesome ABC\\FOX stations

.79 - Small tweak to code that determines frame type

.78 - Fix bug that could cause the second file to be ignored during scan if Exclude section had been specified

.77 - Fix bug that would sometimes not let you type letters in the Output FileName and Ext boxes

.76 - Fix to scan code to handle pre-defined Include\\Exclude sections correctly during scanning

.75 - F5 refreshes the input file data if you are editing a TS file that is currently being recorded.

.74 - Files are now opened with FILE_SHARE_READ and FILE_SHARE_WRITE

.73 - Minor changes

.72 - Ability to disable .TRI file creation using the CreateTRIFiles=0 flag in the ini file

.72 - .TRI files not created until an Include, Exclude, button is pressed

.71 - Fixed major bug when Scan filters were being used

.70 - New Commercial Scan code (HDTV support only)

.70 - Jockeyed around the manual editing buttons

.56 - Changed default Channel to now be the first stream with a horizontal size >= 1280.

.55 - Default output file type is now transport stream instead of mpeg2. Last used setting is still saved in the .ini file.

.55 - File type list order is now Transport Stream, HiDTV, Mpeg2.

.55 - NOTE: You will need to manually update the Ext edit box

.54 - Default Channel is now the one with the lowest video PID number

.53 - Fixed possible crash caused by code being too slack when parsing for video start_codes.

.52 - Fixed bug where channel would not be displayed if the audio was of stream_type 0x06(PES private data).

.51 - Commented code out that ignored PMT's with CA_descriptors(build 49). This didn't always indicate that the channel was scrambled.

.50 - Fixed possible crash do to too aggressive of checking for corrupt PMT

.49 - Ignore PSIP packets for Channel info if any of the PMT's contain CA_descriptors

.48 - Fixed bug where audio was not being mapped properly if the PMT was incomplete.

.47 - Fixed possible crash when editing QAM files.

.46 - Fixed. If the KeepPSIP and KeepNULL keys were missing from the ini file(you were using an older Beta), H2M would keep the PSIP and NULL packets until you had de-selected them.

.45 - Fixed bug if "PreserveBitRate" is missing from the ini file H2M will ALWAYS include NULL packets

.45 - Fixed. When using "PreserveBitRate" a small number of packets would get stripped instead of being converted to NULL packets

.44 - Fixed PID swap bug when 2 PMT's referenced the same video PID

.43 - Fixed bug introduced in build 39. Files would not load if duplicate file was encountered

.42 - Fixed PID drop down list box for W2K displaying only 1 entry

.41 - "PreserveBitRate" option flag available in ini file

.40 - Increased scan buffer size a bit

.39 - 0 byte transport streams are now discarded with warning.

.38 - commented out some new code that could cause a crash if the MGT had a large number of tables

.38 - PID swap bug fixed

.37 - Fix to bug introduced by code to detect corrupted PAT and PMT

.36 - New Clip feature

.35 - .TRI files not created now until the "Include" or "Exclude" button is pressed

.34 - Contrast and Brightness settings saved

.33 - detect and strip corrupted PAT and PMT

.32 - write multiple PAT and PMT's at the beginning of the file to make sure H2M will have Channel info to display

.31 - PID 0x1fef recognized as a NULL packet (old HBO recordings?)

.31 - Fixed small memory leak

.30 - ??


1.11 Beta

NEW

- PID stripping support

- MPEG-2 audio now supported when converting TS to MPEG-2

- Option to create sequential "clip" files

- Ability to preserve the stream bit rate using the "PreserveBitRate" flag in the ini file.


FIXED

- Fixed an old memory leak in scan_channel_info() function

- Fixed problem with some transport streams not displaying a preview when "Show Progress Image" is checked

- Fixed missing preview display on Progress dialog for MPEG-2 conversion on some transport streams

- Fixed missing video\\audio on MPEG-2 conversion for some transport streams

- Fixed input path not being applied after selecting files from Explorer(Note: if a .tri file is selected, the path is set to the path to the transport streams NOT to where the .tri file was selected)


1.10.6

NEW:

- Drag and Drop support for TS and .TRI files onto H2M desktop icon from Explorer

- Double click of .TRI file in Explorer starts H2M and auto loads files


FIXED:

- Tweaks to scan_channel_info() to improve pid detection


1.10.5(Merger of Alan Haverson's code and Cris Moore's code)

NEW:

- Hot key short cuts

- Basic Contrast\\Brightness controls

- Specify fixed location for .TRI files through .INI file


FIXED:

- Possible freeze during file transitions


1.10.4

NEW:

- Auto save of region edits

- Mouse wheel steps by i-frame


FIXED:

- >>| Seek to End would not move the bar to the end of the slider

- Crash in Progress dialog when showing progress image

- |
 

·
Registered
Joined
·
140 Posts
Thanks for the great work!


Can I ask for one small thing.

Can you make the mouse wheel act like the left and right arrows (forward and reverse 1 time slice)


That would make editing even easier.

Thanks!
 

·
Registered
Joined
·
239 Posts
One thing I've noticed is that HDTVtoMpeg2 is having troubles reading the transport streams capped from the new cable boxes with firewire.


Any ideas as to why?


Some representative samples are at :

My Website


All of the above sometimes load with incorrect resolution readings.


All of the above always have incorrect time length readings.


And all of the above WILL crash hdtvtompeg2 if you try to scroll within the file. Editting also is choppy (it's like the timeslices are way too far apart).


thanks !


- litz
 

·
Registered
Joined
·
13,423 Posts
Is there a file size limit in this version?


I've tried to edit two HDTV broadcast movies and the application always crashes around 9.5 GB when converting.
 

·
Registered
Joined
·
1,151 Posts
litz,

- Problems with TS files

I'm downloading your sample files and will look at then when I get a chance. Please be patient though, debugging display preview problems takes a lot of time and effort.


- Timings are wrong

I believe H2 uses the bit rate specified in the TS file for the Video Rate and for calculating the times. Tools that remove items like NULL packets and PID's change the TS file from a CBR to a VBR file and the times are no longer accurate. I have the same problem when I record with TSReader when I have it remove NULL packets and PID's. I believe STB are removing items like NULL and possibly PID's(sub channels) too in order to save HD space. To fix this problem, I believe H2 would probably have to be changed to calculate the average bit rate of the TS file itself in order to be more accurate on the times.


Cliff,

This is probably a side effect of a "fix". Is the file your using 1 big huge file?
 

·
Registered
Joined
·
13,423 Posts
"Cliff,

This is probably a side effect of a "fix". Is the file your using 1 big huge file?"



Both


One was 17+ GB recording from ABC (720p) and the other was 1080i (NBC) recording with 32 files of 650 MB. Both crashed around 9.5 GB processed file size.
 

·
Registered
Joined
·
13,423 Posts

Quote:
Originally posted by Cliff Watson
"Cliff,

This is probably a side effect of a "fix". Is the file your using 1 big huge file?"


Both


One was 17+ GB recording from ABC (720p) and the other was 1080i (NBC) recording with 32 files of 650 MB. Both crashed around 9.5 GB processed file size.

Cris,


The NBC 1080i file were actually 38 files (650 MB each) so I split them into 3 groups of 13 and the application didn't crash.
 

·
Registered
Joined
·
239 Posts
Cris - no problem on the wait, these things do take time.


That is representative of the three different types of streams you can capture off a Motorola cable box (via the Mac firewire method) and they don't work very well with H2, so there's something fundamentally different about them versus a traditional MyHD .ts capture.


MyHD has fits playing them too, but it eventually gets around to it.


The goal is to figure out a way to transcribe these to something more "normal" so not only tools but playback works better.


- litz
 

·
Registered
Joined
·
239 Posts
Oh ...


In addition to the other clips, there's one more :


crash.m2t.


This is *guranteed* to crash hdvtompeg2 when you try and scroll within it.


Load load, and click the forward 1 arror a few times, bang!


Same dir as before on my web site


thanks !


- litz
 

·
Registered
Joined
·
5,344 Posts
the problems you're having with .ts files from your Motorola box do NOT occur with my Scientific Atlanta box....

 

·
Registered
Joined
·
1,151 Posts
litz,

"All of the above sometimes load with incorrect resolution readings."


I wasn't able to reproduce this with the files you supplied. What other resolutions are displayed?

"All of the above always have incorrect time length readings."


Covered in an above post.

"And all of the above WILL crash hdtvtompeg2 if you try to scroll within the file. Editing also is choppy (it's like the timeslices are way too far apart)."


Wasn't able to make H2 crash with the files you supplied by scrolling withing the file. Perhaps the sample files are too small. I wasn't able to download crash.m2t. The website is no longer available.


Choppy editing - I counted the i-frames in 3 of the samples you supplied and H2 displays all of the i-frames. Granted some of the preview pics don't look so good(528x480i) I think because of the scaling, but they are displayed. Again, perhaps the sample files are too small and I need a bigger file to see the problem.


Cris
 

·
Registered
Joined
·
239 Posts
I don't know what's going on with the domain.


For some reason, an old IP is coming up in a lot of nameservers.


It should be 24.30.63.124 so if you're not getting that as the IP, your DNS is broken ...


If still no go, please post back and I'll provide a FTP link.


As for the crashing, I cannot scroll at all in ANY .m2t file I have. Two or three clicks, BAM.


They also will not write out to .mpg, but they WILL write out to .ts


Is there any way to *verify* the version of HDTVtoMPEG2 I'm running?


Yes, I've redownloaded it, but that doesn't mean windows doesn't have some cached copy sitting around it may be running instead ...


Is that something that can be easily added to the current version?


thanks


- litz
 

·
Registered
Joined
·
239 Posts
Oh ...


Also - I've verified these crashes on multiple computers each with different OS (2k / xp) and processors (athlon vs p4) ...


Two or three right arrow clicks, bam.


thanks !


- litz
 

·
Registered
Joined
·
239 Posts
okay,


Unless and until this DNS issue gets resolved, directories are NOT going to work on my web site. For some reason, an old IP has been distributed internet-wide for DNS for this connection.


Until the fix propagates, it's dead.


If anyone has any ideas on alternates, please let me know ...


thanks


- litz
 

·
Registered
Joined
·
1,151 Posts
litz,


I'm downloading crash.m2t right now. Its going to take a hour or so.


choppiness -

I just wanted to make sure you understand that H2 only shows the i-frame pictures in a stream. The p-frames and b-frames between each i-frame are not decoded since H2 only allows editing on i-frame boundaries. The number of i-frames available in a stream will determine how smooth the "steps" are between each click. The number of i-frames available in the stream is up to then encoder I would guess.
 

·
Registered
Joined
·
239 Posts
By "choopiness" I mean two things ...


1) large jumps from one Iframe to another. If I decode to mpeg2 video and frame step it in, say, virtualdub-mpeg2, I can see it's missing I-frames as it steps


2) occasionally there is NO frame displayed at all for a step, or it's the same as the previous step.


- litz
 
1 - 20 of 2239 Posts
Top