ReplayTV 5K Tools (Revision 4)  

"suitable for DVD authoring" means your audio should stay in sync. :)

Beautiful! Thanks to Lee & anon....

Does this mean no need to buy womble to fix the mpeg? Is this as fast as Womble?

You're my boy blue! Uh... I mean... way to go Lee! Especially adding the conversion functionality.

I love these tools. Just been keeping quiet waiting for others to talk about it so it's spelled out to me and all I have to do is add it up. I suspect others are doing the same because not much has been said in this thread so far.

In a different thread, some smart-type person showed how to make a little batch file script as an icon. All you have to do is drop a file(a replay mpg in this case) on to the icon and several command line programs are run in sequence making the process a nobrainer. Where did I see that? I might search for it later and post back here or EDIT this post. It seems perfect for these tools (batch files are so easy to reconfigure too).

There's other programs that will wait till a file arrives in a directory and then processes it how ever you have it set up(however you want). I'm eyeing out a way to use all of the above to automate the process from DVArchive transfer scheduling to ripping out the commercials and saving space or sending dvd-ready mpgs to a folder so all is left is to burn them.

I don't like the grammatic structuring of this post, but I'm tired right now; hope it's readable.

I love Lee for being the bringer of all good things. Hope the creator of the program understands how much the work is appreciated. The praise goes unheard from people all over who celebrate the tools quietly in their unmarked and uncounted casas with little grins and smiles. There have been bigger software productions that are more formidable, but these little tools are the ones fueling the revolutionary process. I want to give recognition. Hope I didn't hit too many cheesey notes telling you how I feel. I'd leave it said by the poets, but they never show up.

cow

Re: rtvconvert for DVD authoring

Is it to say that the output mpeg (assuming no demux)
of rtvconvert on a 5K mpeg can be streamed back to the replaytv as well as be used in DVD authoring tool?

WOW!! Thanks!! Wife just asked me about copying off some kid's show that is only airing this weekend to DVD and I was thinking I'd have to crack open the wally. If this all works as billed, are you asking for anything via Paypal?

:cool:

Ok, I have been too lazy to try these tools until last night. My compatibility-exchange units are arriving tomorrow, and I want to be able to keep some of my old recordings. I ran rtvedit on my 4k files and converted them to "5k" files. Seems to have worked and the files do stream back to the 5k's using DVArchive. I'm amazed at how quickly this works. The longest it rans was 5 minutes!

The basic procedure (for those who haven't tried it) is to create a text file with the commands shown in one of the readme files. I think it is all of three lines long. Then execute "rtvedit edit.txt" on the command line. Once the files are converted you will need to notify DVArchive that the MPEG is now compatible with the 5k's. Go into DVArchive and select the shows properties. Under the Advanced tab there is a pull-down for configuring the MPEG RTV version.

I did receive a couple of different errors... first was something about an invalid ndx file. It was just ignored and it contued straight on. The second was something about an improper index. It just skips to a valid index and continues converting. So far I have not found any problems in the new MPEG files, but I haven't watched them all the way through yet. Of course, YMMV.

Thanks a lot Lee and 'anonymous' for producing these tools. I suspect 'anonymous' is one of the RTV programmers, but this is only a guess.

Cheers

Hey, I've never tried these tools and I'm thinking of doing so this weekend -- just a couple of questions first:

Since this tool appears to depend on RTV's detecting commercials, it seems for those that are not detected, you have to go through the video and find the beginning/end times of the commercial segments and put those in the evtdump file manually? If this is the case, my other question is, in order to view the video beforehand (to find the commercials) I have to use VLAN, and since the video is recorded in VBR, the time fluctuates constantly, making it impossible to tell exactly when the commercial segment commences/ends. Any suggestions?

Thanks!


CC

Well, if you do look closely at the evt dump file, you would notice that all the commercials and more (scene changes) are detected and dumped, it is just that the CA logic didnt identify them as commercial breaks. So all you need to do is flip a few A's into D's :)
Here is how I do it. I run the evtdump on the file and open it in editor. I than run rtvedit and take a note at what points the show was edited (it is also obvious from the evt file, but much easier to see in the output of the edit file (you can also just redirect the output of the editing program to a file and open it for viewing later). For example is the 60 minutes show shows that the edited file is 43 minutes, good chance is nothing needs to be done, and all commercials were detected correctly. If the file is smaller or bigger, I'll open the original video file in WMP or VLAN and drag to the first edit point, than second than so on. I do not need to determine points, I just want to find, where they are approximately. Once I see that any of the points are out of wack, I go to the opened evt.txt file and sure enough, if there is a porblem, that means that there is A in front of that edit point - flip it to D. Try it and you get a hand of it, really the whole process is a lot faster than wombling, cause you dont really have to detect frames and such.
I use far.exe from rarsoft.com instead of the run interface of windows, but maybe somebody will write a gui interface to the tools.

Neat, I have to play with the -p and -i to see if I can create templates for specific shows, and just use them instead of editing evt file.

I can't get my mplex (linux/unix) to recognize the
output of rtvconvert. In my simple script for making
DVD, mplex is used to add VOB header to mpeg2
file before dvdauthor (linux/unix). Is adding VOB
header perhaps something that can be incorporated
into a future release of rtvconvert?

Great job.

And, for what it's worth, an 'rtvconvert into regular dvd' not only makes a great input to Ulead DMF2, it also makes the Hauppague MediaMVP a lot happier. MediaMVP can open a directory with dozens of 'rtvconverted' mpg files virtually instantly -- first time, every time.

Um, not trying to nitpick or anything, but is there a separate/different RVTconv utility, as opposed to RTVconv? If there is, I don't see it for any os. Just wondering, thanks. Keep up the great work!

-e

Lee, couple of questions for you...

1) The -i and -p options that can modify the commercial detection, is this just the way the events are handled or is there acutal reprocessing of the mpeg stream?

2) The cut on rtvconvert... it's limited to cutting on GOP boundries? Or are partial GOPs reconstructed?

Looks cool!

I'm about to use event dump to look at the commercial detection differences between 5.0 and 5.1. I might need to bone up on events quickly, any documentation on the events?

Thanks to you guys!

mplex recognizes the elementary streams just fine. Use -d in rtvconvert to write demuxed streams and use -f 8 or -f 9 in mplex for dvd.

mplex -f 8 -o dvd.mpg rtvprog1.m2v rtvprog1.mp2

agent-x care to clue me in on what you've done with the audio stream? I'm very curious to see how your findings compare with what I know. I'm sure you know more, I'm very curious. =) The clock values and PTS times always seem wrong, but I haven't figured it out exactly. It's almost like the MPEG system clock isn't running at exactly 90Mhz?? I've been confused by this for a while and never figured out exactly what is going on.



On another note...
Is it possible to evtdump, rtvedit and import that file back into DVA for serving? I've tried to do both of the following and neither worked when imported back into DVA...

evtdump
rtvedit using output from evtdump
import that new file, no luck

evtdump
rtvedit using output form evtdump
rtvconv with output from rtvedit
import
still no luck

both result in the replay hanging and rebooting.
NOTE: I did this at 1:30AM so I might have screwed something up. I tried to follow the read me as best I could.

Lee,
A guy made a mpg player for hos 4xxx laptop that skips commercials when it gets to them like a replaytv.

He says he can make a 5xxx version if he has some 5xxx documentation on the .evt files.

Here's the link:
http://www.avsforum.com/avs-vb/showt...hreadid=314993

I was hoping you would point him in the right direction

cow

Jeff D,

I've added a PS demux filter in rtvconvert that acts like a P-STD demuxer which is needed to synchronize elementary streams from MPEG-2 program streams. Just demultiplexing PS alone isn't sufficient, yet this is what most DVD authoring programs will do and they will break end-to-end timing unless the elementary streams are already aligned. DVD players are P-STDs, which is why the raw RTV streams play in sync, but may be out of sync after authoring. You need to have one of these filters somewhere in your path and depending on your system setup and tools, it might be coming from DirectShow or embedded in your encoder or editor. This version of rtvconvert has a simple one built-in.

MPEG-2 system clock is 27MHz. PTS clock is 90KHz. The SCR extensions don't seem to be used in RTV mpegs, so the timing model only relies on a 90KHz clock (which is perfectly fine). I haven't seen this clock drift out of tolerance and am curious what makes you say the "PTS times always seem wrong".

I'm not sure what problem you're having with importing and streaming. I haven't seen any issues other than the known ones. TCP stack settings and back-to-back DVA streaming are known to cause problems, but they're not anything specific to edited/converted streams. You might find some useful information in the DVArchive event log. Also, if you're editing a downloaded show, you don't technically need to re-import the stream - you can just replace the fileset that's already there (rename the originals if you want to keep them) and update the guide entry.

Are there similar tools for the 4K series, I have 4504 RTV. Please help
jingcha

OK, stupid newbie question on these tools. I have a show called "Tiny Planets". I've downloaded it to DVA and extracted the files:
Tiny Planets.mpg
Tiny Planets.ndx
Tiny Planets.evt

I am just not getting the documentation on how to simply process the file so that the audio doesn't get out of sync.

I created a text file called TP.txt which contains the text:
Tiny Planets.mpg
E

I then go to DOS and run:
rtvedit.exe tp.txt.

This process gives me the output:
Target: iny Planets.mpg
New Program Time: 000:00.000
Edit Time: 000:00.172

I look in my directory and low and behold, a 2k file called "iny Planets.mpg" now exists. I'm figuring this is what is supposed to happen, the first character being trimmed off. I'm not sure why the file is so small. I then try to run:
rtvconvert "iny planets.mpg" DVDTinyPlanets.mpg

The resulting DVDTinyPlanets.mpg file is about 2k and unusable.

What am I doing wrong and are there other directions somewhere that are clearer for a newbie like me?

(On a side note, I also tried the process against the "Baby Looney Tunes.mpg" file but when I try to run rtvedit against that file it says that it doesn't recognize the command "B".)

I think I follow, but I know very little about the differences between T-STD and P-STD. What you're saying is the timing relationship between the two streams is lost on a plain ol' split the stream demux. So, it would be best to split the streams with rtvconvert -d to go to dvd source files. Is the only time the rtvconvert demuxer used when using the -d option?

Right, sorry, as I understand it the PTS clock is derived from the STC.

My comment on the PTS always being wrong is based off several things some of which can be explained by your previous description of the demux process and how the PC can screw with the streams.

One of the crazy things I'm trying to use rtvconvert for....
I don't know if you saw that some of us are having problem on the 5k recordings where video freezes for 2-4 seconds on playback. After the video starts back up the audio may go out of sync. I've noticed the part of the stream where the replay freezes has a "good" video and audio stream. Demuxing and remuxing the mpeg the remuxed stream plays fine on the replay but all sync is lost. I'm curious rtvconvert can fix what I suspect is garbage in the source recording. The error output I mentioned earlier gives a file offset, but not a time offset, so I'm having a tough time figuring out if the two are one in the same.

Thanks!

I've been wanting to use my replay as a DVD server for a while, and now these tools have allowed me to (mostly) do so.

On Linux, I am using transcode and mpeg2enc to transcode the vob into a 480x480 mpeg2 stream using the settings for an SVCD. Mplex then muxes the elementary streams for me, and rtvconvert allows me to import them successfully into DVarchive and stream back to my 5040.

Now for my questions. First, I have not been able to figure out how to create one large mpg from the vobs that will work with rtvconvert. The only settings that seem to work are the ones for SVCD, and this forces mplex to split the output at approx 730 Mb boundaries. Does anyone have any suggestions on how to create one large mpeg that can be converted with rtvconvert?

Here is the transcode command I am using:
transcode -H 10 -a 0 -x vob
-i path/to/vobs -w 2088 -F 5,'-B 180 -S 736 -I 0 -g 9 -G 18 ' --export_asr 3 -b 128 -s 1.419 -V -f 24,1 -
B 0,30,8 -y mpeg2enc,mp2enc -E 48000 --psu_mode --nav_seek /path/to/navlog/created/by/dvdrip --no_split -o
/outputfile/foo

Here is the mux:
mplex -f 4 -V foo.m2v foo.mpa -o foo-%d.mpg

Does anyone have any suggestions on how to create one large mpeg that can be converted with rtvconvert? When I have tried to prevent the splitting, I have ended up with files that rtvconvert would not handle, failing with an invalid stream id message. The svcd settings above are the only ones I've found thus far that appear to set the corrrect stream ID.

Also, if anyone tries to use the above method there is a quirk to getting the files to work correctly. The first file of the series can be rtvconverted and imported directly. The subsequent files must be rtvconverted then rtvedited to make them work. The edit step appears to trim around 250ms from the beginning of the clip, and if it is left out the clip will cause my 5040 to hang and reboot.

Thanks for all the great tools, and as things stand I am already VERY happy with my new ability to stream my DVD's to my replays. If anyone has suggestions on how to improve the above process, or can shed some light on what causes some of the behavior I'm describing, I would be quite grateful.