AVS Forum banner
Status
Not open for further replies.
1 - 20 of 499 Posts

·
Registered
Joined
·
42 Posts
Discussion Starter · #1 ·
Ha! I've done it! :D


I've succeded in convincing my 4080 that my PC is another 4080!!


I had a small lower-case / upper-case errro in my UPnP:Alive SSDP multicasts, but I've got that fixed now!


As soon as I fixed my UPnP:Alive packet, my 4080 attempted to retrieve "Device_Descr.xml" from my PC. You only get one shot at this. If the PC fails to deliver the requested xml file, the 4080 won't ask again. Had to keep rebooting the 4080 until I got it right.


So I hacked up a quick www server in my ReplayPC application and fed the 4080 a "Device_Descr.xml" that describes my PC as a 4080 called "ReplayPC".


Sure enough, the replay guide now allows me to select a remote 4080 called "ReplayPC". When I select the ReplayPC ReplayGuide, the 4080 sends an HHTP request to my PC:

http://192.168.0.4/http_replay_guide...4080K000000000


Note, the serial number in the above request is the serial number of the 4K requesting guide data. I've zeroed it out here, the 4080 does not seem to care.


Point your www browser at the above url (correct the ip address as required) and the 4K will dump the entire ReplayGuide through the HTTP connection. InternetExplorer does not understand this data, so it chokes on it. But if you watch with a packet sniffer, it's easy to see that the entire contents of the ReplayGuide is in the transmission. (10.5KB of data in my case)


Well time for some sleep, and then back to work.


Anyone else out there willing to help? ;)
 

·
Registered
Joined
·
323 Posts
...192.168.0.x is all private IP address

space and not globally routable.


You're behind a network address translation

device (NAT gateway) and possibly a multiple

address to one using port address translation

(PAT) such as a D-LINK DI713P or LinkSYS BESWFR4

or whatever it is ;-)


If you truly want us to connect to your 4080,

you need to

1. Give us the globally routable address

2. If you're using NAT or PAT, make your router

map incoming connects on port 80 on the

extransl address to port 80 on the .4 address.


Cheers, and happy new year


M
 

·
Registered
Joined
·
42 Posts
Discussion Starter · #3 ·
Why would I wan't you to connect to my 4080???


I'm telling YOU how to get the ReplayGuide from YOUR 4xxx.


The hope being that toghther, we can reverse engineer the guide data, and then move onward to reverse enginerring how one 4K unit requests an mpeg stream from another 4K unit.


The end goal being to write a PC application that can retrieve desired mpeg files from a 4K at will.


Understand now?
 

·
Registered
Joined
·
70 Posts
Dude, sweeeeet! I was parsing the data right from My ReplayTV, which of course, I was worried about being sued over. This way I could negate them completely and have more info than I need.


So much for sleep tonight. You rock!


Cheers,

Chad



____________

Share your shows, only at:
Planet Replay
 

·
Registered
Joined
·
752 Posts
mlinehan,


very awesome! Even with my non-engineering background, that seems like a pretty big breakthrough!


keep up the great work- and let's make our 4000's the absolute best machine ever!
 

·
Registered
Joined
·
42 Posts
Discussion Starter · #6 ·
Well I now have my ReplayPC program retrieving the ReplayGuide from my 4080 at start up and saving it to disk.


It then sends out some UPnP:Alive packets to get the attention of my 4080.


The 4080 then requests Device_Descr.xml from my PC, and I supply it one.


At this point my 4080 thinks my PC is another 4080 on the local network called ReplayPC.


Next I tell my 4080 to access the ReplayPC ReplayGuide


The 4080 then request the guide from ReplayPC, and my program then feeds the saved guide file back to my 4080, and the 4080 dutifilly allows me to select a remote program for streaming playback. (I'll work on decoding this file later)


Now things start to get interesting. The 4080 then issues a request for the following url to my PC:


/httpfs-fstat?name="/"


I've used i.e. to probe what the proper response is. But I'm a bit confused on where to go from here at the moment. I don't think my PC program is properly emulating the response just yet.


Time for a lunch break, then I'll get back on it.


Anyone else out there ready to report any sucesses?
 

·
Registered
Joined
·
42 Posts
Discussion Starter · #7 ·
A summary of what I know so far.


The 4K's find each other on the local network via Universal Plug and Play. This involves SSDP multicast packets to 239.255.255.250, udp port 1900. These multicast packets contian an HTTP url with wich to retrived the UPnP Device description XML file: "n.n.n.n/Device_Descr.xml"


When one 4K first becomes aware of another 4K, it request retrieves the "Device_Descr.xml" to discover the "Friendly Name" of the new 4K on the network.


Note: The "Device_Descr.xml" file that the 4K is serving up is NOT complient with the UPnP standard. UPnP requires that the decripton file enumerate the available services, and provide directions on how to use the SOAP remote procedure call mechinisim to access them. These are MISSING!!!!


At this point, the 4K ReplayGuide will allow access to the new 4K on the network.


As soon as a remote ReplayGuide is accessed by a 4K user, the 4K sends a HTTP request to the remote 4K to retrieve the Remote ReplayGuide. The url is n.n.n.n/http_replay_guide-get_snapshot?guide_file_name=0&serial_no=RTV4080K0000000000 (The serial number is that of the requesting 4K unit, I don't know what name is. 0 seems to work consistantly)


The remote 4K unit then dumps it's entire ReplayGuide back in one lump sum. I do not yet know the format for this data. There appears to be both binary and null terminated text mixed in. For now, I just write this to disk, and then echo it back when my 4K asks my PC for it's guide file.


At this point, the 4K user can browse the remote 4K's replay guide. If the user select "Play" for a remote program, the 4K then sends a request for: n.n.n.n/httpfs-fstat?name="/" to the remote 4K.


The correct response appears to be

"0\

type=d\

"


Next the 4K sends a request for: n.n.n.n/httpfs-fstat?name="/Video" to the remote 4K


The correct response appears to be:

"80820024\

"


One time, and one time only, I saw the 4K request: n.n.n.n/httpfs-fstat?name="/Video/1009609197.mpg"


I do not know the correct response, and I have not been able to repeate the feat. But I would assume that this is the actual name of the mpg file I'm attempting to stream!!! I do not see an obvious connection between the mpg file name and data in the ReplayGuide file yet.


More research will be needed.


I need to take another break.

Is anyone else out there working this?

I could use some support here. I really don't want to be a one man reverse engineering crew!


Happy hunting!!! ;)
 

·
Registered
Joined
·
3,686 Posts
/Video is where the mpeg partition is rooted in the... UPnP??? system.


Looking at the replay's config file /sys2/etc/httpfs-export, you see three different roots being exported: the program guide, the Photo partition (apparently named "storage," exported as "Photo") and the mpeg partition, exported as "Video."


The name should indeed be the filename that the program appears under in the mpeg partition.
 

·
Registered
Joined
·
3,686 Posts
ps - I'm betting that the 100xxxxxxx filename is probably derived from some sort of timestamp, for instance when the program was created. Probably a unix date/timestamp. For instance, with the example you gave, backtranslating from unix timestamp gives December 29, 2001, 1:59:57.
 

·
Registered
Joined
·
3,686 Posts
Quote:
Originally posted by mlinehan
Next the 4K sends a request for: n.n.n.n/httpfs-fstat?name="/Video" to the remote 4K


The correct response appears to be:

"80820024\

"
Interesting... last night, while horsing around with WS_PING (the most handy tool I have for flinging HTTP requests around without using a browser), I was doing a n.n.n.n/httpfs-ls?name="/Video", and the sole response I got back was that "80820024\

" (from memory... if not that, something that looked very much like it).


Wonder what that number means?
 

·
Registered
Joined
·
42 Posts
Discussion Starter · #11 ·
Thanks Toots!


The filename is indeed the unix time stamp.


Said mpg file was recorded on:

12/29/2001 at 2:00AM


The answer was right in front of my face and I didn't see it.

For shame, I have used Unix Timestamps like this myself several times in the past. I should have seen it!


Thanks! ;)
 

·
Registered
Joined
·
42 Posts
Discussion Starter · #12 ·
As I understand it, the RPTV's are running VXWorks, a unix like real time OS.


Could the 80820024 be a representation of the file access rights in decimal?


I've never been good unix access rights, owner, group, other, and all that.
 

·
Banned
Joined
·
316 Posts
Well I'll be damned...


Nice work Mlinehan!


Stepping back, it looks like the HTTP FileService is trying to stat (access) the Mpeg partition, and speculating, in that access request should be some kind of hash for the desired .mpg file, which the RP is not sending in your example. Looks like it accesses the drive, then the part, then should come the file request.


Wow, I can't believe this is not encrypted.


They is gwine lose dey lawsuit...


Better lock em down boys, to prevent a feature-ectomy.


I'm envisioning a "TV Engine"(TM ), with IR/UHF in/out on ttyS1, firewire tapped into the sat receiver, and video card going directly to the projector!


Me, I'm still waitin' for my 4k. (snif)TM
 

·
Registered
Joined
·
42 Posts
Discussion Starter · #14 ·
The url to request the mpg file is:

n.n.n.n/httpfs-readfile?pos=0&name="/Video/xxxxxxxx.mpg"


But you can't do this from a normal www browser. The HTTP request must contain an "Authorization: Basic aaaaaaaaaaaaaaaaaaaaaaaa" option or the 4K won't reply.


I leave it as an exercise to the reader to discover how to coax the "aaaaa" auth code from the 4K


The file is being transfered via HTTP with chunk transfer encoding. I didn't understand chunk transfer encoding, so my first wack resulted in a 1.7GB file on the PC. But the file was not useable because the HTTP protocol headers used for chunk encoding were written into the file.


I've since updated my program to understand chunk encoding, and I have another file transfer underway!!!!


BTW: each xxxxxxx.mpg file has an associated xxxxx.ndx file. The 4K's transfer the nxd file before requesting the mpg file.


Anyone have any ideas what the ndx file may be??? For my 1.7GB mpg, the ndx is 241KB.


L8tr

L8tr
 

·
Registered
Joined
·
42 Posts
Discussion Starter · #15 ·
OK, I'm packing it in for a while since I have to resume my day job! ;) At least until next weekend.


Even though I won't be working on my ReplayPC program for a while, I'm willing to answer others questions. E-mail me....


I have retrieved a 1 hour medium quality file from my 4080, and I think that I've properly stripped out all the HTTP protocol stuff.


I copied the code from extract_rtv that corrects the mpg header and put it in my program, but windows media player still insists that it does not have a codec with which to play the file.


Here is a hex dump of the first 500H bytes of my resulting mpg file. Can someone who knows a bit about mpg files take a look at this and let me know if they see anything incorrect!


00 00 01 BA 44 00 04 00 04 01 00 A5 03 F8 00 00

01 BB 00 0C 80 52 81 06 E1 FF E0 E0 E0 C0 C0 20

00 00 01 BC 00 12 E1 FF 00 00 00 08 02 E0 00 00

03 C0 00 00 AA 58 7B F9 00 00 01 BA 44 00 04 00

04 01 00 A5 03 F8 00 00 01 BB 00 0C 80 52 81 06

E1 FF E0 E0 50 C0 C0 50 00 00 01 C0 02 AA 81 80

07 21 00 01 8C B3 FF FF FF FD B4 04 66 66 33 44

66 45 44 44 44 44 44 6E 46 DB 48 90 00 01 22 00

00 00 AA AA 5A A5 54 5F 5F AA AA 8F 3D 55 5F 7E

28 A4 82 48 23 8E 49 27 96 79 65 8E 58 E5 92 5A

24 9E 39 E3 92 59 25 96 39 63 9E 49 E4 92 49 A6

9A 65 96 5E DA 64 F6 D3 27 8C 9F AC 64 FD 69 AE

9A D1 94 65 CB 1B 96 35 FE 2D D6 55 96 BA AE 9B

26 C8 C1 30 4E 83 A2 25 12 66 D9 B3 51 A9 58 AC

76 ED 29 28 51 AB 17 20 58 B9 00 AC 31 85 61 8D

4E B4 EA AD AB 6E 96 DB 2D B7 22 F0 B1 AC 65 C1

70 B8 AD 09 F2 7C 94 25 01 B0 DC 1D 07 57 AB CA

35 1B E7 CD 6E 8D 0D 7C C6 AB E6 35 4B 64 70 5B

23 8A A5 AA 56 01 80 54 78 A8 F1 7E 6B 73 24 C9

4A 92 A2 2C 8B 58 96 21 A0 B9 49 A4 E0 F8 3C B4

5A 8D C6 C8 91 73 8A 14 9C 4C B9 A2 65 CC E7 09

57 38 4A DE AD EA A1 88 67 64 AE C9 4B F3 D3 3E

4D 0C 1F 07 B6 8D A2 10 84 2A 4D 1B 4D A6 2F 15

F2 29 12 69 34 E1 C4 21 A3 41 A2 8C 31 14 61 8A

74 47 52 A2 39 65 96 59 BF 6F E8 EB D1 F7 5D 5E

1D A2 6A E0 C8 31 C5 6F 21 09 8C 1D 1B AA 55 26

D9 B6 51 A8 E3 51 A4 08 1A D5 2A 68 DB 04 06 D7

E0 69 57 07 4A B8 28 DA 8D AC 03 01 77 B2 EF 62

34 CE CD DB 76 93 24 D0 64 18 BA 24 33 7C DE 5B

2D AD 4B 9C 4A 24 94 4A 56 AD 6B 53 A6 6C 38 A3

61 C5 1A E1 69 97 0B 4C 50 B5 0B 3A 0E 85 0E 4A

1C 87 07 52 45 11 42 BC AF 71 DC 75 91 01 B2 AC

A8 64 31 63 58 E5 D2 E4 62 31 E3 C8 C5 4A 9A 14

2A C0 A1 66 14 87 72 A4 3F 8A 8A A8 A8 72 1C CD

D4 93 A8 D9 B2 CA 34 8D 37 8D E3 88 E2 39 0E 9A

38 8E 17 0B 8A BA AE DA 6D 44 A2 50 9E C1 AF 9F

1A 62 B9 73 15 CB 4B 0A EA 58 57 75 DF 5D CD 33

4E 10 84 21 08 59 66 CC C3 32 C1 B0 6B 1A C4 7A

1E 96 D5 B6 4D 26 46 91 A2 89 45 4C A6 1A 35 8C

7C FA 8F E6 E3 7F 37 10 C7 F8 46 BF C3 38 13 81

0C 43 1B 6C 76 D8 AB 50 90 20 08 05 B1 6C 5A 16

8F 03 C0 6B 9A F4 3A 1B AA EA 86 43 2D C6 E4 58

A7 3A 54 A6 D0 E4 CE 87 26 86 E7 3E 37 39 F3 95

39 5A C2 B0 90 E5 21 C8 C8 47 44 85 21 36 8D 97

1D D0 9C E1 83 2C CB 92 49 03 00 C2 B5 52 95 CA

DC B9 6C 29 D3 A2 BB 90 56 5C 82 F7 2A 27 B9 51

2C B2 CB 2C 0B 03 5B 06 B6 0B 63 BB A5 A9 6A 3C

8F 2E 8B A2 20 88 B7 8D E0 44 22 DF 37 D6 BB 58

84 42 E5 CA 94 A5 49 80 00 00 01 C0 02 AA 81 80

07 21 00 01 9D 93 FF FF FF FD B4 04 33 55 33 44

45 44 55 33 33 22 33 92 49 23 92 44 89 00 00 00

00 00 AA 50 50 1A 4A 00 00 05 50 54 10 51 47 DE

7D E8 A5 7A 29 5E 89 D8 9D 96 48 25 92 09 64 76

57 61 86 58 65 A2 18 E3 9A 48 26 96 09 E6 76 79

9D 96 47 66 91 D9 A5 72 69 5C 9E 67 A8 99 F9 DD

9D D9 DC 9D CA 67 76 99 DD 99 D9 9D 30 83 08 73

01 9C C0 66 9A 69 A4 45 11 AB D5 F6 43 90 E1 56

15 5B 4D A7 43 A1 CB 96 C3 62 3F 7B 8D 22 05 F5

7D 61 F5 AA D6 B5 C1 D0 74 48 8E 77 2A F2 AE A9

43 A2 50 CB DC BD C1 50 53 7F 6F 29 42 62 04 20

43 1A 8D 36 93 56 2C 2A 95 46 71 9C 65 18 2B C2

F0 98 6A 9D F7 7D 5B 56 D2 24 6B 61 DA 1D AF 45

FB D1 7F 49 74 D6 A8 AA 1A C9 48 27 09 C5 D8 5D

88 9C 4D 3A 9D 50 A2 CB 65 9C 67 1A 9E 9E 41 D0

73 4D C2 0B 83 0B 22 27 5E C1 08 D1 8D 19 68 3D

5A 0F 58 A7 8A 6A 52 94 B5 AB 39 DA 76 B9 B3 9B

B6 9B 4E 47 23 E7 D1 38 9B 6A DA 99 A3 E0 4A 12

D4 38 E6 21 88 AE EB B9 D3 9A D8 97 89 72 E9 A4

BA 69 88 38 83 AA AA A1 E6 1C C4 11 0A 5B 65 B9

32 99 86 61 6E DD 4F 17 8D 83 62 0F 83 E9 9A 64

8E B0 AE EB BA 30 8C 3D 7A D6 AA 42 A4 29 15 0A

45 42 55 25 51 AE 6B B8 18 D2 61 18 4C D2 AB 4D

36 99 48 20 F1 E4 72 38 2D 8B 6B 42 D0 D9 36 48

9B 49 07 3F 96 15 86 5C B6 31 CD 1C D0 8E 47 23

91 D6 F1 6F 0C 53 11 C7 99 85 B5 6D 86 78 57 6D

B6 E2 74 B7 CF 8B 65 B7 11 C4 4B 52 D6 A9 AA 7A

98 AE 7B 9E B0 AC 2F 9F 10 92 8B 28 B6 C2 C5 B0

B0 C9 0C 90 DA 36 B0 23 2D 2D 6B 53 C4 C4 53 6D

B6 A3 56 24 48 DB 4E 4B 12 C2 48 92 36 6D 98 E0

D1 68 9A 24 21 08 7A F6 D6 9D C9 DC 92 15 64 85
 

·
Registered
Joined
·
752 Posts
could the ndx file be the commercial skip information?


At some point, someone has to write a program to interpret the ndx into Womble and automatically cut out the commercials from the mpeg! :) :)


Have you tried playing the file in a software DVD player such as PowerDVD or WinDVD? I know that WinDVD will play straight mpeg2 files. If you want to send me the file, I could try opening it in any of those or Womble.
 

·
Registered
Joined
·
3,686 Posts
I doubt that it has commercial skip information. I distinctly recall them being present on the 2000/3000 series MPEG partitions as well.


I always assumed they contained specific show content information as well as some kind of back-pointer to the replay channel database on the system partition.
 

·
Registered
Joined
·
42 Posts
Discussion Starter · #19 ·
My new years gift to the comunity, the source code for my ReplayPC application. This compiles under VC6.0, and runs on my Win2K system. You mileage may varry.


Note: The output mpg file is not correct yet, but I am getting close.

Code:
Code:
//**********************************************************************************************
//
// ReplayPC.c  Retrieve the guide file and .mpg files from a 4000 series ReplayTV via network
// WARNING - This is ALPHA software. Don't try to use this unless you know what your doing!
//
//
//  Release history:
//  v0.1    01/01/2002  First Souce Code Release, Just to let others help in this effort
//                      This code comiles with VC++ 6.0, but I've attempted to keep it as
//                      portable as possible to other operating systems. Thats why I chose
//                      to write this as a Win32 Console Application. Microsoft makes this hard!
//                      Development and testing were done on a PII-400 Win2K system     
//
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
//
// If you distribute a modified version of this program, the program MUST tell the user
// in the usage summary how the program has been modified, and who done it!
// Modified source code MUST be distributed with the modified program.
//
// Please submit patches and sugestions to  [URL='mailto:[email protected]'][email protected][/URL] 
//
//**********************************************************************************************
#include 
#include 
#include 
#include 

//Tell the linker to link with the winsock library
#pragma comment(lib, "ws2_32.lib")


//**********************************************************************************************
// Global Variables and Defines
//**********************************************************************************************



//**********************************************************************************************
// Function Prototypes
//**********************************************************************************************
void GetReplayGuide(char *IPAddress);
void GetMpgFile(char *IPAddress, char *FileName);
int  RecieveHTTPData(SOCKET s, char* Buffer, int Len, char Terminator);


//**********************************************************************************************
// Main Loop
//**********************************************************************************************
int main(int argc, char* argv[])
{
    WSADATA wsaData;            //Winsock Garbage
    LPVOID  Dummy       = NULL; //Winsock Garbage
    int     fGetGuide   = 0;    //Flag, true if were getting the guide file
    int     fGetFile    = 0;    //Flag, true if were getting a file
    char    *FileName   = NULL; //pointer to filename
    char    *IPAddress  = NULL; //pointer to ip address as a string
    int     i           = 0;    //temp counter

    //fire up windows sockets version 2.2
    if(WSAStartup(MAKEWORD( 2, 2 ), &wsaData ) == -1)
    {
        perror("WSAStartup() Failed\
");
        exit(1);
    }

    //Print usage summary if options are not supplied
    if(argc > 8);
                    sys_hdr[8] = (unsigned char)(rate_bound & 0xff);
                    fwrite(sys_hdr,1,sizeof(sys_hdr),fileptr);

                    //Finally, write the program stream map
                    fwrite(psmap,1,sizeof(psmap),fileptr);
                }

                //write the data to disk
                fwrite(Buffer, len, 1, fileptr);

            }while(chunklen);

        }
        else
        {
            //zero lengh chunk marks end of transmission
            Done=1;
        }
    }while(!Done);
        
    //all done, cleanup as we leave
    printf("\
Done, output written to [%s]\
", FileName);
    fclose(fileptr);
    closesocket(s);
}


//**********************************************************************************************
// A helper function for dealing with HTTP chuncked data
// We have to do this when using WinSock, because WinSock sockets are NOT streams
// in a Unix system a socket is also a file descriptor allowing use of stream I/O
//**********************************************************************************************
int RecieveHTTPData(SOCKET s, char *Buffer, int Len, char Terminator)
{
    static int PacketLen=0;
    static int SrcIndex=0;
    static int DstIndex=0;
    static char PacketBuffer[1500];
    char c;

    //Destination is zero lenght each call
    DstIndex=0;
        
    if (Terminator)
    {
        //Were retrieving terminated data
        do
        {
            //Retrieve HTTP data as required into PacketBuffer[]
            if (SrcIndex == PacketLen)
            {
                PacketLen = recv(s, PacketBuffer, sizeof(PacketBuffer), 0);
                SrcIndex = 0;
            }

            //Copy a byte from PacketBuffer[SrcIndex] to Buffer[DstIndex]
            c = Buffer[DstIndex++] = PacketBuffer[SrcIndex++];
            Buffer[DstIndex]=0x00;

        }while(c != Terminator);
    }
    else
    {
        //Were retrieving binary data
        do
        {
            //Retrieve HTTP data as required into PacketBuffer[]
            if (SrcIndex == PacketLen)
            {
                PacketLen = recv(s, PacketBuffer, sizeof(PacketBuffer), 0);
                SrcIndex = 0;
            }

            //Copy a byte from PacketBuffer[SrcIndex] to Buffer[DstIndex]
            c = Buffer[DstIndex++] = PacketBuffer[SrcIndex++];

        }while(DstIndex
 

·
Banned
Joined
·
316 Posts
Very nice work, Mlinehan. Thank you.

I'll see if this'll compile on Unix libraries.


And FreezeFrame, any dorkus can say he 'knows the secret but can't reveal it', and he'll still look like a dorkus for saying it. Happy farging new year.
 
1 - 20 of 499 Posts
Status
Not open for further replies.
Top