After my DMR-EH55 froze up in error mode last January during a 'divide title' operation, I searched all over the forums and even posted to few, but everyone said there is no way to retrieve the recorded programs. I finally attacked this problem on my own and came up with a way!! ( Of course your drive still has to be physically working )
All you need is a decent hex editor program ( I use HxD ) and a way to connect the the HD to your PC ( An IDE to USB adapter cable is what I use ). It doesn't matter if Windows can't see the drive, a hex editor can retrieve the binary data from any storage device connected to your PC! By studying the hex codes of DVD vob files, and doing some web research, I found out the start codes for all of your recorded entries on the HD. Also it is possible to locate every part of a fragmented program and reassemble them into one file. I've already retrieved two complete 30 min. shows from just the first 20 GB of the drive. ( Both were in 3 fragments each ). The data is NOT encrypted in any way (just standard DVD vob file format), It's just recorded to the HD sequentially until the drive reaches the end, then goes back to the beginning and starts recording over the places where you 'erased' entries or parts of shows (which are not really erased, just recorded over). This is what causes the fragmentation. It does take a considerable amount of time and patience, since you have to do a lot of string searching, copying and pasting data blocks, and looking at the count patterns ( and have a reasonable knowledge of the hexadecimal number system ). The beginning of the drive does not contain video files, but may have info on where to find the fragments. After I finish the whole drive ( it may be several weeks ) I'll investigate this further.


Part 1 - preliminaries:
If you can't install an IDE drive into your PC, you will need an IDE to USB adapter cable with power supply and ac cord ( available on eBay or websites ).
Go to these websites and download & install the following two programs:
http://mh-nexus.de/hxd/ HxD 1.7.1.0 beta (Hex editor)
http://www.freebyte.com/hjsplit/ HJSplit 2.3 (file joiner/splitter)


Carefully unplug the ribbon cable and power plug and remove the hard drive from the unit.
Move the jumper from J48 to J50 ( CS to Master ) and remove the horizontal jumper connecting J44 to J42 ( Maxtor HDs have a diagram on top ). Place jumper and screws in a small zip-loc bag or container where they won't get lost! Romove the two screws near the connecters holding the drive to the metal frame, loosen slightly ( but do not remove ) the two rear screws. Lift the front of the drive just enough so the screw holes are above the frame, then put those screws back in just enough so the screws will rest on top of the frame and hold the drive up at a slight angle ( so you can get the connectors in ). Plug in the power and data connectors, then plug in the power cord. When the drive spins up, plug the USB cable into your PC. Don't let your OS try to format the drive in case it asks for this.
Create a folder in one of your top directories to keep all your work. Open up notepad and save an empty dummy file there, calling it something like "zero" or"0000". Open up HxD, under file tab, open this file and be sure it is empty. If not, type "ctl A", hit delete key, and under file, "save as" (same filename). You will need this for pasting data into from your HD.
To access the drive, click the "Extras" tab, then "open disk...". Under logical disks, you should see only C: ( and any drives or partitions you added ). Under physical disks, it will say "Hard Disk 1", "Hard Disk 2", etc. You will see one more disk here than what is listed under logical. This will be the DMR drive. Click that one (the second or last one). You will then see the data displayed in hex code. To see if this is the right drive, move the slider down to the end of the file and note the hex offset address of the last line. For a 100 GB drive, it will show a 10 digit number starting with "17" or close. A 200 GB drive will show something like "2E" or "2F". Note that a 100 GB drive is really around 94 GB. Also if you are using the adapter cable, an orange light will flash on the connector each time you access the drive. Important: When opening up HxD to read the drive, the 'read only' box should stay checked to prevent you from writing to the drive by accident.

Part 2 - Data transfer:
Create a notepad file, copy and paste the following info into it:



VOB file START STRING 000001BA4400040004
2KB data block start string 000001BA

hex
100 = 256 bytes
1000 = 4096 bytes = 4KB
1:0000 = 65,536 b = 64KB
10:0000 = 1 MB
100:0000 = 16 MB
1000:0000 = 256 MB
1:0000:0000 = 4 GB
10:0000:0000 = 64 GB

6 MB = 600000
12 MB = C00000
24 MB = 1800000
48 MB = 3000000
96 MB = 6000000
192 MB = C000000

FILE 00 ( 1st 4 gigs of HD 0000000000-00FFFFFFFF )



You will need this often for reference. From now on, everytime you locate a program or part of a program, you will need to record the hex addresses, file length, and part of the header string for each program segment you find in this notepad file. Be sure to SAVE this file often as you work on it ( even make backups to another directory each day ).

Note: If you use a PC where you can connect the drive internally, you shouldn't have to transfer all the data to your C: drive first, you can skip down to part 3 below. You can do your string searching directly from the drive. I just can't do it with my USB connected DMR drive because it takes so long even to copy a 256 MB chunk off.

If you are using a IDE to USB adapter cable:
Before you begin, you need at least as much free space on your PC as the capacity of your DMR drive ( plus another 50% or so to work with ). You may have to add a second drive. Don't use a USB connected drive for this, it will slow things down considerably. The hex editor itself does not need much free space to display the data ( it only retrieves what is visible in the window ) but to search for programs and move data around, you will want it all on your PC's HD. With the hex data displayed on your HxD window, note the cursor flashing to the left of the first data byte. Under the 'edit' tab, hit 'select block', choose the 'Length' selection, then type in 10000000 (seven zeros) and hit OK. This will select the first 256 MB of data. Type 'ctrl C' (copy) A progress window will open, and the light will blink on the connector as it copies data to the clipboard. This will take at least a minute. When done, click to the left of the first byte of the very next line below the blue highlighted data. Leave the file open, Under file menu open up your empty file and type 'ctrl V' (paste). Click OK when it asks about the file size. You'll see the data appear in red. In the file tab, click "save as..." and call this file 00.001 hit save, then close the file. Under the HD file, The cursor should still be at the beginning of the line address 0010000000. Now repeat the steps above
(select block, 10000000 size, copy, paste into zero file, save - only this time call the file "00.002". Repeat this process 14 more times, being very careful not to lose track of where you are ( When you finish saving file 00.002, you will see a 2 at the start address of the next block you will copy ). Also, if you are worried about losing track, you can copy down the last few bytes of each 256 MB data block and put it with the correct filename in notepad. After you save the 10th file (calling it "00.010", you will see an A at the next start address. When you finish with the 16th file, the next hex address showing will be 0100000000. Now you can join the 16 files into a 4 GB file. Run 'HJSplit' hit "join" browse for the 00.001 file and hit 'start'. After a while, you will end up with a file called "00" with the first 4 GB of your data. You can delete all the ~.001 - .016 files.
Now you can either are start analyzing the data ( see part 3) or continue with the next 4 GB block.
To continue, copy this into your text file ( a few spaces below the last entry )
FILE 01 ( 2nd 4 gigs of HD 0100000000-01FFFFFFFF )
Starting with hex address 0100000000 retrieve the next 16 blocks ( just like above ) only this time title each file 01.001 thru 01.016 instead of 00.xxx If you close the program and want to do it later, use the 'search' tab and 'Goto' to type in the address of where you need to start. Note that to transfer all 100 GB, you will have to transfer about 375 blocks of data this way. I was unable to transfer larger blocks as I would get an 'out of memory' error each time I tried. There may be a way to copy the entire drive at once, but I yet don't know how and couldn't do it anyway as I am still making room on the hard drive.
Ther is a reason for naming the 4 GB blocks "00","01","02", etc. By adding the numbered name of the 4 GB file you are in to the left of the 8-digit hex address of what line you are looking at in that file, you will have the complete 10 digit hex address for that data line for the whole drive.


Part 3 - Finding your recorded programs:
Open the 00 file in HxD. Hit search and Find. Change the data type from Text-string to Hex-values (it always defaults in text mode when you first use it - so you need to change this every time you first open HxD on your PC). Copy and paste the VOB file start string code 000001BA4400040004 into it and hit OK. Once it finds one, copy the 8 digit address code (or 10 digit code if you are reading from the drive) on the left of this line into your notepad file right below the heading
"FILE 00 ( 1st 4 gigs of HD 0000000000-00FFFFFFFF )"
Note: If you are accessing the drive directly, no need to create the above heading - this is only if you have to copy everything to your C: drive in 4 GB files.
All DVD vob files start with this. This string also appears at the start of anything you record to the hard drive. Now hit edit and select block (leave the highlighted area alone) and type in 1000000 in the length box. Copy and paste this into your zero file, then save it as an .mpg file, calling it something like "a1.mpg" You will have a 16 MB file. Be sure to close the file in HxD once you save it (or it won't play). Now play it with your favorite video player ( I use Media Player Classic ). In LP ( 4 hour ) mode, this will give you about 50 seconds of video. This is usually enough to tell if it something you need or fragments of old deleted shows. If it is old fragments, make a note of this next to the address in your text file so you won't look there again. If it is good, and the entire video is the same program, Then you will need to go back to the same address (Find and Goto) on the 00 file, only this time select a 256 MB block ( 10000000 length ). Copy and paste it the same way, you can even overwrite the first mpg file with the same name when you save. If the video switches over to another fragment part way through the file, you will then need to edit it down. This is where it gets more complicated. First, observe about how many minutes and seconds are in the portion you need to save. I find most fragments fall into the following file lengths: 6 MB, 12 MB, 24 MB, 48 MB, 96 MB, 192 MB and 384 MB. 192 MB fragments are fairly common, this is exactly 3/4 of the way through a 256 MB file, or about 10 min. of video in LP mode (in SP mode, cut the times in half). So if your video plays about 10 min in a 256 MB file before switching shows, you have a 192 MB chunk. First, open the mpg file in HxD, type in 000001BA in the search / find window. Now press and hold down the F3 key on your keyboard and watch the first three bytes to the right of the blue selected string. You will see a '44 00 04' there which will slowly count up as you advance into the file. The 6th digit will only count from 4 to 7 and C to F, skipping the other 8 digits. The first 5 digits count up in the normal way. (The 44 won't change until you get 30 or 40 min. into a program) Let up on the F3 key, then move the slider bar down about a tenth of the way, press F3 again and observe the count. Keep doing this to get a feel for about how fast the count is going up. Move almost 3/4 down the file and look again. Sooner or later, you'll see a discrepancy in the count ( the numbers will jump to much higher value, or back to a lower value ). Work the slider up and down (moving it as little as possible) to zero in on the discrepancy. You can hold the shift key down while pressing F3 to make it search up (backwards). Also, if you have an idea of the chunk length, just go to that hex address on the left. Here are the address codes for the following chunk lengths: 6 MB = 600000 ; 12 MB = C00000 ; 24 MB = 1800000 ; 48 MB = 3000000 ; 96 MB = 6000000 ; 192 MB = C000000 . Hit the F3 and shift F3 keys to verify this is the end of the chunk. The next chunk starts at a hex address ending in several zeros.
Click the start of this chunk, scroll to the end of the file, hold down 'shift' and click the cursor on the right side of the last data byte. Now hit the delete key to remove this. Resave
it under the same filename. Close the file and play it again. The playing time, total time and seeker bar on your media player should now be in sync once you remove extraneous chunks off the end. ( There's another way to tell, if you play a 256 MB file, which should be about 13 min in LP mode, but if the program length time showing is way off from this, then you have mixed programs. Also the seeker bar won't move at the right speed ). Once you have an intact chunk you want to keep, rename it with a number like 01 and the name of the show so you know what it is - followed by "inc" or incomplete. Reopen the saved chunk and search for 000001BA again, but move to the end of the file, using shift F3 to find the last occurance of this string. Copy down the four bytes after the highlighted search string bytes into your text file, next to the info you copied about this chunk. You will need this to find the next chunk of this show. Continue searching for all the start codes in this 4 GB file, Then you can proceed to the next file.
Note1: Commercials you deleted on the DMR are NOT deleted from the HD, they will all still be there. This info is probably stored in the flash memory or at the beginning of the drive.
Note2: The 000001BA code occurs every 2 KB on the drive. Any data chunk starting with this code and ending with the byte preceding this code ( no matter how long ) will play as an mpg file. It does not have to begin with the entire string 000001BA4400040004. You can even name a whole 4 GB file as an mpg and play it! ( Except for the first one, since there are no 000001BA codes near the beginning of the drive ).
Note3: A "byte" in hex code is always a pair of digits shown in the editor


To find the second and subsequent chunks from a show, You will need the first 8 bytes of data from the start of the last 2 KB data block of the first chunk. Another words, the code 00 00 01 BA plus the four bytes I told how to find above.

Here is an example of a 30 min show I retrieved ( It was in the 2nd 4 GB file, about 7 GB
into the drive )
E5180000 [show #1 - part 1] (192MB 10:57)
00 00 01 BA 44 38 6E 57
00 00 01 BA 44 38 6E 5C
This is the last two 2 KB block start codes. Note how the last digit increases from 7 to C
( a jump of 5 units ) This digit always seems to advance in steps of 4 or 5.
To find the next fragment, I did a search for 000001BA44386E ( everything except the very last byte ) Each time your editor finds this string, look at the first byte after the blue selected code. In this case, it should be about 4 or 5 bigger than 5C ( it turned out to be 60 when I found it ). If the number is way smaller ( usually near 00 ) then you are in the wrong program. Hold F3 down until it gets past this show ( you will see a rapid flash of the strings, then a pause and the 'searching' box appears - immediately let up on the key and look again when it finds the next string. If the number is still way off, repeat the above.
If you find a string that looks right, select, copy and paste 16 MB of data into the zero file, save and play it. If it is not the right one, keep searching. In my case, the second segment was in the 4th 4GB file ( about 14 gigs into the drive ) Once you find it, Use the procedures above for selecting larger blocks and trimming the file to the right size. You will also want to record the addresses of all additional segments you find and again record the first 8 bytes of the address from the beginning of the last 2 KB data block of this file ( so you can find the third part ) and so on. Once you get the second segment fixed up, open the first segment in HxD, move the cursor to the very end, open the second part, then select all, copy and paste this to the end of the first part and save it. (Maybe under a different name in case you messed up) Once you play it and see all is OK, then you can delete the two shorter segments.

Wade

Wow:eek:You're the man:cool:
Good reference for anyone who has crucial programs on there HDD.
I'm going to guess most people will will just live with the fact that there programs are lost forever, feeling sad but moving on. It's great you posted your workaround if they really want to go the extra mile and get there data back.
Thanks for sharing your solution:) I'm sure a few people will try it, if the time comes.

"Impressive...most impressive (http://www.geocities.com/Hollywood/Bungalow/3606/impress.wav)."

All I can say is, Well Done! Some of us have been waiting a long time to hear of success in this. So can we assume that at least for the moment, it is quite possible, but time-consuming and very, very tedious? This is a BIG step forward from impossible.

Info on how to do the equivalent thing on a Toshiba XS32 hosed drive would be greatly appreciated.

Well done, really inspired me to try something similar with my DMR-EX75. I seem to have a "MEIHDFS-V2.0" harddisk layout... Am abit stuck with trying to get the "VOB file" start information out of the title information section though...

I erased 48 minutes of a one hour funeral. I'm not done putting it together, but your instructions are great.

The info i had lost was already backed up on DVD's. However, some had not been finalized (the final step).

Other than recovering data, were you able to get DVD- Recorder repaired with reformatted hard drive?

Hi !

I'm a french guy -sorry for my english-, and I've got a PANA DMR-EX87, with 250 Gb HDD.

The drive is full of 150 cartoons for my childs, and I don't want to burn a lot of DVD-R before erasing them :o .

I putted the PANA Hard Drive ( Western Digital 2500 BB in fact) in my PC, but Windows doesn't recognize it.

BUT.....

I tried an EXT2 File system Reader (a LINUX File System) => the program want to scan the entire HDD. Il let it work during 10 minutes, and it scanned +/- 5 % of HDD. So I stopped it, but IT FOUND 4 DIRECTORIES AND a lot of MPEG and TXT FILES !!!!!:cool: :cool: :cool:

It was a surprise, because I though there was a FILE INDEX, the EXT2 Reader did'nt find it. But it found back files.
Some MPEG files are only 2 Kb size... Perhaps because I stopped the search.


I had to reinstall HDD into PANA -> no restart problem but a long time to start (with a little panic :rolleyes:)


I will try in next days or next weeks to start on a SUSE LIVE CD (or another LINUX LIVE CD), to view if it can read the PANA HDD.

Or if someone has left time to try this....

Hi !

I'm a french guy -sorry for my english-, and I've got a PANA DMR-EX87, with 250 Gb HDD.

The drive is full of 150 cartoons for my childs, and I don't want to burn a lot of DVD-R before erasing them :o .

I putted the PANA Hard Drive ( Western Digital 2500 BB in fact) in my PC, but Windows doesn't recognize it.

BUT.....

I tried an EXT2 File system Reader (a LINUX File System) => the program want to scan the entire HDD. Il let it work during 10 minutes, and it scanned +/- 5 % of HDD. So I stopped it, but IT FOUND 4 DIRECTORIES AND a lot of MPEG and TXT FILES !!!!!:cool: :cool: :cool:

It was a surprise, because I though there was a FILE INDEX, the EXT2 Reader did'nt find it. But it found back files.
Some MPEG files are only 2 Kb size... Perhaps because I stopped the search.


I had to reinstall HDD into PANA -> no restart problem but a long time to start (with a little panic :rolleyes:)


I will try in next days or next weeks to start on a SUSE LIVE CD (or another LINUX LIVE CD), to view if it can read the PANA HDD.

Or if someone has left time to try this....

I'm suprised it didn't try to reformat the drive when you re-installed it.

Why not hs dub to DVD-RAM instead?

I'm suprised it didn't try to reformat the drive when you re-installed it.
No reformat asked by the DMR.
I didn't know it could ask me to reformat, before reading posts here... :eek:

But if I understood what was done, someone putted a NEW HDD, then PANA asked to format it. After reinserting PANA's original HDD, it was then not recognized.
=> I think that PANA tags the HDD, as Windows does (I think in the MBR).
It can explain why I didn't have to reformat my disk, I said no to 'make this drive used by Windows' - Even if Windows and PANA's tag use same location, no problem...
I used a freeware program to read EXT FileSystem, which does'nt use Windows HDD manager.

Why not hs dub to DVD-RAM instead?
I don't understand "HS" ('remember, french guy...;))
You mean using PANA to dub my videos to DVD-RAM ?
What is a difference between DVD-RW - the size ?

HS=high speed.

No reformat asked by the DMR.
I didn't know it could ask me to reformat, before reading posts here... :eek:

But if I understood what was done, someone putted a NEW HDD, then PANA asked to format it. After reinserting PANA's original HDD, it was then not recognized.
=> I think that PANA tags the HDD, as Windows does (I think in the MBR).
It can explain why I didn't have to reformat my disk, I said no to 'make this drive used by Windows' - Even if Windows and PANA's tag use same location, no problem...
I used a freeware program to read EXT FileSystem, which does'nt use Windows HDD manager.


I don't understand "HS" ('remember, french guy...;))
You mean using PANA to dub my videos to DVD-RAM ?
What is a difference between DVD-RW - the size ?

It seems to me that you could readily dub the content off your Hard Drive using DVD-RAM disks in HIGH SPEED dub mode (HS) quicker than removing and fiddling with your HDD. The main advantage of the DVD-RAM vs. DVD-RW is that they preserve the original VR mode format of the videos recorded to your hard drive (which enables you to losslessly dub them BACK to your HDD should you choose to do so in the future) and they do not have to be finalized. I suspect that you could dub a full disc in about 15 min which means that you could unload between 2 to 4 hours of content (depending on record mode - i.e. SP or LP) every 15 to 20 minutes. Seems a lot more straight forward and less risky than fiddling with the hard drive on a PC. However, it would be interesting to see what success you might have going the PC/Linux route. Good luck, in any event.

Thanks for the idea (and for HS translation. In french, HS is for Hors Service = out of order ;)).

I didn't use DVD-RAM 'till now.

When I'm dubbing to DVD-RW, it's done at 1x (due to LP or SP mode I used to record to HDD, if I remember).

If I use DVD-RAM, It could be faster, as you said ?:rolleyes:
:) If can dub2 hours within 15-20 minutes.... That could be easier !
and moreover a normal use of the PANA

Thanks for the idea (and for HS translation. In french, HS is for Hors Service = out of order ;)).

I didn't use DVD-RAM 'till now.

When I'm dubbing to DVD-RW, it's done at 1x (due to LP or SP mode I used to record to HDD, if I remember).

If I use DVD-RAM, It could be faster, as you said ?:rolleyes:
:) If can dub2 hours within 15-20 minutes.... That could be easier !
and moreover a normal use of the PANA
As you're using a European model Panasonic, I'm not sure if this will apply to your recorder but...

High Speed dubbing works-on the North American Pannys, with all discs types, allowing you to dub material from the hard drive to a disc in the 15-20 minute time frame. Now -RW discs come in a couple of different speeds...some are 1x-2x, and others are 4x speed. The lower speed (may) mean it will take closer to 20 mins to dub at high speed (I've never used 1x-2x -RW discs in my Panasonic hdd recorder). 4x discs should take about 15 minutes.

The American hdd model also has a setting in the Set up menu where High Speed dubbing must be enabled. If it is not, then all material recorded on the hdd, can't be hi-speed dubbed-only dubbed in real time, the actual running time of the show. So you might need to check the manual, to see if your model also has this setting, and if it needs to be turned on.


Also be aware that RAM discs can only be played back on other Panasonic recorders, or pcs with RAM capable drives. They won't play on any units that aren't RAM compatible.

The American hdd model also has a setting in the Set up menu where High Speed dubbing must be enabled.
:o I forgot that, and my option was set to "NO".
:( but it's only used for new records to HDD, and forces 4:3 screen size and audio M1 and M2.

It also turns HS for DVD-RAM...

So no way for the already recorded cartoons on my HDD.

Sorry if this violates old post policy.
Many thanks to the author, saved my bacon :)

My optical drive had died on my HS2 so I had no way to get footage off.

I went a slightly different way which allowed me to see what I was cutting up.
Rather than piecing together the footage using the hex editor I just used the editor to dump contents of the drive and used a mpeg splitter to work on it.
At first I dumped all 40gb but then went to 20gb chunks.

Program used was
SolveigMM Video Splitter

It handled the mixed video resolution file just fine.

I finally broke down and added a 500 GB HD to my PC so I could finish what I started over a year ago. In a matter of weeks, I have now recovered EVERYTHING I had recorded on the panny HD before it crashed
I have also made the following new observations:
Data fragments always start with HEX addresses ending in 0000 and the fifth to the last digit is always a 0, 4, 8 or C. Whenever a new recording is started, the DMR will scan the drive for blank contiguous spaces of at least 384 MB in size, and begin in 384 MB fragments. ( An empty drive may use larger fragments, or just record the whole title on one huge chunk ) If none exist, it will then try 192 MB ( half the size each time ). All programming will start out in either 384 MB, 192 MB, 96 MB, 48 MB, 24 MB, 12 MB, 6 MB, 3 MB, or 2.25 MB fragments depending on what is the largest size of the empty spaces left on the drive. The fragments are always recorded sequentially from the beginning of the drive to the end, and are always the same size, until the end of the drive is reached. It will them go back to the start, and record the next set of fragments at half the original size. This process will continue until the recording is ended or the drive runs out of 2.25 MB spaces, in which case it will stop and show a "drive is full" error. Of course the final fragment will be smaller as it depends on when the recording is stopped.
I had one 30 min. show that started out in 6 MB fragments and ended up in 2.25 MB fragments, and I had to locate and rejoin 168 fragments in all!
When my unit started acting up ( like losing divided titles ) I was trying to get everything on to DVDs, so the last week or two I used it, the drive was only about 25% full, and most of these shows were in a few large ( 192 or 384 mb ) fragments. But I had a few older items in over 50 tiny fragments which were recorded when I let the drive get over 95% full.
Also some of the shows no longer started with usual 8 byte HEX start code, but instead had a much higher number in the 5th to 8th byte. These may have been the 2nd ( or 3rd ) part of a divided title. I had to find these by adding the .mpg extension to each of my 4 GB data files and playing them with a video player while advancing the seek bar about 2% of the way in increments until I found the show I was looking for, then trying to zero in on the beginning of a fragment by selecting various smaller & smaller segments of the file and playing each. You can also use my method to search backwards for preceeding fragments, it's just more difficult. Of course you will be counting backwards with the numbering sequence and also looking for the last code in each fragment, where the HEX address willes will end in
3F800, 7F800, BF800, or FF800 instead of 00000, 40000, etc. One program near the very beginning of the drive was missing the first 12 KB of data ( and thus the start code ). But the video was still playable. This may be where my errors are coming from. I also found some of the filenames I assigned to some of the titles (over in the text column on the right side of the editor) - they seem to be scattered all over the drive. I spotted a few by accident right after the end of a fragment I had selected to copy off.
I put the drive back in last night but the error is still there - no attempt at an automatic reformat. I may have to fill the drive with zeroes to clear this silly error. I have it all on my PC now, so it doesn't matter. The next step will be to find a way to write the shows to a DVD-ram on my PC to get them back into the DMR's HD without any compression loss.

Wade

Great work Wade. I've followed your trailblazing path after accidentally (re)formatting the 250GB hard drive on my Panasonic DMR-EX85EB.

I'm searching and retrieving fragments with HxD but want to automate one or two things. Any ideas on programming languages, scripts or whatever to do the following?

Search for a hex pattern (01BA44) and then look at the next three bytes. Move on if it's a contiguous video fragment (the next three bytes are incremented normally) or stop the search when those three bytes show something other than the regular increment, i.e. when the fragment has finished.

Thanks in advance for any ideas.

Ian

I've achieved what I wanted by setting up a macro in Macro Express (Insight Software Solutions) to cut and paste from HxD into Excel (to conduct analyses) and then conditionally return to HxD to continue searching.

I have just gone through the same problem with an EX75 unit. The HDD started clicking like the head was bouncing off something at startup, resulting in HDDERR on the display, but after several attempts it would boot normally allowing us to watch a few of the recorded programs before it stopped working again. Obviously the HDD was on the way out, and with only 47mins free, we had rather stupidly got rather a lot of soon to be lost recordings.

After reading a few forums regarding other ppls attempts at recovering / replacing the HDD, coming from a SW engineering background, I managed to get the HDD attached to my PC directly through the IDE connector, and wrote a quick tool to read off the raw data from the HDD into 1GB files giving me 149 1GB files.

I then ordered a new 160GB WD replacement HDD (almost the same spec as the original). I then wrote another tool to take all the the 149 1GB files and write them as raw data onto the new HDD.

After fitting the HDD into the EX75, to my suprise after reading all of the failures on the net, after a tense 'PLEASE WAIT', the unit came up with 47mins remaining, and operating completely normally.

I did start looking at reconstructing the movies using the info posted here, with some success, but would need to work out how the table of contents works to link together all the sub segements of movies, and gave up once I got the new HDD working.

Happy days, looking fwd to another 2 years usage from the unit.

Sounds like cloning to me;) What about the digi's that reformat when they see a different hard drive is installed?

Sounds like cloning to me;) What about the digi's that reformat when they see a different hard drive is installed?

I guess cloning would do the same, but I wanted to have a close look at the data with a view to recovering lost movies, so a hand made approach made sense.

If the format on different HDD detection is true, it would be a problem, I'm not convinced that this is the case, as there doesn't seem to be a convincing argument why Panasonic would do this, especially on some units in the same range and not others, but then I guess anything is possible. I suspect that its more likely the data on the original is corrupt sufficiently, say in the table of contents, for the unit to demand a reformat, as this is what happened when I replaced the original HDD the very last time.

What if you did like you did, except putting the data on a larger hard drive? Or are they only capable of a 160 gig drive?

Panasonic DVD recorders all have the size of their hard drive set in the firmware. If you put in a larger drive, it will format to the size of the drive that originally came with the machine. This characteristic is common to nearly all HDD equipped DVD recorders.

As far as swapping drives is concerned, everyone who as tried and reported back here has the same story. Once a drive is removed from a machine, if you try to use it on another machine, even the same make and model, the machine will demand it be formatted at first power-up. If you then return it to the original machine, IT will demand the drive be reformatted. I haven't tried any of this myself, but many others have stated this many times.

That may be, but if mindeater "cloned" to a new drive, he already broke the other rule that said it would reformat the drive if it did not read the same identification id of the original drive. Maybe in looking at the info he transferred, he could brake the code allowing it to recognize a larger hard drive.

Panasonic DVD recorders all have the size of their hard drive set in the firmware. If you put in a larger drive, it will format to the size of the drive that originally came with the machine. This characteristic is common to nearly all HDD equipped DVD recorders.

As far as swapping drives is concerned, everyone who as tried and reported back here has the same story. Once a drive is removed from a machine, if you try to use it on another machine, even the same make and model, the machine will demand it be formatted at first power-up. If you then return it to the original machine, IT will demand the drive be reformatted. I haven't tried any of this myself, but many others have stated this many times.
The Philips 3575/3576 and Magnavox 2080/2160 are one exception... larger HDDs, swappable w/o reformatting, multiple drives for portability, etc., as described here. (http://www.avsforum.com/avs-vb/showpost.php?p=12355769&postcount=39#HDDpioneers)

My expressoin "nearly all" implied some exceptions. The Polaroid 2001G is another exception.

Panasonic DVD recorders all have the size of their hard drive set in the firmware. If you put in a larger drive, it will format to the size of the drive that originally came with the machine. This characteristic is common to nearly all HDD equipped DVD recorders.

As far as swapping drives is concerned, everyone who as tried and reported back here has the same story. Once a drive is removed from a machine, if you try to use it on another machine, even the same make and model, the machine will demand it be formatted at first power-up. If you then return it to the original machine, IT will demand the drive be reformatted. I haven't tried any of this myself, but many others have stated this many times.

That's all I keep hearing from people that it never works, but the drive I did it with is not even the same model, old one a WD 1600BB the new one a WD 1600AA, I didn't do anything I would consider clever and it worked first time. As I said, I think people making a lot of assumptions over what is actually happening when they replace a drive and it doesn't work.

The concept of requiring 'firmware' when it comes to a HDD is very odd, as the only firmware on a WD HDD is written onto the drive by WD themselves an the time of manufacture, I don't think Panasonic are going to go to the trouble of re-writing HDD firmware, I don't think they'd have the budget or the inclination to go to that sort of trouble.

Are you saying that you put in a different hard drive and it formatted to a size greater than the one that originally came with the machine? If so, that is the very first time I have ever heard of this. Many different drives will work, and many people have put in drives that were larger than the one that came with the recorder, but everyone who has done this reported that the drive formatted to the size of the original one that came with the machine. The recorders just ignore the extra space. They work just fine, this way, but the extra space is ignored by the recorder. It does not read the drive for its size and use that number of GB, it uses the number of GB that is in its firmware. It does not write any firmware to the drive, just video data.

Maneater used the same size drive. I think I was trying to hint that if he still had the info on his computer, if he would try to see if it would work if he put it on a larger drive. I was reading the tivo hacks and they have been able to mount the linux drives and have been able to resize when going to a larger drive and retain the "recorded material." I think it can be done on the Philips, Panos, and Maggies, if the right person new how to read the correct info to do it. Maybe the tivo hackers could branch out to other types of PVR's

I then ordered a new 160GB WD replacement HDD (almost the same spec as the original). I then wrote another tool to take all the the 149 1GB files and write them as raw data onto the new HDD.

After fitting the HDD into the EX75, to my suprise after reading all of the failures on the net, after a tense 'PLEASE WAIT', the unit came up with 47mins remaining, and operating completely normally.

I did start looking at reconstructing the movies using the info posted here, with some success, but would need to work out how the table of contents works to link together all the sub segements of movies, and gave up once I got the new HDD working.

I'm a little confused, Mindeater.

If "the unit came up with 47mins remaining, and operating completely normally," then why did you "start looking at reconstructing the movies using the info posted here"?

If it only had 47 min. remaining, what about all of the other time on the HDD? Wasn't it viewable on the Panny?

If so, why did you need to reconstruct?

If not, what happened to all your drive space?

bob

That's all I keep hearing from people that it never works, but the drive I did it with is not even the same model, old one a WD 1600BB the new one a WD 1600AA, I didn't do anything I would consider clever and it worked first time. As I said, I think people making a lot of assumptions over what is actually happening when they replace a drive and it doesn't work.

The concept of requiring 'firmware' when it comes to a HDD is very odd, as the only firmware on a WD HDD is written onto the drive by WD themselves an the time of manufacture, I don't think Panasonic are going to go to the trouble of re-writing HDD firmware, I don't think they'd have the budget or the inclination to go to that sort of trouble.

I'd hardly call two WD1600's different models--at least for the purposes used here. The Panny isn't *that* sophisticated. ;)

The *firmware* that was referred to above is not the drive firmware, but that on the Panny's motherboard. Drive size is burned into the unit on manufacture to keep you from buying a cheap model and upgrading the drive. My guess is, if you swapped the prom from an E-95 and put it and a larger drive into an E-85, you'd probably have another E-95.

bob

Originally Posted by mindeater
I then ordered a new 160GB WD replacement HDD (almost the same spec as the original). I then wrote another tool to take all the the 149 1GB files and write them as raw data onto the new HDD.

After fitting the HDD into the EX75, to my suprise after reading all of the failures on the net, after a tense 'PLEASE WAIT', the unit came up with 47mins remaining, and operating completely normally.

I did start looking at reconstructing the movies using the info posted here, with some success, but would need to work out how the table of contents works to link together all the sub segements of movies, and gave up once I got the new HDD working.

a little confused, the same as VideoBob

you should execute a data recovery as soon as possible:D

hi guys, i need some help depserately, i just formatted my hdd it is a panasonic DMR-EX77. i meant to format the dvd disc but accidently formatted the hard drive. and hence lost all the television programs which were saved, these programs are so important and is it possible to get these back? do i use the method that wade described earlier
THANX IN ADVANCE

andre

pleaseeeeeeeeeeeeeeeeeeeeeeeeeeeeee can any1 help !??

pleaseeeeeeeeeeeeeeeeeeeeeeeeeeeeee can any1 help !??Sorry to say, but it is a generally held view that if you format the HDD, you are cooked. There is no recovery from that.

hi guys, i need some help depserately, i just formatted my hdd it is a panasonic DMR-EX77. i meant to format the dvd disc but accidently formatted the hard drive. and hence lost all the television programs which were saved, these programs are so important and is it possible to get these back? do i use the method that wade described earlier
THANX IN ADVANCE

andre

I'm sorry to say that I have never heard of recovering anything from a Panny HDD after formatting. We have been jumping through hoops just trying to get usable files off of a *good* drive, without losing them.

I agree with Kelson, you are probably SOL on this one--but I *could* be wrong.

There are programs to unformat a DOS/Windows formatted drive--but I have yet seen one to even *read* a Panny-formatted one.

bob

You are in luck. I posted my findings in two different, but similar themed, threads:

http://www.avsforum.com/avs-vb/showthread.php?p=16984815#post16984815
http://www.avsforum.com/avs-vb/showthread.php?p=17178491#post17178491

So let me state, that at least for the DMR-EH75V, the 'Format' fucntion is only a quick format that erases the MBR (or whatever is equivalent in teh file system used), but not the files stored on the drive. I'd be willing to bet all other Panny HDD recorders (and other brands as well) are similar. If it takes < 1 min. for the format to complete, it's surely not a low level format and most likely the data can be saved. But there is no way that I know of to simply undelete them and have them show up in the navigator screen. It's up to the end user to decide how to get them off and create video DVD's from them, but the information in this thread is a good start.

Whatever you do, if you absolutely want to save those recordings, do NOT record anything else on the newly formatted drive, as it will overwrite the old files. Image the drive first.

I didn't do anything I would consider clever and it worked first time.

You're being WAY too humble: you're too clever by half;). You continually gloss the fact that you "just happened to write a quick software routine, based on your engineering background". This allowed you to essentially clone the intact video recordings over to a fresh HDD, without losing anything, and in a form that the recorder would still accept, read and work with as if it were still the original HDD.

THAT IS UNHEARD OF. All previous attempts at HDD cloning also cloned the corruption, a zero sum game.

So every other person on this thread is doing the insanely tedious "use an expensive obscure hex editor to reconstruct each little three second clip and painstakingly reassemble the videos onto their PC drives as coherent standardized files they can then salvage using PC authoring tools". That is a task for savants, or fools with a great deal of time on their hands: I wouldn't attempt such a thing if the videos I lost were priceless, never mind the typical mundane TV fare. What you did is something entirely different and the "holy grail" people here really want to discover: how to do a quick and dirty clone of a failed recorder HDD onto a fresh HDD, cloning the video data but not the corruption, and in such a way the recorder accepts the transplant without incident and without insisting on a reformat.

I'm not sure if some of the people responding to this thread after your post understand there are now two distinct recovery methods being discussed, it can cause confusion. It should be made clear that your "holy grail" method requires your "just tossed it off in an hour" special software hack that allows the cloning of the corrupted drive intact onto a new one. Unless you're willing to publish/sell this software, and an instruction manual, your miracle cure is academic: no one else here has any idea how to follow your footsteps. You are a genius, sir.:)

I've just spent 40 minutes composing a response to all the above (giving details of my success at recovering data from a Panasonic HDR) and then been told I'm not logged in and was not given any way of retrieving my text.

So, essence of this message is ....... never hit the Submit button before you cut and paste your text to Notepad, or whatever, or you'll have to start typing again.

Wait on for me to re-compose the original message and post it here. I may, however, post one or two more dummy messages because it seems I can't send an email to poor old andreasn91 (see his post above) until I've posted at least 5 messages.

Here's my 4th (dummy) post. The next will be my promised addition to all the above.

im2020

See my posts above in June 2009. I accidentally (re)formatted the 250GB hard drive on my Panasonic DMR-EX85EB thinking I was formatting a DVD-RAM.

The HDD contained 100 hours of treasures that I had not yet watched so after reading worley45 (wade)'s posts in this thread I was inspired to mount a project.

I took the HDD out of the Panny and connected it to my PC (with an IDE to USB adapter cable) and followed Wade's suggestions for using HxD (Hex editor) and HJSplit (file joiner/splitter) to retrieve my treasures and then burn them to DVD. I created a few tools to streamline the process and ended up with 25 DVDs containing all my treasures.

In the meantime, so that I was able to continue using my Panasonic DMR-EX85EB, I bought a replacement Western Digital Caviar WD2500BB 250GB 7200rpm IDE/PATA for £30. Before fitting this into my Panny I copied onto it the first 320KB from the old HDD (Panasonic's proprietary formatting code) and bingo on switching the Panny back on I was welcomed with the information that I had 111 hours of Standard Play recordable capacity.

So my message is that almost anything is possible.

If you want more details of anything described above or if you want a copy of the code to format a replacement 250 KB HDD post here and I'll respond.

im2020

Hi Im2020

went round my folks yesterday for xmas dinner and their EX85EB is exhibiting all the symtoms of a HDD failure. Why do things always go wrong at xmas?? :confused: That said, there was nothing on worth recording - as usual! :D

They dug the receipt out and most annoyingly its 3 months outside its 2 yr warranty . It wasn't cheap... :mad:

So I've pulled the HD out and stuck it in my PC this morning and sure enough it's knackered - BIOS cant even 'see' it - just the 'thunk' of the arm hitting the end stop on power on. :(

Soooo....I'd be most appreciative of a copy of your code and partition image so I can stick a new HD in it for them. Would that be possible?

cheers
Mark

If you are not trying to get anything off the old hard drive, you won't need his code. Just get a new hard drive and put it in. Upon power up, the recorder will tell you that the hard drive is not formatted properly, and asks to format it. You say "yes" and it does so. End of story, it formats the drive and you can use it. Only one thing, the new drive you install must have a capacity equal to or greater than the original. It will format to the size of the original drive, even if you put in a larger one. A search here will turn up the drive models people have used in the past.

Hi

oh ok, thanks for that.

I thought I'd read elsewhere (before finding this site) that you needed an installation CD that was only available to authorised dealers in order to format a new drive due to the fact that it contains a separate partition with firmware on it.

Confused.. :-/

dummy post

In fact, i just found it... http://www.avforums.com/forums/dvd-blu-ray-recorders-recording-media/825094-change-hdd-panasonic-dmr-ex85.html

Your link doesn't work:confused: and since your EX Panny is a European model it is possible it requires extra steps. The US Pannys only requires a compatible drive be installed and the firmware will format the drive correctly.
Pioneers in the US/Canada require a special service remote to replace the hdd but not the Pannys.

Your link doesn't work:confused: and since your EX Panny is a European model it is possible it requires extra steps. The US Pannys only requires a compatible drive be installed and the firmware will format the drive correctly.
Pioneers in the US/Canada require a special service remote to replace the hdd but not the Pannys.
His link worked for me.

I was very surprised, to say the least, when I read that thread he was pointing us to. It seems that such an HDD replacment procedure is not as simple as with E85, EH50, Eh55, EH75 models *I* am familiar with. It's odd that Panasonic would change their design and put the firmware on the HDD in these machines. It changes the process for replacing the hard drive from trivial, to nearly impossible. Why?

mbslk280, I am very sorry for misleading you in this. I saw your model, EX85EB, and never really mentally processed the "X" in the EX, so I was speaking of a different model completely. All my mind processed was the "85" and that was my mistake. They shouldn't re-use model numbers, it confuses people like me.:confused:

I bought some IPOD accessories as a Christmas gift this year, making the HUGE mistake of getting IPOD NANO Gen 4 items, instead of IPOD NANO Gen 5 items. They are incompatible apparently. Well, *I* didn't know!:rolleyes:

Hey Church AV Guy

mbslk280, I am very sorry for misleading you in this.

lol - no worries. Thanks for clarifying. :D

Yep, its most frustrating. They bought this model specifically 'cos it has a built in SD card reader and so can copy their photos to DVD (they dont have a PC).

The HDD itself (which I believe has a 3yr warranty) is dated 02 SEP 2006 :rolleyes: Ho hum...

Happy holidays :)

See my posts above in June 2009. I accidentally (re)formatted the 250GB hard drive on my Panasonic DMR-EX85EB thinking I was formatting a DVD-RAM.

The HDD contained 100 hours of treasures that I had not yet watched so after reading worley45 (wade)'s posts in this thread I was inspired to mount a project.

I took the HDD out of the Panny and connected it to my PC (with an IDE to USB adapter cable) and followed Wade's suggestions for using HxD (Hex editor) and HJSplit (file joiner/splitter) to retrieve my treasures and then burn them to DVD. I created a few tools to streamline the process and ended up with 25 DVDs containing all my treasures.

In the meantime, so that I was able to continue using my Panasonic DMR-EX85EB, I bought a replacement Western Digital Caviar WD2500BB 250GB 7200rpm IDE/PATA for £30. Before fitting this into my Panny I copied onto it the first 320KB from the old HDD (Panasonic's proprietary formatting code) and bingo on switching the Panny back on I was welcomed with the information that I had 111 hours of Standard Play recordable capacity.

So my message is that almost anything is possible.

If you want more details of anything described above or if you want a copy of the code to format a replacement 250 KB HDD post here and I'll respond.

im2020
Hi, I've stumbled across this site after trying to fix my sisters EX85EB. HDD was dead (clunking) so have fitted a replacement and it's asking for the service CD.

Would really appreciate the code to format so it works, and also confirming what file system I should format in?

Thanks, Mark

Hi Mark

I PM'ed im2020 around Christmas time inquiring about obtaining a copy of the code to format the HD but never received a reply.

If you fair better, or manage to get hold of the holy grail installation CD, please do let me know!

cheers
Mark

Hi Mark

I PM'ed im2020 around Christmas time inquiring about obtaining a copy of the code to format the HD but never received a reply.

If you fair better, or manage to get hold of the holy grail installation CD, please do let me know!

cheers
Mark
hi i am new to this group and i have been reading all the replys, i have a Panasonic DMREX-77 and the hard drive has packed up i have a spare hard drive but need the software to instal on the new hard drive, i have spent days looking for this, does anybody have a coppy i could have or tell me were i could get hold of a copy.
this machine is to good to just bin, i would be really greatful if anybody can help.
many thanks Diddywaxman.

Well Done Worley45!! Yesterday I pressed the wrong buttons and instead of deleting a DVDRAM, I deleted (not formatted) my nearly full hard drive on my DMR-HS2 !!! I found your post, and am starting the long process of trying to recover some precious recordings. I have succesfully replaced the hdd in the -hs2, so can continue using it. I have put the hdd directly in the pc, and it can see it as "hard disk 3", there are already 2 hdds in there, but cant play the saved "a1.mpg" file with any DVD player prog. It also is a lot bigger than 16Mb. Are there any more tips that might help?

I recently tried installing a 500G HDD to my Pana DMR-XW300 to replace an ageing 250G. Earlier posts suggested any larger HDD would be formatted to 250G. No luck for me, my system simply reported "Format Unsuccessful".

I then cloned the 250G onto the 500G using a program called Selfimage. I installed the 500G and it worked just fine. All recorded titles were available and remaining disk space was as per original 250G HDD. So far so good.

While the 500G was installed I deleted one recorded title. I then removed the 500G and reinstalled the 250G. The DMR wanted to do an automatic format as it could no longer access the titles. It seems the DMR records the title information (index) somewhere other than on the HDD. If the HDD does not match the recorded title information, the DMR tries to reformat the HDD, ie all titles are lost. This could explain many of the earlier automatic reformat comments in this post. Once you install a different HDD and do any reformat, even the original HDD will no longer work as there is a mismatch between the title records and the HDD.

I reinstalled the 500G HDD and because the HDD contents match the title records, this continues to work fine. I am now stuck with using this HDD as it is the only one that matches the title records.

Hope this info helps someone.

I recently tried installing a 500G HDD to my Pana DMR-XW300 to replace an ageing 250G. Earlier posts suggested any larger HDD would be formatted to 250G. No luck for me, my system simply reported "Format Unsuccessful".

I then cloned the 250G onto the 500G using a program called Selfimage. I installed the 500G and it worked just fine. All recorded titles were available and remaining disk space was as per original 250G HDD. So far so good.

While the 500G was installed I deleted one recorded title. I then removed the 500G and reinstalled the 250G. The DMR wanted to do an automatic format as it could no longer access the titles. It seems the DMR records the title information (index) somewhere other than on the HDD. If the HDD does not match the recorded title information, the DMR tries to reformat the HDD, ie all titles are lost. This could explain many of the earlier automatic reformat comments in this post. Once you install a different HDD and do any reformat, even the original HDD will no longer work as there is a mismatch between the title records and the HDD.

I reinstalled the 500G HDD and because the HDD contents match the title records, this continues to work fine. I am now stuck with using this HDD as it is the only one that matches the title records.

Hope this info helps someone.

You could try cloning the 500 back onto a new 250 if the software allows it. You only need the first 250 gig copied.

Hi there!

My uncle had a problem with his Panasonic DMR-EH56: The DVD-drive broke down and the whole recorder seems to be useless. But I wrote a script to copy most of the data from the hard drive disk. The data will be saved as MPEG files. So almost every application should be able to read them.

For this special Panasonic recorder the script worked really good. From the 160 GB (~ 149 GiB) hdd I could automatically recover about 70 GiB (this means, that the script found parts of the same recording itself and the film was complete without manual work). But I cut the files, so the size is without any advertisements.

The script also found the other data (so probably around 70 GiB), but it wasn’t able to recognize which of the pieces belong together. But I think, that many of this recordings were deleted, because most of them are very short. The script has also problems, if advertisements are cut out, because after this process the timecode is not continuous anymore. I don’t really know the content of the hard drive disk, because it wasn’t actually my recorder.

Now the problem: The script should work, but I can’t test it more than I already did. Also some parts of the code are far from perfect (and the very last thing is not completed yet), but my time is limited.

Is anyone here who has interest? It’s a Python script (tested with Python 2.6 ). I used it on a Linux based system with a copy of the hard drive disk (copied with dd). So I needed the space for the copy of the hdd and for the MPEG-output of my script (~ 2× size of hdd). But it should be possible to use the hdd directly as input (haven’t tested this, but it definitly requires root-priveleges.). I don’t know if it’s possible to use the script on Windows, too. If Windows can create a exact copy of the disk there is no problem. On Linux you can use the hard disk directly as input file (/dev/sdX), don’t know if this is also possible on Windows (I’m not a Windows user anymore).

It took me a while to write this script, so I would be happy if my solution could help others too. But be aware: Maybe it’s a bit complicated (especially for Windows users), but not as complicated as looking for the MPEG-chunks by hand ;)

If there are no interested people, further work would be unnecessary in my opinion. Then I would use my time for other things :) But would also be a pity if the (currently not fully completed) script would lie around on my hard drive.

Cheers

Hi there!

My uncle had a problem with his Panasonic DMR-EH56: The DVD-drive broke down and the whole recorder seems to be useless. But I wrote a script to copy most of the data from the hard drive disk. The data will be saved as MPEG files. So almost every application should be able to read them.

For this special Panasonic recorder the script worked really good. From the 160 GB (~ 149 GiB) hdd I could automatically recover about 70 GiB (this means, that the script found parts of the same recording itself and the film was complete without manual work). But I cut the files, so the size is without any advertisements.

The script also found the other data (so probably around 70 GiB), but it wasn’t able to recognize which of the pieces belong together. But I think, that many of this recordings were deleted, because most of them are very short. The script has also problems, if advertisements are cut out, because after this process the timecode is not continuous anymore. I don’t really know the content of the hard drive disk, because it wasn’t actually my recorder.

Now the problem: The script should work, but I can’t test it more than I already did. Also some parts of the code are far from perfect (and the very last thing is not completed yet), but my time is limited.

Is anyone here who has interest? It’s a Python script (tested with Python 2.6 ). I used it on a Linux based system with a copy of the hard drive disk (copied with dd). So I needed the space for the copy of the hdd and for the MPEG-output of my script (~ 2× size of hdd). But it should be possible to use the hdd directly as input (haven’t tested this, but it definitly requires root-priveleges.). I don’t know if it’s possible to use the script on Windows, too. If Windows can create a exact copy of the disk there is no problem. On Linux you can use the hard disk directly as input file (/dev/sdX), don’t know if this is also possible on Windows (I’m not a Windows user anymore).

It took me a while to write this script, so I would be happy if my solution could help others too. But be aware: Maybe it’s a bit complicated (especially for Windows users), but not as complicated as looking for the MPEG-chunks by hand ;)

If there are no interested people, further work would be unnecessary in my opinion. Then I would use my time for other things :) But would also be a pity if the (currently not fully completed) script would lie around on my hard drive.

Cheers

I'm definitely interested. Three potential problems:
1) I'm running WinXP (but I know how to run a knoppix disk)
2) My data is now stored as an image file
3) The data came from a HDD that required a format. Panny HDD formats are 'simple' in that they only overwrite the TOC (or whatever the correct linux term is) - the actual mpeg data is still there, just no direct pointers to what is what.

Let me know if you think it'd still be OK to use and what I'd need.

Wow, this was a fast response. :)
I'm definitely interested. Three potential problems:
1) I'm running WinXP (but I know how to run a knoppix disk)
As far as I can see the script should work also on Windows (Python interpreter required of course).

2) My data is now stored as an image file
This should make the process easier.

3) The data came from a HDD that required a format. Panny HDD formats are 'simple' in that they only overwrite the TOC (or whatever the correct linux term is) - the actual mpeg data is still there, just no direct pointers to what is what.Yes, I definitly found deleted recordings. But the script doesn’t touch the TOC of the filesystem. Reverse engineering the filesystem is a bit too hard for me ;) It looks for mpeg-headers and uses the timecode to split the chunks.
If some parts are too small (size below an adjustable value), the script will ignore them. I used a value of 50 MiB.

Let me know if you think it'd still be OK to use and what I'd need.
There should be no problem at all. The last feature of the script (dump chunks to file) is not ready yet. I’ve also found another bug in the code. Give me a few days.

Nice to hear that somebody has interest.

@0x1BBE898
wow, this is amazing. 2 days ago I found this thread. Today I sit in front of my computer to actually start creating the very script you're talking about. Just with luck I saw your post that you just posted yesterday. After 1 year of inactivity!

This said, I'm *very* interested in your script... I am running Windows, but I have a VirtualBox running Ubuntu, that should do. I've got an image of my HDD. It's split into 1GB chucks so there might be some work there.

Another drawback is that I'm not really familiar with Python, I was planning to write C++... but I'll jump into it right now.

Hope you find time to do the 'dump chunks to file' bit, although I understood that you had already recovered your drive?

I've got an image of my HDD. It's split into 1GB chucks so there might be some work there.The current script can’t handle this situation. But if I have the time, i will try to solve this. Shouldn’t be too hard...

Another drawback is that I'm not really familiar with Python, I was planning to write C++... but I'll jump into it right now.The first prototype I’ve written in C. But Python has much more power and there is no need to reinvent the wheel ;) The Python script is only insignificant slower than the C program.

Hope you find time to do the 'dump chunks to file' bit, although I understood that you had already recovered your drive?You’re right. The export feature was only a fast hack to quickly review the result.

Have a look at: http://sites.google.com/site/haliner/dvr-recover

A short introduction:

If you think your hdd image is important, first make a backup ;) There is absolutely no warranty.

Download and extract the script (or checkout the git-repo) and open a shell:
$ cd /path/to/script
$ python dvr-recover.py usage

This should print the usage information.

Now you can create a sample configuration file:
$ python dvr-recover.py sample_settings
Edit the settings file. (See usage for more information. The most import keys are 'hdd-filename', 'chunk-filename' and 'export-dir'.)

Create chunk list: (May take a long time! I used only a very small part of the whole hdd.)
If you abort this process, all gathered information are lost! The script saves the information only at the very end.
$ python dvr-recover.py create
[ 46.8%] 471420/1006929 blocks (15713.9 bl/s; 30.7 MiB/s): 6 chunks
[ 93.5%] 941768/1006929 blocks (15678.2 bl/s; 30.6 MiB/s): 6 chunks

Finished.
Read 1006929 of 1006929 blocks.
Found 7 chunks.
Took 64.08 seconds.
Average speed was 15714.0 blocks/s (30.7 MiB/s).

Show chunk information:
$ python dvr-recover.py show

Sort chunks:
$ python dvr-recover.py sort

Show chunk information again:
$ python dvr-recover.py show
--+--------------+--------------+--------------+--------------+-------------
| Block Start | Block Size | Clock Start | Clock End | Concatenate
--+--------------+--------------+--------------+--------------+-------------
0 | 391379 | 615548 | 250031969 | 402836926 | False
1 | 223188 | 168189 | 401478251 | 484558534 | False
2 | 107989 | 115197 | 546892241 | 592303341 | False
3 | 80992 | 26995 | 1085209725 | 1100156779 | False
4 | 53995 | 26995 | 1100157071 | 1115282776 | True
5 | 26998 | 26995 | 1115283069 | 1129877061 | True
6 | 0 | 26996 | 1129877208 | 1144930337 | True

Extract fourth chunk: (To <export-dir>/chunk_####.mpg)
$ python dvr-recover.py export 3
Current chunk: #3
Chunk start: 80992
Chunk size: 26995
0.95s (28436.33 blocks/s; 55.54 MiB/s).

Concatenate chunk: #4
[...]

Concatenate chunk: #6
Chunk start: 0
Chunk size: 26996
3.22s (8388.86 blocks/s; 16.38 MiB/s).

Extract all chunks:
$ python dvr-recover.py export
Current chunk: #0
Chunk start: 391379
Chunk size: 615548
42.22s (14579.92 blocks/s; 28.48 MiB/s).
[...]

Please let me know, if the script is working or not (and if you like it or not). If you have any additional questions, feel free to ask me!

This was only a very very short intro, hope it’s comprehensible. But now I need a break ;)

Have fun!

Thanks for this. Give me a few days or so to practice with Python, then I'll tackle this project and report back with results.

I must say 0x1BBE898, that's a terrific job you've done there!

As I said, my problem is I've got my 160GB split up in smaller files. They're actually 4GB (not 1GB as I stated before) or around that. They're always split on block borders (so each file starts with the mpeg code 00 00 01 BA).

So I guess I need to change your script to do the following:
-with every chunk: save an extra parameter that says in which file the chunk was found.
-Now I've got the advantage that I can run the script on the smaller files, and append the chunk data to the chunk file, rather then overwriting. So the script doesn't have to process the whole 160GB before the chunk data is written.
-Sorting the chunks doesn't change.
-exporting the chunks: always check in which file the chunk was found, then open that file and export from it.

Am I correct to say that these are the only changes that need to be made?

Thanks for the great script.

I guess you’re right. But I have another idea... There should be only a few changes necessary:

Instead of one input file we will use a list of those. The settings file will simply contain more than one „input-file“-entry. For every file in the list, the script runs os.stat(path) and so it’s easy to determine the size of each file.

Only one more simple change is necessary: The offset must be calculated with all previous file sizes in mind. But that’s quite easy :)

I think this solution also works with files which aren’t split on block borders.

Maybe till next week I have a working script for split files. Shouldn’t take too much time.

It was much more easier than I had expected. Only a small amount of time was necessary to implement handling of multiple input streams.

http://sites.google.com/site/haliner/dvr-recover

Additional info: The resulting mpeg streams can easily edited with tools like Avidemux (http://avidemux.sourceforge.net/). I removed unnecessary content at the beginning / end.

I'm afraid I'm lost already...
So I've installed Python 2.6 (ActiveState ActivePython). I've set Python in my path... When I use the GUI and Dos Shell, and run "dvr-recover.py sample_settings" I get the following error:

Traceback (most recent call last):
File "dvr-recover.py", line 779, in <module>
Main().run()
File "dvr-recover.py", line 771, in run
self.load_settings()
File "dvr-recover.py", line 388, in load_settings
f = open(self.settings_filename, 'r')
IOError: [Errno 2] No such file or directory: 'dvr-recover.conf'

So, is it expecting that I've already created a .conf file? Even when I create a dummy dvr-recover.conf with the following settings:

hdd-file=[L:\DMREH75recover2-img\DMREH75V.img]
chunk-file=[L:\chunk]
export-dir=[L:\recover]
blocksize=[2048]
min-chunk-size=[25600]
max-create-gap=[90000]
max-sort-gap=[90000]
I get the following error:

Traceback (most recent call last):
File "dvr-recover.py", line 624, in <module>
Main().run()
File "dvr-recover.py", line 616, in run
self.load_settings()
File "dvr-recover.py", line 251, in load_settings
self.blocksize = int(value)
ValueError: invalid literal for int() with base 10: '[2048]'

So, I've got to be doing something wrong. Any help?

I'm afraid I'm lost already...
So I've installed Python 2.6 (ActiveState ActivePython). I've set Python in my path...I have never heard about ActivePython. I recommend using the plain CPython implementation (2.5 or never, but not 3.x): http://python.org/download/ But that’s not the point for the error messages.

When I use the GUI and Dos Shell, and run "dvr-recover.py sample_settings" I get the following error:

Traceback (most recent call last):
File "dvr-recover.py", line 779, in <module>
Main().run()
File "dvr-recover.py", line 771, in run
self.load_settings()
File "dvr-recover.py", line 388, in load_settings
f = open(self.settings_filename, 'r')
IOError: [Errno 2] No such file or directory: 'dvr-recover.conf'

So, is it expecting that I've already created a .conf file?Aaargh. That’s a bug. The script tries to read the settings file at startup, but this is definitely wrong for the „sample_settings“-parameter.

Even when I create a dummy dvr-recover.conf with the following settings:

hdd-file=[L:\DMREH75recover2-img\DMREH75V.img]
chunk-file=[L:\chunk]
export-dir=[L:\recover]
blocksize=[2048]
min-chunk-size=[25600]
max-create-gap=[90000]
max-sort-gap=[90000] [...]

So, I've got to be doing something wrong. Any help?The square brackets are wrong. The documentaton is somewhat imprecise in this context. Should look like this: hdd-file=L:\DMREH75recover2-img\DMREH75V.img
chunk-file=L:\chunk

//edit: See http://sites.google.com/site/haliner/dvr-recover
I hope the square brackets in the usage message are now a bit clearer.

Thanks for your quick reply. I uninstalled ActivePython and installed CPython 2.6.5 . I still must have something wrong though. First, I can't "import" the dvr.recover.py script into the Python sys.path or whatever, I need to go to a DOS shell, change to the directory, then run the script from there, so I'm sure I've overlooked something. But I did edit my .conf file and removed all brackets. When I run "dvr-recover samples_settings", no errors are noted, and the command line comes up again. So I then run "dvr-recover create" and I get the following:

C:\Python26\Lib\site-packages\dvr-recover-0.2>dvr-recover create
Traceback (most recent call last):
File "C:\Python26\Lib\site-packages\dvr-recover-0.2\dvr-recover.py", line 779,
in <module>
Main().run()
File "C:\Python26\Lib\site-packages\dvr-recover-0.2\dvr-recover.py", line 773,
in run
func()
File "C:\Python26\Lib\site-packages\dvr-recover-0.2\dvr-recover.py", line 621,
in create
reader = FileReader(self.input_filenames)
File "C:\Python26\Lib\site-packages\dvr-recover-0.2\dvr-recover.py", line 245,
in __init__
part.size = os.stat(part.filename).st_size
WindowsError: [Error 3] The system cannot find the path specified: ''

Any ideas what is happening? All directories/files in my .conf file exist.

Thanks for your quick reply. I uninstalled ActivePython and installed CPython 2.6.5 . I still must have something wrong though. First, I can't "import" the dvr.recover.py script into the Python sys.path or whatever, I need to go to a DOS shell, change to the directory, then run the script from there, so I'm sure I've overlooked something.No, that’s exactly the way to go. It looks a bit uncomfortable first, but I think every Linux (well Unix, ...) user get’s addicted to it ;)
Importing is much more complicated.

But I did edit my .conf file and removed all brackets. When I run "dvr-recover samples_settings", no errors are noted, and the command line comes up again. So I then run "dvr-recover create" and I get the following:
[...]
Any ideas what is happening?You should run the script with the parameter „sample_settings“ and after that edit the settings file.

If you edit the file first and run „sample_settings“ afterwards, the script will overwrite your changes.

@bameije: Have forgotten to mention it, this small script helps to build a settings file for many input files:
#!/usr/bin/env python
# -*- coding: utf-8 -*-

f = open('hdd-files.txt', 'w')
for i in xrange(0,50):
print >>f, 'hdd-file=path-to-hdd-files/part_%03i.img' % i
f.close()It creates a file called „hdd-files.text“ containing the string „hdd-file=...“ multiple times. „%03i“ will be substituted with the number in variable i. The resulting string has at least 3 digits. The „0“ in front of the length-modifier will pad the value with zeros:hdd-filename=path-to-hdd-files/part_000.img
hdd-filename=path-to-hdd-files/part_001.img
hdd-filename=path-to-hdd-files/part_002.img

No, that’s exactly the way to go. It looks a bit uncomfortable first, but I think every Linux (well Unix, ...) user get’s addicted to it ;)
Importing is much more complicated.

You should run the script with the parameter „sample_settings“ and after that edit the settings file.

If you edit the file first and run „sample_settings“ afterwards, the script will overwrite your changes.

Hello,
Thanks again for that speedy response. You are correct, after I ran w/ the sample_settings switch, it overwrote my old .conf file and erased my directories, which is why it errored out. However, aarrrggh! I got 'create' to work, and after it reached 100%, it gave me this:

Finished.
Read 39092004 of 39092004 blocks.
Found 150 chunks.
Took 15404.80 seconds.
Average speed was 2537.7 blocks/s (5.0 MiB/s).
Traceback (most recent call last):
File "C:\Python26\Lib\site-packages\dvr-recover-0.2\dvr-recover.py", line 779,
in <module>
Main().run()
File "C:\Python26\Lib\site-packages\dvr-recover-0.2\dvr-recover.py", line 773,
in run
func()
File "C:\Python26\Lib\site-packages\dvr-recover-0.2\dvr-recover.py", line 626,
in create
self.save_chunk_list()
File "C:\Python26\Lib\site-packages\dvr-recover-0.2\dvr-recover.py", line 376,
in save_chunk_list
f = open(self.chunk_filename, 'w')
IOError: [Errno 13] Permission denied: 'L:\\chunk'

Notice the very last line, "L:\\chunk" I checked and double checked my .conf file and the entry is "L:\chunk" (one forward slash only).

EDIT:
'test-settings' showed this:
C:\Python26\Lib\site-packages\dvr-recover-0.2>dvr-recover test_settings
input-file: ['L:\\DMREH75recover2-img\\DMREH75V.img']
chunk-file: L:\chunk
export-dir: L:\recover
blocksize: 2048
min-chunk-size: 25600
max-create-gap: 90000
max-sort-gap: 90000

So it added an extra slash to my input file directory. But seeing as it found it anyway, seems like it should work, right? Weird that it has brackets around the input file (I did not enter those), but not on the chunk or export directory.

Oh, and is there a way that I can select just a portion of the .img to run this on so that I can check to make sure it works before doing the whole thing?

implement handling of multiple input streams.

Thanks! It works like a charm... I only tested it on 8 x 1GB test chunks. Can only unleach it on my real 40 x 4GB data in 2 weeks, but I'm sure it'll do just fine.


Oh, and is there a way that I can select just a portion of the .img to run this on so that I can check to make sure it works before doing the whole thing?
Yes. One of many ways to do it: just open your .img in HxD and save only the first 1GB in another file (eg .img.part). Use this file in your dvr-recover.conf file. The script should find a few chunks in this partial file as well...
Good luck!

I got 'create' to work, and after it reached 100%, it gave me this: [...]
Notice the very last line, "L:\\chunk" I checked and double checked my .conf file and the entry is "L:\chunk" (one forward slash only).That’s a pity. After this long time an error...

In Python “\” is an escape character. So internally a single “\” in real world will be handled as something like “\\”. That’s not a problem. So if you specified the path in the settings file correctly, you’re fine. (And even if, Windows is also able to handle “\\” in paths correctly.)

There must be another problem. Probably your user has no permissions to write to L:\. You should check this.

The script should try to open the chunk file before scanning the input files. This way the script crashes before and not after the time-consuming process. A bit quick and dirty, but better than nothing at all. (*)

It would be fantastic if the script could save it’s current state to a file. So a later resuming without losing information would be possible. Never implemented something like that, but it’s a nice playground. Some ideas are growing in my head... ;)

Weird that it has brackets around the input file (I did not enter those), but not on the chunk or export directory.Also a minor cosmetic mistake ;) I order Python to print the list of all filenames. And Python is using square brackets for lists. (*)

Oh, and is there a way that I can select just a portion of the .img to run this on so that I can check to make sure it works before doing the whole thing?No, but see bameije’s post. (You can get HxD from here (http://mh-nexus.de/en/programs.php).)

@bameije: Ok, great :)

(*) = Fixed in current release, see http://sites.google.com/site/haliner/dvr-recover

//edit: Tested the current release on Windows. Everything went smoothly.

OK, I don't know how it happened, but I got it to work. I ended up retyping in my directory values... I think I may have had an extra space after my chunk file value in sample_settings. I then followed the remaining steps using a 1GB portion of my image and it found and exported a part of a much larger file from the HDD, so I've got it working. Now I just need to run it again for the 4 hours and hope for no other problems. I'll report back when finished. Thanks again for the help from both of you!

Well, that worked quite well. Thank you muchly for such a great tool that's an insane timesaver. I was able to recover 45 files, about 67 GB total. I can't remember how full the drive was, but that seems to be everything. There was only a couple small hitches. One .mpg is nearly unplayable. It starts, but 10 secs in or so and it causes the player to shutdown. I've tried it on WMP, WMP Classic, Player, PowerDVD, etc. (will try it on VLC later). It won't even play on VideoReDo, which will play and fix pretty much any .mpg frame/header error, so I don't hold much hope on recovery. The other problem I had was that the program concatenated 2 different files. It combined the beginning of one file with the end of another. Then it saved the end of the first file and the beginning of the other separately from chunks later down the list. Other than that, if worked perfectly! It even had the bits intact that I originally trimmed from the recordings.

One question. The smallest file it found was ~ 52 MB (0:50 seconds or so), as per the settings file. I don't recall exactly, but I may have had snippets smaller than that. If so, is there any way to re-run this program with the smaller chunk size to find ONLY files smaller than that 50MB limit and save only those to a chunk file to export again?

Thanks again for such a great tool!

Panasonic have proved themselves leaders in the field of Freeview DVD Recorders with integral hard disk technology. Their latest products include the DMREX88, with it's massive 400GB disk and the DMREX768, a replacement of the popular DMREX77 DVD recorder with a 160GB HDD.

In this review we take a look at the DMR-EX77 and DMR-EX87 DVD recorders, both of which incorporate a Freeview compatible DVB-T tuner and large capacity , together with providing an HDMI interface with up-scaling to full HD (1080p / 1080i). In keeping with other Panasonic DVD recorder models, both offer exceptional picture quality when connected to an HD ready flat screen TV.

^^You do know that flaunting Panasonic's non US based models can cause grown men to weep, don't ya?

I released dvr-recover 0.5: http://sites.google.com/site/haliner/dvr-recover

The script uses a sqlite database now. It is now possible to interrupt the process of gathering all the chunk info. The script will continue the process automatically.

You can find documentation on the project page and in the source tarball or zipfile. Ask whenever you have a question or something is unclear.

I think I need not to say that feedback is very welcome ;)

Regards,
Stefan

I have a DMR-XW300 that the DVD drive has packed up on - been using the scripts to try and extract some video and getting nowhere fast. REtrieved 4 video's but then the script crashes.

In desperation I downloaded the drive using dd to my mac (all 250GB of it) into a disk image file, and ran ISOBuster over it - its' found just about every video file and I'm in the process of extracting them now....

In desperation I downloaded the drive using dd to my mac (all 250GB of it) into a disk image file, and ran ISOBuster over it - its' found just about every video file and I'm in the process of extracting them now....Interesting. I’ve forgotten to check if IsoBuster can open the image. Unfortunately I removed the image of the hdd, because it wasted my disk space. So I can’t test it now.

Maybe Panasonic uses some kind of UDF as file system. According to Wikipedia a block size of 2048 byte is typical for the UDF file system. In the most common case UDF is used as a read-only file system on CDs and DVDs. But it is also designed to be writable. A UDF like file system would explain why IsoBuster is able to open the image.

I would be interested in a screenshot of IsoBuster showing the content of the image (filenames and directory structure). Anyone who can provide one?

Thanks in advance :)

IsoBuster can open the image (like some other image programs). But also like those other programs, it just finds a series of mpegs (it labels them as .VOB's) and some very large .IFO files that must also be mpeg video. On my image, aside from 5 small files, it separated the 80 GB image into 80+ ~1 GB files. I don't have a license, so I couldn't test it beyond that, but it seems it's the same as all the others; it'll copy over all the video files, but it won't concatenate them.

I burn UDF DVD's when I have files larger than 2GB. I don't know why recorders would need to create a file that large, as it appears they create 1GB chunks. Who knows?

its' definitely using UDF of some description for the storage - it kind of makes sense - then it doesn't matter if you are writing the file to DVD-RAM or HDD it would be treated the same.
However I also think they are using some other file system for managing the title list -

ISOBuster could open the image file, but only after you force it to as it doesnt' think it's a UDF file at the start (there's a LOT of padding at the beginning of the image as well as some references to another file system type) and definitely recovered a lot of files.
However I found it corrupted the output files with various streams overwriting each other ans starting half way into each other.

I have another patched version of the python scripts that has allowed me to extract many of the files, however its' still missing some key files so my journey to recover this thing still continues.

I really don’t know if the recorder is using UDF or any similiar file system. I only come accross the fact that the block size of UDF is 2048 byte (like the one of the hdd) by chance. I would make sense, because the recorder doesn’t have to convert between file systems and file types.

As far as my script is useful for someone I’ll try to maintain it, even if IsoBuster would open the image accordingly.

The firmware of the recorder is also located at the beginning of the disk. So it’s really suprising that IsoBuster can open the image.

I fixed a bug claiming that there are multiple references to a chunk. You can find the new release (the one mentioned by gsouth + a minor source code cleanup) like all other releases on the homepage: http://sites.google.com/site/haliner/dvr-recover

Personally I think that the default value for “min_chunk_size” is quite too high. So if someone can prove that a lower value is better, I’ll change the default. See section “Tuning the parameters” on the project page.

As far as my script is useful for someone I’ll try to maintain it, even if IsoBuster would open the image accordingly.

Definitely do not stop working on/hosting this program. As I stated, most ISO/data recovery programs will find the mpeg data, but that's only half the desired result. None of the ones I have tried will even remotely reassemble the data into the original watchable file. To do what your program does, one would need an ISO/recovery program, an editing program (like VideoReDo) AND countless hours. But if someone wants to reinvent the wheel, I guess they could try.


Personally I think that the default value for “min_chunk_size” is quite too high. So if someone can prove that a lower value is better, I’ll change the default. See section “Tuning the parameters” on the project page.

Well, for someone like me, it may be a bit large. 50MB translates to ~ 50 secs @ XP, 100 secs @ SP and so on. I recorded little snippets of kids programs off Nick JR for my daughter, so I'm guessing some of those were missed. Also, perhaps if one has a drive that at one point had been nearly full, then when recording a long program that doesn't have all the required contiguous space, it could potentially place < 50 MB portions of the recording throughout the drive, which could be skipped. I have no idea what a good new minimum chunk size would be though.

As I stated above, it’s not my intention to give up development. The topic is also quite interesting for me and hosting doesn’t cost me anything.

The 50MB limit is taken over from an older version of the program (which worked differently).

I have another idea: Why limit the size while gathering the data? The script should analyze the disk completely and save everything. A limit while exporting whole recordings makes sense, because very short recordings are unusual. But the limit shouldn’t be too high either, because it can be just a part and the program hasn’t detected this correctly.

I expect that the results will be much better and more data can be restored.

//edit:
For those who wants to experiment: python dvr-recover.py setup min_chunk_size 0After that go through the whole process again (create, sort, export).

No chunks will be ignored after running the command above. Hopefully more files can be recovered.

I’ll implement my idea in some time. But setting min_chunk_size to 0 is almost identical. (Only the small files are not ignored during the export.)

Hi,
ISOBuster can read teh file as it simply scans through it looking for UFD fragments and saves them - it works ok however as it has non knowledge of the underlying organisation I hve found it corrupts more files thatn it recovers.

I tried with min_chunk_size at 1000 last night and did get more files back but still am missing around 20-30%.

Going to look at the file structure more later this week - if I can extract an intact UFD file form the overall disk, put it back onto a blank drive then I should be able to mount it and read it......

All,

I have been watching this thread with must interest. The DVD-Drive on my Panny (DVR-EH55) died a while ago (and before people suggest cleaning etc..., done all that). The HDD is fine, I can watch the programs etc etc, just can't burn them to DVD, and there is stuff I want/need to keep (e.g copys from VHS tapes)

The quote for repair was AUS$250, which I figured wasn't worth it (thats half the cost of the latest twin-HD DVD/HDD recorder). Well, being an IT person, I figured, there HAS to be a way of pulling the data from the hard drive. Also, the HDD was just about full with many-many programs (my wife had done a lot of recording, and cutting out 3-5 min segments from various shows for me to watch).

Anyway, the picture I want to paint is that I want to be sure that I get ALL of the data off the HDD.

Following along the lines of what some others here have done, I have started examining the HDD via a hexeditor (I use Winhex) in an attempt to try to decipher the file system being used (There has to be some way that the Panasonic can piece together the fragments for a given show, and to provide a listing of all recorded programs etc etc)

So anyway, I did some scanning and started getting hits on programs I knew I recorded. These hits looked like Meta-data - Name of the program, date recorded, ...

Well, one of these strings "dvdvrx", kept coming up. I thought, oooh, maybe this is attributes (in a Unix way) for the files. So I did as anyone does these days, I googled the string.

Immediate success. I went to this page - freepatentsonline dot com Slash EP1091577 html. I can't post urls, as am a newbie to this forum, so hopefully you can work outthe link.

This is a Patent from MATSUSHITA (which is also Panasonic). I have downloaded the attached PDF and am currently reading it. All of a sudden the data in HDD is coming to life :-)

I still need to do a lot of investigation, but wanted/needed to share this information, in the hope that Stefan's dvd-recover could be tweaked to pass through the file directory. I don't know python so not really up to offer coding suggestions for it.

REgards

David

Hello Guys

Set out below is a copy of a post I submitted last July. It seems there were one or two requests for info and/or code around Christmas time but I wasn't alerted to them (think email notification was turned off) and have only just signed in for the first time this year.

So post again if anybody needs help on a situation similar to mine and I'll respond.

IM

============================

Copy of post on 12 July 2009

See my posts above in June 2009. I accidentally (re)formatted the 250GB hard drive on my Panasonic DMR-EX85EB thinking I was formatting a DVD-RAM.

The HDD contained 100 hours of treasures that I had not yet watched so after reading worley45 (wade)'s posts in this thread I was inspired to mount a project.

I took the HDD out of the Panny and connected it to my PC (with an IDE to USB adapter cable) and followed Wade's suggestions for using HxD (Hex editor) and HJSplit (file joiner/splitter) to retrieve my treasures and then burn them to DVD. I created a few tools to streamline the process and ended up with 25 DVDs containing all my treasures.

In the meantime, so that I was able to continue using my Panasonic DMR-EX85EB, I bought a replacement Western Digital Caviar WD2500BB 250GB 7200rpm IDE/PATA for £30. Before fitting this into my Panny I copied onto it the first 320KB from the old HDD (Panasonic's proprietary formatting code) and bingo on switching the Panny back on I was welcomed with the information that I had 111 hours of Standard Play recordable capacity.

So my message is that almost anything is possible.

If you want more details of anything described above or if you want a copy of the code to format a replacement 250 KB HDD post here and I'll respond.

im2020

Hello Guys

Set out below is a copy of a post I submitted last July. It seems there were one or two requests for info and/or code around Christmas time but I wasn't alerted to them (think email notification was turned off) and have only just signed in for the first time this year.

So post again if anybody needs help on a situation similar to mine and I'll respond.

IM

============================

Copy of post on 12 July 2009

See my posts above in June 2009. I accidentally (re)formatted the 250GB hard drive on my Panasonic DMR-EX85EB thinking I was formatting a DVD-RAM.

The HDD contained 100 hours of treasures that I had not yet watched so after reading worley45 (wade)'s posts in this thread I was inspired to mount a project.

I took the HDD out of the Panny and connected it to my PC (with an IDE to USB adapter cable) and followed Wade's suggestions for using HxD (Hex editor) and HJSplit (file joiner/splitter) to retrieve my treasures and then burn them to DVD. I created a few tools to streamline the process and ended up with 25 DVDs containing all my treasures.

In the meantime, so that I was able to continue using my Panasonic DMR-EX85EB, I bought a replacement Western Digital Caviar WD2500BB 250GB 7200rpm IDE/PATA for £30. Before fitting this into my Panny I copied onto it the first 320KB from the old HDD (Panasonic's proprietary formatting code) and bingo on switching the Panny back on I was welcomed with the information that I had 111 hours of Standard Play recordable capacity.

So my message is that almost anything is possible.

If you want more details of anything described above or if you want a copy of the code to format a replacement 250 KB HDD post here and I'll respond.

im2020


Hi im2020,
Having encountered same problem with my pana, I shall be glad to receive copy of the code as you mentioned above to my mail address. I'm not 100% sure this will work since my apparatus was bought in France but at least I should have tried!

As soon as received, I'll keep you informed of the result

Best regards

Alainl4462

Hi

Yes I have the same problem with my EX-75 would like to try the code if you can mail it to me as well? I have a 250GB hard drive, its worth a go before I prob have to fork out for the disc, been a disappointing unit this one, the 81 errors all the time, they seemed to go away and then there were the colours fading in and out on channel 4 (I know there is a firmware fix for it now) and now the HDD has gone and I have lost all my programs.

I dont think I will buy another panny after this unit :mad:I know they dont make the hard disc but making the unit so you cant simply but another drive straight in like most other manufacturers is so annoying, hard disc drives always go :mad:.

Anyway if you could message me and mail me the code, its worth a try.

Knoxie

Hello Alain and Hello Knoxie

I will email you the code but can I just check that when you both say you have the same problem as I had that you're both truly on the same wavelength:

1. You are/will be taking out your original 250GB HDD from your Panny and replacing it with a newly bought 250GB HDD

or, not like me, you have nothing special on your Panny HDD but would like to re-install the firmware to see if it will start working from scratch again.

2. In either case can you confirm you are able, using Worly's original instructions, to connect the new (or removed) HDD to your PC
All you need is a decent hex editor program ( I use HxD ) and a way to connect the the HD to your PC ( An IDE to USB adapter cable is what I use ). It doesn't matter if Windows can't see the drive, a hex editor can retrieve the binary data from any storage device connected to your PC!

If you can't install an IDE drive into your PC, you will need an IDE to USB adapter cable with power supply and ac cord ( available on eBay or websites ).
Go to these websites and download & install :
http://mh-nexus.de/hxd/ HxD 1.7.1.0 beta (Hex editor)


Carefully unplug the ribbon cable and power plug and remove the hard drive from the Panny.
Move the jumper from J48 to J50 ( CS to Master ) and remove the horizontal jumper connecting J44 to J42 ( Maxtor HDs have a diagram on top ). Place jumper and screws in a small zip-loc bag or container where they won't get lost! Romove the two screws near the connecters holding the drive to the metal frame, loosen slightly ( but do not remove ) the two rear screws. Lift the front of the drive just enough so the screw holes are above the frame, then put those screws back in just enough so the screws will rest on top of the frame and hold the drive up at a slight angle ( so you can get the connectors in ). Plug in the power and data connectors, then plug in the power cord. When the drive spins up, plug the USB cable into your PC. Don't let your OS try to format the drive in case it asks for this.

3. If you're set up for this you can use HxD to copy the code I'll send you to the new (or original) HDD before installing/re-installing in your Panny.

Let me know.

Regards

IM

Hi Ian

Many thanks for the reply, I checked out the post on the forum, the problem that I have is that the hard disc is not working, clunking all the time the usual sound of hard drive death I have unplugged it and connected it to my PC but the bios wont even see it, I have seen so many hard drives do this.

I have contacted Panasonic and they directed me to the local service centre, they have told me its 20 quid to just look at the unit then whatever the costs are after that! this is ridiculous and if you take the cost of the hard disc in to account and their fee it would almost make more sense to buy another unit.

If you could send me the code that would at least be something that I could try first, my unit is the DMREX-75 not the 85 but I will be running the code on a 250GB drive, it may work it may not but as I already have the drive and I know the drive is good its something that will cost me nothing and no time to at least try before I go ahead and pay through the nose.

Panasonic are sending me the updated firmware disc though? there is a colour shifting problem on channel-4 which you may have seen? they offer a firmware update disc for free for that! just call them and they will send it to you, why cant they offer the hard drive format CD as well? its no different to you swapping out the hard disc in your PC?

Thanks again for offering the code I will let you know if it works.

Thanks

Paul

Paul

Download the code from www.im2020.com/Panny/. Unzipped it's a 355MB file called firmware.txt.

Using HxD go to the Extras Tab, Open Disk and Open the Physical Disk that you want to copy the code to. Make sure it's the right one. In the contents the last offset for a 250GB disk should be 1BF08EAFF0. The default for HxD is to Open as ReadOnly so unclick this (when you're sure you're working on the right one).

Open the firmware.txt file and copy and paste the contents to your physical disk starting at Offset 0. The limitations of your clipboard capacity will mean you'll have to copy in 2 blocks, the first, say, from offset 0 to FFFFFFF and the second from 10000000 to 163EDFFF. The Edit, Select Block tab helps you to set these. Finish by double-checking that the contents of 163EDFF0 to 163EDFFF on your physical disk are the same as in firmware.txt then close all files.

Good Luck.

:)

Thanks so much for doing that! I will try this later when I get home, its worth a try at least.

Knoxie

:):):):):):)

Hey ian!! yep it worked!! your a star thanks so much for the code, I got in from work with the code and the hex editor on my memory stick, quickly installed the hex editor with no problems.

I took your advice and checked that I was selecting the right drive, I did this simply by looking at the amount of sectors that were showing in the hex editor, I used the block select function to select the chunks of data as the clipboard is limited as you said.

I pasted the first lump of code in there and went back and pasted the second block in there, I then checked that they matched each other which they did. I saved the data on to the drive and shut the machine down.

I returned the new 250GB drive to the DMR, i just layed it on the side in case I needed to do it again, I powered up the machine with a little trepidation I must say! the unit said hello, then on screen it said the hard disc was not formatted would I like to format? OK then hit format and away it went.

Bingo! then it came in 74 hours!! so it wont let you use the full 250GB (My unit is a DMR-75) it only formats 160GB but thats not a problem this must be fixed in hardware in the unit, not a problem the hard disc was just what I had laying about anyway.

So pleased it is working, I had no joy with Panasonic they refered me to the local dealer who were going to charge me a 20 quid inspection fee? before they do anything so it would have been at least 100 quid with the drive as they wouldnt fit my drive anyway.

BIG BIG Thanks for taking the time to do this, your code works!! took me about 30 mins to do it, please let me know your paypal e-mail address on e-mail and I will send you a drink for doing this, I suggest anyone else reading this may want to do the same if they also get it working.

I tried everything to get the proper disc but to no avail, this fix cost me nothing and I have got my unit back, after speaking to Panasonic they are also going to send me the new firmware on a disc, this will fix that annoying colour fade problem on channel-4.

The U80 error has also stopped, I think this was because of over the air transmissions affecting the unit, so happy days again.

So if anyone else is trying it follow this post and its pretty easy to do, just make sure that you select the right drive in the hex editor, never assume drive-1 is your c-drive! check the sectors!!

Good luck

Knoxie

I have the same problem with my Panasonic DMR-E85H. Accidently hit the format button thinking I was formating a DVD, but instead did the HDD. I tried to turn it off immediately, but now it says I need to format the HDD. The display shows that I have 55hrs remaining of free space, so it looks like my files are there because thats about right. However they don't show up when I press the Direct Navigator. I am not that computer savvy to go through the steps you did...any suggestions? Do you think a Panasonic service place would know how to do all that? or a computer repair place? HELP! I've lost 7 months of programming!

Hi All,

simply use ***.diskinternals.com/download/

Linux Recovery 2.7
Freeware Linux Ext2/Ext3 Recovery for Windows!

to recover a DMR-EH65 HD(250GB) connected to Win7 PC System.

Hope this make things easier ;)

BR elfeuz.k

Paul

Download the code from . Unzipped it's a 355MB file called firmware.txt.

Using HxD go to the Extras Tab, Open Disk and Open the Physical Disk that you want to copy the code to. Make sure it's the right one. In the contents the last offset for a 250GB disk should be 1BF08EAFF0. The default for HxD is to Open as ReadOnly so unclick this (when you're sure you're working on the right one).

Open the firmware.txt file and copy and paste the contents to your physical disk starting at Offset 0. The limitations of your clipboard capacity will mean you'll have to copy in 2 blocks, the first, say, from offset 0 to FFFFFFF and the second from 10000000 to 163EDFFF. The Edit, Select Block tab helps you to set these. Finish by double-checking that the contents of 163EDFF0 to 163EDFFF on your physical disk are the same as in firmware.txt then close all files.

Good Luck.

Dear IM2020,

Can you please advise if you have same code for Panasonic DMR-EH65.

Thank You,
Kirill

Thank you everyone for posting up the information and the firmware text. It worked like a charm!

Thanks to Stefan and this thread (Stefan inside!)
I recovered accidentaly formated HDD in Panasonic EH-57.
(and save all cam-footages of my little daughter!)
As computer half-illiterate I have a few words for
similar fellows.
(-old Toshiba laptop with XP-)

Don't forget to move jumpers when you connect HDD to PC.
(I used old Icy Box and copied jumper positions from HDD inside),
also -jumpers back when you're finished.
Don't waste your time with numbers of Linux recovery programs!
(Scared of DVR-recover guide and manualy entered prompts,
I tried with 5 Linux recovery programs, and did nothing-
every scan running 3-4 hours,
only Disk Internals made some totaly messed mpgs)

So- DVR-recover was last hope (and real thing!)
Just be patient and read the guide carefuly!
I was stuck at the same beggining:
1. exact copy of HDD?
I used some already installed Linux recovery program
which gave me options of exact or compressed copy.
That's all what I know about it.
2. Python interpreter?
Google didn't help my with that,
so I installed official Python for Windows.
But DVR-recover was dead.
Next, I used Pythons help to prepare it for Windows XP.
The only thing I had to do was setting enviroment variables,
(easy step, but you have to find it in help)
and then command prompt -python- works.

Than read DVR-recover guide and do your work!
It's really easy.
Luckily, I remember few DOS commands from decades ago,
(and use only -cd-)
First I used defaults and go. I get all my movies in mpgs, but...
Lots of short movies missed the beggining.
Than I set min_chunk_size to 1600 and that's it.
From about 200 mpgs just a few long ones was cutted in 2-3 pieces.
And also, all mpgs had excesses only a second or two at the beggining and
the end which was cutted in Panasonic. It seems that Panny cut mpgs
at key-frames like avis, but reads them correctly like they are cutted at frames.
(Sure, I can be totaly wrong)
However, Stefan said that smaller the chunk size, more junk you'll find
and also more completely recover your mpgs.

I was so happy (and proud) about success that it was like a fun
to trim and sort my treasure for a few days.
I used VideoReDo Plus for trimming...

Now, my HDD in Panny is empty, movies are saved
and there's no more making room in last second before
important recording, heh.

So, good luck

Based on my understanding, members who ever tried video files recovery from HDD in Panasonic dvd recorders reported success but it involved lots of effort and time.

For those who are not able to recover data from failed Panasonic dvd recorders or do not have time to do it and are willing to pay, I found a link for a company in UK for Panasonic & Pioneer models claiming that they developed software that can read files on HDD and transfer these files to another HDD that would be viewable by PC. Here is the link:

http://www.apextechnology.co.uk/reco...o-recovery.asp

There is also a Czech-Korean company in the Czech Republic that charges between $25-$250 for this type of recovery (depends on the file system used and a conversion currency between USD and CZK).

I know that these HDD files recovery prices are similar to a new dvd recorders, so these services are not for everyone. They can be helpful for people who have valuable recordings (family movies, etc.) on their dvd recorder HDD and really want them to be recovered.

Good catch on finding this. For some, it just might be worth the cost.

I have a quick alternative method (to original post by Wade) which restored & recovered my old hdd in another panasonic machine with all the recorded titles showing as normal in the navigator. Done in approx less than 3 mins.

I have spent a total of 4 hours researching the internet today (this afternoon), and was only able to find two worthy solutions.

One was this post by Wade, and the other post is by Violetfi at..(near end of page) at afterdawn(dot)com
I was very impressed by Wade's solution but it is too much for me as I don't know anything about bytes etc. Violetfi also mentions a process in WinHex which I downloaded, but honestly, I have no idea what all those numbers mean and I realised, this was way out of my league, and I got out quickly before I bit off more than I could chew or fix!!!

I was impressed by Violetfi being able to use the one panasonic machine for 3 hdds, and the information he/she provided about the panasonics making a unique code on each hdd, thereby knowing that you have a different hdd in it, requiring reformatting...before using.

I prayed to (the Creator) God, Jesus, for a solution. And this is the idea that I received:
Make the panasonic put it's unique code on the hard drive that you need recovered, that way you'll have access to it, (it does this by formatting) but as soon as it's done that, don't let it have any more access to the hard drive so it can't do the next step and that is to wipe out all recorded titles on it (you do this by removing the power to hard drive). It does this whole process to format and wipe clean in 1 minute, so that means formatting is only going to take seconds.

DISCLAIMER:
Enter at your own risk. I don't know anything about bytes, rams etc. I just followed the answer to my prayer and it worked for me. I am not responsible for what you might lose off your hdd. Okay here goes...

1) With the machine off, connect the old hdd into the new machine. Make sure it is not fastened down by any screws, and that you have easy access to be able to pull out the power cord from the hdd, QUICKLY. This is not the power cord to the panasonic. It is the power cord to the actual hard drive. If you have a IDE hard drive, it will be the white end cord that attaches to the hard drive.
2) Turn on the Panasonic as usual.
3) When the prompt to format comes up, You will press enter or ok on your remote.
4) And press again to start - BUT be ready to RIP OUT the power cord to the hard drive!!!! I counted about 4 secs...then ripped out the power cord..ONLY!!! Someone suggested to try to remove it as straight as possible. (I think the objective here is SPEED!!).
5) A message prompted - I can't remember what it said. Something about turning off the machine and when it returns to standby mode, restart or something. Follow the instructions on the screen but don't plug the power cord back in yet.
6) When the machine is off (not unplugged from wall - you know like with the timer still on), connect the hard drive power cord. If you are not comfortable with this. unplug all power etc and then connect the hard drive.
7) Connect panasonic in the usual way and turn on machine.
8) It should power up in with the hard drive formatted but all the titles there in the usual looking way. And remote has full access to use it to edit and copy to disc as you were previously able to do.

If this works for you, the thanks & praise goes to my God, Jesus.

what page is this in the forum??

what page is this in the forum??

http://forums.afterdawn.com/thread_view.cfm/143317#834940

http://forums.afterdawn.com/thread_view.cfm/57/143317#2976112

thanks T

Sorry, but I wasn't allowed to put links in my post. That was my first post (minimum of 3 posts required before links allowed). Thanks for helping me out on that one.

I encountered a problem with this DVD recorder and needed to recover the videos off the hard drive. Using worley45's method, I wrote a program to search for the VOB pattern from an image of the disk and save them clips as mpg. It definitely beats doing it manually.

His link worked for me.

I was very surprised, to say the least, when I read that thread he was pointing us to. It seems that such an HDD replacment procedure is not as simple as with E85, EH50, Eh55, EH75 models *I* am familiar with. It's odd that Panasonic would change their design and put the firmware on the HDD in these machines. It changes the process for replacing the hard drive from trivial, to nearly impossible. Why?

mbslk280, I am very sorry for misleading you in this. I saw your model, EX85EB, and never really mentally processed the "X" in the EX, so I was speaking of a different model completely. All my mind processed was the "85" and that was my mistake. They shouldn't re-use model numbers, it confuses people like me.:confused:

I bought some IPOD accessories as a Christmas gift this year, making the HUGE mistake of getting IPOD NANO Gen 4 items, instead of IPOD NANO Gen 5 items. They are incompatible apparently. Well, *I* didn't know!:rolleyes:

Hi church Av Guy.

Not sure if you still follow this post. You mension in your post you have experience with EH55 drive replacements, I have the need to replace my EH55 hard drive but not coming right - new firmware running and formatted drive in Windows machine but still not being detected. Not sure what format drive should be. Could you maybe shed some light on the process you know of to replace the drive step by step?

thanks

I formatted my hdd on my dmre80h today accidently . Please help, I have been reading this thread today and see some hope . Problem is that I have very little computer knowledge. The only good thing I can think of at this point is that as soon as I realized what I had stupidly done, I unplugged the unit and placed it in a corner. What would you do if you were in my place? I will pay one of you if that"s what it takes. I am located near chicago il. Any possible easy solutions would be greatly appreciated. I do have a couple of friends that are ver computer savvy, but have never attempted this type of recovery. Thanks in advance . joe :o

Members here usually try to restore files from HDD when HDD suddenly fails in the dvd recorder. I am not sure whether their techniques are applicable to a situation when HDD is accidently formatted.

You can find one of my previous posts (look above) for companies who specialize in files recovery from failed HDD; there is a link to a company in UK that does it. You might try to contact them to find out if their services is applicable for your situation and start there. Be aware that these services are usually expensive, so you need to know how valuable are your recordings and how much you are willing to pay for their recovery.

I formatted my hdd on my dmre80h today accidently . Please help, I have been reading this thread today and see some hope . Problem is that I have very little computer knowledge. The only good thing I can think of at this point is that as soon as I realized what I had stupidly done, I unplugged the unit and placed it in a corner. What would you do if you were in my place? I will pay one of you if that"s what it takes. I am located near chicago il. Any possible easy solutions would be greatly appreciated. I do have a couple of friends that are ver computer savvy, but have never attempted this type of recovery. Thanks in advance . joe :o

You should be fine, just don't use the unit and pull out the HDD. Panasonic DVDR formatting is generally a 'quick format' where only the MFT (or whatever it's called that lists where the files are located) is erased and the data itself is not touched- so it's all still on there. Same thing happened to me when I needed to have PCB in my DMR-EH75V replaced after it was broken in shipping, they had to reformat the HDD when the unit was first booted up. You'll need a spare HDD of roughly 2x the size of the original to move files around. What you'll get is a bunch of .mpg files that you can then author with a DVD program and burn to a disc. Not quite as easy as using the unit, but at least you'll have the data. I think others have posted they were able to image the HDD and re-install it in the unit and have the unit recognize all the recordings as normal, but I didn't try that. I can assure you that you do not want to use those file recovery programs. They work, but will not accomplish what you want. They'll give you the small chunks that make up your normal program (as one continuous program may be spread out over multiple parts of the HDD), sometimes mashing up two different programs within one file. Use the instructions posted in the links listed in this message. If you still can't get it, PM me.

Members here usually try to restore files from HDD when HDD suddenly fails in the dvd recorder. I am not sure whether their techniques are applicable to a situation when HDD is accidently formatted.

Yes, they are applicable. I just used Stefan Haller's excellent dvr-recover program to recover of the titles (200+ hours) after an accidental HDD format. Sure, it requires knowledge of Linux and the ability to take apart a computer. But it's better than nothing!

As a newbie I can't post the link for "dvr-recover" so Google it.

P.S. Stefan, you the Man! :D

Could you possibly reach me at 847 254 7130 ? Thanks so much for the hope I have been looking for. Joe

There is now someone (here?) on eBay selling a service (http://www.ebay.com/itm/280745058949) to restore videos from a Panasonic DVD Recorder hard drive, starting at $200.