DCT6412 Hard Drive Upgrade? - Page 3

So, does it work after all? I appreciate you wrote:


but I guess that was just the cloning part, whereas I'm interested to find out if the cloned disk worked inside the PVR.

If it worked I can see a method by which somebody that has an eSATA-enabled Motorola DVR can help anybody else to format his/her bigger disc(s):
1. get a new (empty) bigger SATA disc (e.g. 320GB, 500GB, 640GB or 1TB)
2. connect it to a computer and fill it with zeroes (e.g. dd if=/dev/zero of=/dev/sd#)
3. connect it to the eSATA port on the Motorola DVR and let the device initialize it
4. test 1-2 minutes to see if the new disk is recognized, but do NOT record anything on it!
5. disconnect the disk from the DVR and connect it to the computer again
6. create a compressed image of the disk (e.g. dd if=/dev/sd# | gzip -c >dd_image.gz)
7. share the resulted image with the community (e.g. via torr ents)

Because of the steps 2 and 4 above and thanks to the compression I estimate that the image will be reasonably small to easily transfer via torr ents. Try to keep step 4 as short as possible because all the buffered data will show up in the image file, making it very large (Total buffer space taken by both tuners is 19280MB).

If several people do this with the most usual disk capacities (i.e. 320GB, 500GB, 640GB, 1TB) we may very well end up with "templates" suitable for everybody!

So... anybody up for this? I'd do it myself, but I don't have the spare disk yet (I'm not currently in the US and here disks are twice the price - that is *when* you can find them... I'd rather go and buy one after we find out if this actually works). On top of this - how do I find if the eSATA port on my DCT3412 unit is enabled? I checked d11 (interface/port status) in the diagnostics menu but eSATA is not listed at all.

LE: after all I may not need this for a while since I accidentally deleted everything from my disk by testing the HDD reset procedure and foolishly assuming that pressing "Exit" means "cancel"... Now I suddenly have 88GB free again (i.e. max. space available on a 120GB disk)

Cheers!

PS: Some very good info can be found at http://en.wikibooks.org/wiki/How_to_use_a_Motorola_DVR

It worked. I took his response to mean that everything worked, not just the cloning. He stuck it in his other cable box and it works and sees the full 1 TB of space.

I'm skeptical that compression will be able to reduce the image size by a lot (after all, GPFS is an unrecognized partition type), but this is worth a try.

I'll test it out with the 160 GB default hard disk size and see how much smaller things are with compression. I don't live in an area where the eSATA port works and am unwilling to buy a pre-formatted 1 TB drive, so someone else will have to help out if the compression turns out to be worthwhile.

If you observe steps 2 and 4 it will work, I can promise you that This is how:
a. In step 2 you fill the *entire* disk with zeros, regardless of any (future) partitions;
b. Steps 3 and 4 are the only ones when the DVR writes anything to the disk; if you minimize the testing duration in step 4 you'll also minimize the info written on the disk (remember, in step 3 the DVR just "formats" the disk, it does not actually fill it up, but all the buffering in step 4 is done on the disk);
c. most of the image realized in step 6 would consist of only zeroes, which gzip will compress extremely well regardless of the type of information of the disk (i.e. partitions, filesystems etc.) - dd simply does a raw "disk dump", sector by sector.

Great, please do so and report back. Don't omit step 2, it's essential even if the disk is brand new.
Warning: You need to pay extra attention to the disk letter you use (e.g. /dev/sda or /dev/sdb or /dev/sdc etc.) in steps 2 and 6.

Cheers
Dan

Yeah, the more I think about it, the more I like this idea and believe it will work.

Yeah, I'll be mega careful about choosing the correct drive! Hopefully I'll find time this evening to try this.

Great, looking forward to hear how it went

In the meantime, do you have any idea how I can check if the eSATA port on my DCT3412 box is active? I checked the Diagnostics and I couldn't find anything relevant. Also - I don't have an extra empty SATA disk to test.

LE: Never mind, I found out this: http://www.motorola.com/staticfiles/...er%20Guide.pdf :
"In order to use Motorola's External DVR Storage feature, your cable service provider must authorize your account for use of this feature. If your account has not been authorized to use this feature, you will be notified by an on-screen message upon connection of an external drive to your Motorola DVR cable box."

Cheers

You know what worries me? This:
http://www.motorola.com/staticfiles/...er%20Guide.pdf
Chapter 3: "Once the formatting process is complete, the external drive can only be used with the last cable box that was used to format the external drive."

I don't believe that. Any recordings on it will only play back on the cable box that recorded it, but the drive itself is usable by other cable boxes.

I think their statement is just being overly broad. As in, if you cart this external drive (loaded with recordings) over to your friend's home, they aren't going to play back.

As for your question about e-SATA, the only way to know is to hook up the drive and see if an error appears on the screen. A firmware update removed the requirement of calling the cable company and giving them your external drive's serial number. But the cable company has to allow for the use of external drives, which very few do.

Here's what the error message looks like if your cable company doesn't allow external drives (this automatically comes up as soon as you hook the drive to the cable box):

Okay, so I changed my mind on using the 160 GB drive...it has too many shows that I want to transfer via firewire or Hauppauge HD-PVR (for anything 5c'd to copy-once or copy-never).

So instead I'm using a spare 250 GB hard drive.

So far I've written zeros to the 250 GB drive using the method described above. My command line was:
dd if=/dev/zero of=/dev/sda

It took about 3 hours to fill the 250 GB drive with zeros.

Later today I will hook the 250 GB hard drive into the DCH3416 and give it just enough time to initialize and format it. As a test, I will also record about 1 minute of SD material...this will be needed for me to be sure that the whole thing wasn't unrecognized and then reformatted. (ultimately made necessary by the fact that the partition size is always the default 160 GB...I need to be able to make sure it's *my* 160 GB partition and not a new reformatted one).

I will then create the image.

So far, this is awesome! The image has been created and is only 2.01 GB in size!!

I didn't use Linux for this initial test. For imaging speed reasons, I used the trial version of Active@ Disk Image (I used the LiveCD bootable disc, http://www.livecd.com/download.htm) and created a "Raw disk image" using Normal compression. It took only 1:11:17 to create. Using dd in Linux would have taken a fair bit longer.

I am now dumping that image to a spare 500 GB hard drive that I have. Hopefully it will show the 2 minute PBS recording that I made.

If this works, I'll do it again without putting a recording on and see how big the image is.

Sound perfectly fine to me, we're waiting your results
I wish I had a spare disk to check if my eSATA port is enabled.

Just one question - why do you think that using linux dd would be slower? After all the bottleneck is the disk itself. I hope your imaging program really creates a full disk bit-for-bit clone regardless of partitions, filesystems etc.

1 minute of SD material doesn't take too much space, it's probably not even significant compared to the 2GB image you created. However if you start again you may want to "clean" the disk again (fill it with zeroes) to keep the size as small as possible.

Cheers
Dan

I know dd is slower b/c I've used it for this purpose numerous times. It took 4 hours to dump my 160 GB DVR drive. That *exact* same image was created using Active@ Disk Image (from a WinPE boot disk) in under 2 hours.

Just look at my screenshots from earlier in this thread; I only got 11.5 MB/s using dd. I get far better throughput using Ghost, Active@ Disk Image, Easeus Disk Copy, etc. (all tested against the exact same 160 GB DVR hard drive with the same recordings on it, etc..a level playing field.)

So I'm not sure why dd is that much slower, but it is.


I will absolutely re-fill the drive with zeros first, otherwise I would not expect the size to be better than the 2.01 GB I now have. I may also consider having Active@ Disk Image use High compression instead of Normal compression, though I don't expect any improvements based on that.


It best! Otherwise they shouldn't refer to it as a "Raw" image. Actually, the 500 GB drive just finished imaging as I was typing this...and it took only 00:59:23 to load the image!

And it WORKED! I see the 2 minute SD recording of PBS! And it plays back just fine.

So now I'm going to go back to the 250 GB drive and write zeros to it again and try this procedure again without recording anything to it. Should I put the STB into standby and never actually power it up? Just let it initialize and format the drive, but just leave it in standby for a couple minutes before pulling the drive and creating the image...
Thoughts?

I'm glad it worked! To quote an uber-geek: "I informed you thusly!" )

Your issue with dd speed may be related to the block size. Try playing with the bs= cmd line arg (bigger blocks=faster transfer, but the computer may not be so "snappy" during the transfer).

Remember that even if the DVR is in standby mode it still buffers 2 channels (i.e. records data to disk). So I would power it off and disconnect the disk as soon as you're sure it is correctly formatted & usable.

Now it would be really interesting to get somebody with an enabled eSATA port to do the same to check your theory that this is how they format the "extension" 1TB drives sold on eBay...

I'm fairly certain that is incorrect. When I put my DCH3416 into standby, it does not buffer the tuners. This is quite noticeable when I have my 750 GB Seagate drive in it b/c that damn drive is one noisy mofo. All that click-click-clinckin' falls silent right-quick once in standby. That, and the fact the buffers are empty upon powerup points to a complete (or nearly complete) lack of hard drive use when in standby.


Agreed, but my crystal ball is currently in the shop


Oh, that's how they do it.
I'm certain that's how they do it. I suppose it is remotely possible that an externally formatted drive really could only be utilized by the same cable box that formatted it, but I highly, highly doubt it.

I'm not doing this to see if that's how others have managed it. It is. I know it, they know it, dogs know it. I'm doing this to see how small the image can be made. And I love your idea of making "community templates" that people could freely download to extend the recording space.

Now me myself, I've decided I don't even want the extra space. It would encourage me to be lazy about transferring shows. I don't really even use the 160 GB. I'm just dicking around with this thing b/c it's fun.

If somebody is willing to download my image and load it on their 250 GB or larger hard drive, we could see if it refuses to operate on any other cable box than the one that formatted it.

Oh, my bad, I don't even know why I assumed that you have exactly the same Motorola model as me (DCT3412). Mine never truly stops, whenever I turn it back on I have the last hour buffered for both channels. While it initially bothered me (the DVR stays hot 24x7 and wastes a lot of energy) I kinda got used to the convenience of being able to check the BBC news from the beginning of the news bulletin regardless of current time


Here's the relevant fragment from the Motorola manual but it's ambiguous:
"When you connect an external drive to your cable box, it will need to be
formatted for DVR use. An on-screen message will prompt you to begin the
formatting process. If the external drive was previously used with another
device, all existing contents on the drive will be erased. Once the
formatting process is complete, the external drive can only be used with the
last cable box that was used to format the external drive."



Glad to "hear" this, I wish I could prepare them myself. I got it the instant I read your post about the disks being formatted using the eSATA port, so you get half the credit for it



Funny you wrote this - I find it very true - the bigger the disk, the more likely you are to record more shows to it hoping you'll be able to "catch up".



OK then, I reckon I'll need to find a SATA disk to test your image. Could I ask you a small favor though? Could you please try making a dd+gzip image? I don't like having to use windows stuff for this kind of job. And on top of this aren't you curious to see if the dd+gzip image is smaller? Oh, and BTW, how to you plan to share the first "community template"?

Cheers,
Dan

Well I can't really prepare the "real deal"...I have no way of connecting to the eSATA and formatting a 1 TB drive

But at least this testing can prove that we're on the right track. The genius of your idea is the method of getting compression...I pondered the idea of a template but dismissed it almost immediately b/c I assumed it would be far too huge to share over the internet. I was mistakenly assuming that the "Foreign" GPFS partition type would stymie any attempt at compression, not considering that the cable box is really just doing a "quick format" and not writing to every sector (which would take quite some time to finish, and thus would be undesirable).


Yes, I will absolutely use the dd+gzip method. I used Active@ Disk Image merely as a test, since I've seen that it gets far better throughput than what I've experienced using dd. I will also test with other imaging programs just to see where I get the best compression. But I'll do the dd+gzip method first.

Also: I'm making a change in this next test. I'm moving around some data and freeing up a 200 GB hard drive to use for this next test. This means that anybody with a 200 GB or larger SATA drive will be able to load my image and test it. I will begin writing zeros to the 200 GB hard drive within the next 5 minutes. That'll likely take about 2 1/2 hours, at which point I'll hook it to my cable box and leave the thing on standby to see how small this image can be. I will then create the image.


I'm not certain how I'll share the result...I've used sendspace.com in the past...they have a 300 MB size limit, so it'd need to be split into chunks (which is probably a good idea anyway). I'm certainly open to suggestions.

If the program you used compresses the image in individual blocks then I would expect dd+gzip to give you a better compression ratio overall.

I did a few simple tests and for example compressing 1GB of zeroes the resulting archive was roughly 970kB in size - that's a 1030:1 compression ratio. For a 250GB zeroed disk this would mean the compressed image would be roughly 250MB. I doubt the DVR writes so much info to the disk during formatting to make up the difference to the 2GB compressed image you managed to get using the windows proggie. So I think you should give linux+dd+gzip a try

I have the feeling that my eSATA port might be enabled... I'm not in the States and the local cable operator agreed to provision my own box that I bought on eBay, so I think they may not be so interested in disabling the external ports... I'll test this as soon as I find a spare SATA disk and I'll report back.



Like I said in my first post - I'd suggest creating torr ents. It would help a lot (both in terms of bandwidth and availability) if more than a few persons are interested in downloading the image. It could also prove more convenient to download using just my router

Cheers,
Dan

It's been well over a year since I've done anything involving torr ents. What would you recommend for a client? I have to imagine that my upload speed stinks, but I guess we'll find out.

Me too, but they're really good for distributing huge files.

I would pick a very basic one, one that may not even need to be permanently installed.
But I use the one that's built in my router so I can't help with an actual suggestion, sorry

However if you can get the image size down to a few hundred MB that should be also manageable by ftp.

It looks like the image is going to be extremely small.

I used WinRAR to compress the Active@ Disk Image I had already created, and the 2.01 GB image compressed to less than 150 MB! And the image of the completely empty DVR drive was under a megabyte! (This was the one where I never once powered up the DVR, I just left it in standby and gave it a couple of minutes to make sure it had a chance to format the drive).

This was done using WinRAR's maximum compression...I'm guessing the dd | gzip files will be slightly larger.

My computer is currently creating the dd | gzip compressed image of the empty DVR drive. I'll upload it to SendSpace once it finishes.

************************************************************ ************************************************************ ***************
*****DISCLAIMER: The following post is outdated and the steps should not be performed. If you are looking to upgrade your DVR hard drive to 1 TB, see this post! *****
************************************************************ ************************************************************ ***************



OK, the dd | gzip operations are done and the resulting image file is 185 MB.

This is the image of the 100% empty Motorola DCH3416 HDD. I never even powered the STB up; It was in standby before I removed the power cord and hooked the HDD to it and, after plugging in the power cord, it intialized (and presumably formatted the drive) and returned to standby. I gave it several minutes in standby to make sure it had a chance to do whatever needed to be done to the hard drive.

During the initialization, it spun up the hard drive and I heard it writing to it for maybe 5-10 seconds. After that it fell silent and I don't think it wrote to the drive again, even though I gave it another couple of minutes in standby.

Now, just for kicks, I took this 185 MB .gz file and ran it through WinRar 4.00 Beta 7 using Maximum Compression...it was able to drop the file size from 185 MB to only 181 KB!!

Here's the download link to the 181 KB WinRar compressed image
http://www.sendspace.com/file/q63cf5

When unrar'd, this will have the 185 MB .gz file which contains the image created using the Linux dd command.
You can download the Linux version of WinRar 4.00 Beta 7 from here:
http://www.rarlab.com/download.htm

I'm currently also uploading the full 185 MB .gz file and will post that link when it finishes uploading.

************************************************************ ************************************************************ ***************
*****DISCLAIMER: The following post is outdated and the steps should not be performed. If you are looking to upgrade your DVR hard drive to 1 TB, see this post! *****
************************************************************ ************************************************************ ***************



The 185 MB .gz image is done uploading. Here's the link:
http://www.sendspace.com/file/luylw3

Later on I'll create another image that has just one short recording on it, so we can see if that is still on there when loaded on another STB (it certainly won't be playable on another STB, but I think I've read that you'll see it in the MyDVR screen).

GRRRRRRR
Downloading it in a minute, but I just hate pages that NEED JavaScript to work... (hehe, I guess this has a name... "paranoia"? ) )

I think something's slightly off... I got the .rar file, extracted the .gz from it and:
Wasn't the image supposed to be 200GB or 250GB?

LE: Turns out it is, gzip -lv reported it wrong:
So are you sure the disk was formatted and usable by the DVR when you created this image?

I used a 200 GB drive, but the way the companies calculate the size is a little misleading. It looks like the true size is 186 GB

But the question on whether the drive was for sure formatted and usable is a good one. I'm 90% sure it was. I may have to set up some tests to try and figure it out...maybe making videos of the STB initializing and timing it.

I think it'd be best for me to use my 750 GB Seagate for that sort of testing, as it is really really noisy compared to any other hard drive I have.

I'll wipe the seagate and play around with how long it takes before it's able to be powered on. My assumption is that it will be able to power up faster if already formatted...hopefully it takes more than a couple seconds for it to format

As you said, "just for kicks" I got the same image down to 14kB using just gzip

So what have we learned from this? That compression algorithms were probably not designed for this type of input (~200GB of zeroes), but if you apply them a few times they manage in the end. As you can see it can not compress it beyond 14kB, but it still is impressive. At this size you could even uuencode it and send it via email or PM

Holy crap! That is both ludicrous and awesome! Boosh!!

It's the GB (gigabyte, 10^9) vs. GiB (gibibyte, 2^30), see for example http://en.wikipedia.org/wiki/GiB


I think it was formatted. Well, at least _something_ was written to the disk, I could see some data in the sea of zeroes. Funny, I assumed that the DVR would ask you if you want to format the drive...

Well, I guess that's the amount of data the DVR writes to the disk when it formats it. Now I just need to find a 200+GB spare SATA disk to test the image and if my eSATA port is enabled or not. Hopefully I'll be able to do this over the coming week-end. I'll post back here when I do. Thanks a lot for taking the time to do all these tests!

Cheers

You're welcome. And thank you for both having the awesome idea of filling the drive with zeros and helping me test.

I'll put that currently blank DVR hard drive back in the cable box and record a super short clip to it and create another image. This will be something easier to test, as I hope it'll show up in the MyDVR screen.