RPTV4K limit on drive size - 750GB? - Page 2 - AVS Forum
Forum Jump: 
 
Thread Tools
Old 07-06-2008, 09:05 PM - Thread Starter
Senior Member
 
markus98's Avatar
 
Join Date: Dec 2001
Location: Tucson, Arizona
Posts: 325
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by hdonzis View Post

Because, you aren't reading that I'm talking about the extract_rtv project and you are talking about the RTVPatch project. They are both in the RTVPatch repository, but there are three projects under that, Extract, extract_rtv, and RTVPatch. Extract_rtv is a much simpler project with just a couple of source files (crctest.c and extract_rtv.c), but the main source file I quoted above has a lot of code out of the RTVPatch project. And, I updated the code back in March to compile on Umbuntu. So, if you want, you can look at the changes that I made to extract_rtv.c and make similar changes to the RTVPatch source code (which I quoted above, like how I fixed the _syscall5 error and the packing errors)...

Henry

At 1609 (GMT-0700) i posted a patchset to two files that makes rtvpatch compile cleanly as well. Is that patch flawed?

Thanks

E

markus98 is offline  
Sponsored Links
Advertisement
 
Old 07-06-2008, 09:27 PM
AVS Special Member
 
hdonzis's Avatar
 
Join Date: Mar 2003
Location: San Antonio, TX
Posts: 2,015
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by markus98 View Post

At 1609 (GMT-0700) i posted a patchset to two files that makes rtvpatch compile cleanly as well. Is that patch flawed?

I didn't see anything flawed about it. I just wanted you to know that I had come up with a cleaner way of fixing the _llseek/_syscall5 problem (which I fixed in March specifically for Umbuntu) and that I had cleaned up the packing and type mismatch warnings. I did all this WITHOUT having to make special IFDEFs using the existing IFDEFs...

Henry
hdonzis is offline  
Old 07-06-2008, 09:33 PM
AVS Special Member
 
hdonzis's Avatar
 
Join Date: Mar 2003
Location: San Antonio, TX
Posts: 2,015
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by markus98 View Post

At 1609 (GMT-0700) i posted a patchset to two files that makes rtvpatch compile cleanly as well. Is that patch flawed?

Actually, just double checking what you patched, I do see a flaw. You got rid of the packing errors by getting rid of packing entirely. I guarantee you that RTVPatch won't operate properly without packing. In addition to removing the pack attribute, you also need to add "#pragma pack(1)" and "#pragma pack()" around every structure definition, like I did for extract_rtv.c...

And, here is the diff for how I fixed the lseek() problem (in addition to some other unrelated code changes)...

Henry
hdonzis is offline  
Old 07-06-2008, 09:37 PM - Thread Starter
Senior Member
 
markus98's Avatar
 
Join Date: Dec 2001
Location: Tucson, Arizona
Posts: 325
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by hdonzis View Post

Actually, just double checking what you patched, I do see a flaw. You got rid of the packing errors by getting rid of packing entirely. I guarantee you that RTVPatch won't operate properly without packing. In addition to removing the pack attribute, you also need to add "#pragma pack(1)" and "#pragma pack()" around every structure definition, like I did for extract_rtv.c...

Henry


Roger that. Unfortunately your fix in March to use __NR__llseek() works only on 32-bit platforms since /usr/include/asm/unistd_32.h defines it but unistd_64.h does not.

E

markus98 is offline  
Old 07-06-2008, 09:44 PM
AVS Special Member
 
hdonzis's Avatar
 
Join Date: Mar 2003
Location: San Antonio, TX
Posts: 2,015
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by markus98 View Post

Roger that. Unfortunately your fix in March to use __NR__llseek() works only on 32-bit platforms since /usr/include/asm/unistd_32.h defines it but unistd_64.h does not.

OK. I'm sure I could add additional IFDEFs which would compile properly on the 64-bit variant. Might be good for extract_rtv as well...

Henry
hdonzis is offline  
Old 07-09-2008, 04:39 PM
AVS Special Member
 
hdonzis's Avatar
 
Join Date: Mar 2003
Location: San Antonio, TX
Posts: 2,015
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by markus98 View Post

Unfortunately your fix in March to use __NR__llseek() works only on 32-bit platforms since /usr/include/asm/unistd_32.h defines it but unistd_64.h does not.

I posted version 75 of extract_rtv5 on SourceForge and added for it to compile on the 64-bit Linux variants. If it compiles correctly for you, it may be a better way to change your RTVPatch code...

Henry
hdonzis is offline  
Old 07-12-2008, 12:34 PM
AVS Special Member
 
hdonzis's Avatar
 
Join Date: Mar 2003
Location: San Antonio, TX
Posts: 2,015
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by markus98 View Post

+static int _llseek(unsigned int fd, unsigned long offset_high, unsigned long offset_low, loff_t *result, unsigned int whence)
+{
+ if (offset_high)
+ return -1;
+ *result = lseek(fd, offset_low, whence);
+ if (*result == -1)
+ return -1;
+ return 0;
+}

This will also cause a problem. You've limited access to only the first 4GB of the hard drive, which is most likely larger than 4GB. Access to more than 4GBs is why it is using the 64-bit lseek in the first place, which was a special system call in the older Linux. If you wanted to continue to correct it the way you have indicated instead of the new way that I changed it in extract_rtv.c, then you'd want something like:

Code:
+static int _llseek(unsigned int fd, unsigned long offset_high, unsigned long offset_low, loff_t *result, unsigned int whence)
+{
+       *result = lseek(fd, (offset_high << 32L) + offset_low, whence);
+       if (*result == -1)
+               return -1;
+       return 0;
+}
Which is suggested if you Google this problem. However, the way I changed it in extract_rtv.c, instead of taking the 64-bit position apart into 2 32-bit numbers in order to use the system call and then put the two 32-bit numbers back together to use the 64-bit lseek(), I simply changed the caller to call the 64-bit lseek(), which is what it does on other operating systems...

Henry
hdonzis is offline  
Old 07-12-2008, 02:57 PM - Thread Starter
Senior Member
 
markus98's Avatar
 
Join Date: Dec 2001
Location: Tucson, Arizona
Posts: 325
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
I have been swamped with "real" work, but I will get to this code
Thanks

Ehud
P.S. Lest it's not clear, I'm not clear on lseek, and my cosmetic suggestions clearly wrong. Thanks for taking the time!

markus98 is offline  
Old 07-12-2008, 03:54 PM
AVS Special Member
 
hdonzis's Avatar
 
Join Date: Mar 2003
Location: San Antonio, TX
Posts: 2,015
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Out of curiosity, is the pre-compile Linux binary not working for you? Or, are you hoping that you are going to fix some problem with using 2TB drives?

In case you are still concerned about 2TB overflowing a signed 32-bit number, remember that a 2TB drive is 2 trillion bytes, not 2 tera-bytes. So, that is really only 1863 giga-bytes, which is much less than the 2047 giga-bytes which a signed number could represent. So, since hard drives are in 1000's of bytes instead of 1024's of bytes, that already gives you more than enough cushion to not have to worry about getting too close to overflowing a signed integer...

Henry
hdonzis is offline  
Old 07-24-2013, 10:02 PM - Thread Starter
Senior Member
 
markus98's Avatar
 
Join Date: Dec 2001
Location: Tucson, Arizona
Posts: 325
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
And here we are 5 years later, and it's time to upgrade the drives.

2001 - bought an RTV4040 and upgraded to dual 80GBs
2002 - the 136GB patch software was out so upgraded to dual 160GBs
2008 - upgraded to 1TB using a SATA-IDE adapter but couldn't get two drives to work - likely had to futz with master/slave/CS and didn't
2013 - upgraded to 2TB using a SATA-IDE adapter

The question of the day in 2008 was how to use RTVPatch. I couldn't find a program that built on modern-day Linux 2.6 kernel systems
and the only modifications that were of use were to extract_rtv not RTVPatch.

Turns out that's a red herring. The 2.5.5 RTVPatch program works fine. HOWEVER, here's how to use it:

1. Ensure you're logged in as root.
2. Ensure you've set a HARD LINK to make the SCSI drive look like an IDE drive to the kernel:
NOTE: this works with SCSI and USB but you need to find the drive letter (dmesg!) and here we assume it's /dev/sdb
# cd /dev ; rm -rf hdb* ; ln hdb sdb
3. # RTVPatch ... it will find the drive

Having verified that, exit out (x) and:
1. copy the .rtv file to the drive (dd if=whatever_replay_image_you_downloaded.rtv of=/dev/sdb)
2. # RTVPatch
t set target drive to the right drive number
p patch the drive
yes to patch it
no to copying MPEG partition
x to exit
3. # sync && sleep 1

Now you have a perfectly good 2TB RTV4K SATA drive that just needs a SATA<--IDE adapter to work.

Enditem.

E

markus98 is offline  
Old 07-24-2013, 11:42 PM - Thread Starter
Senior Member
 
markus98's Avatar
 
Join Date: Dec 2001
Location: Tucson, Arizona
Posts: 325
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
I spoke too soon.

The ReplayTV 4K OS will not support this.

I did a factory default with the 2TB drive on. It let me choose anywhere from 2000GB/0GB Programs/Photo
too 1600GB/400GB Programs/Photo.

No matter where I chose in that range including the edges, the Replay Guide showed a lot of hours
(overflow)... and any attempt to watch anything FAILED. (Insufficient space to record...)

I had to go back to a 1TB drive to get it to work.

Long evening. I had to reverse engineer stuff I've only done twice before in over a decade.

Cheers,

Ehud

markus98 is offline  
Old 07-25-2013, 08:11 PM - Thread Starter
Senior Member
 
markus98's Avatar
 
Join Date: Dec 2001
Location: Tucson, Arizona
Posts: 325
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Here are the instructions as I've been able to distill them down. These apply to 4xxx series.

Note 1: When doing the dd you don't need to specify either bs=1024 or count=520000.
Note 2: When patching the drive you likely need to force the size to get it to work if not autodetected properly.

More good news. Can go up to 1.1TB... so no use buying 2TB drive. [This restriction is either rtvpatch or
rtv4k software... I'm unsure... and since I don't have a buildable source of rtvpatch can't try and fix it.]

Make sure RTVPatch gets the size of the drive right. If it does not, and if you
don't fix it, you'll see your ReplayGuide remaining recording time to be either
above 30000hrs or 3min. In this case, pay attention:

1. connect the drive. If using a USB and the linux boot cd, wait until it's up and
you have the root prompt to connect. Then connect. Then wait about 30
seconds for it to be recognized. You'll know if you're early because the next
step won't give you anything useful until then.

2. find out which drive it is and how big
# dmesg | egrep "sd[a-z]|blocks" ; this will tell you which drive, sda, sdb, etc. and how many 512-byte blocks

3. You'll want to make /dev/sdX (sdb, sdc, etc.) appear to be /dev/hdX (hdb, hdc, etc.) so that rtvpatch can find it.
# rm -f /dev/hdX ; remove the existing unused entry in the device table for nonexistent IDE disk
# ln /dev/sdX /dev/hdX ; make /dev/hdX point to /dev/sdX

Example: rm -rf /dev/hda && ln /dev/sdb /dev/hda ; make sdb appear as hda

4. copy the image file to that drive
# dd if=Factory\ Reset\ Image\ 520430360.rtv of=/dev/sdb

5. patch the disk. You need to force size AND patch target.
# rtvpatch
a. select t for target and the right drive
b. select f for force size, the right drive, and the number of sectors from step 1 (number of blocks)
c. select p for patch, and answer y to patch, n for mpeg, and n for photos if prompted[depends on image]
d. select x for exit

6. remove the drive. It's helpful to #sync and wait some before unplugging.

markus98 is offline  
Old 07-26-2013, 10:26 AM
AVS Special Member
 
Reden's Avatar
 
Join Date: Jul 2000
Location: Cedar Hill, TX
Posts: 1,329
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 2 Post(s)
Liked: 12
Is there enough memory to handle a drive that big? I wonder if things will become unbearably slow when it fills up. I know 2k/3k have problems with the larger sized disks (that they can support)... they were designed for <60G smile.gif

---
If it's worth doing, it's worth doing in PERL!
Reden is offline  
Old 07-26-2013, 10:31 AM - Thread Starter
Senior Member
 
markus98's Avatar
 
Join Date: Dec 2001
Location: Tucson, Arizona
Posts: 325
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by Reden View Post

Is there enough memory to handle a drive that big? I wonder if things will become unbearably slow when it fills up. I know 2k/3k have problems with the larger sized disks (that they can support)... they were designed for <60G smile.gif

Anecdotaly: I've ran a 1TB drive since 2008 without issues. It's been SO issue-free I've not used 243-Zones and
activated the memory indicator or the CPU indicator or the network activity indicator since I put it in smile.gif

The 4000 series were designed for 320GB (2x160GB) and at a time when 136GB was the hard limit
on a drive. (A later firmware revision to the onboard IDE controller corrected that). There appears to
be no memory issue with tripling that.

E

markus98 is offline  
Old 08-12-2013, 05:26 PM - Thread Starter
Senior Member
 
markus98's Avatar
 
Join Date: Dec 2001
Location: Tucson, Arizona
Posts: 325
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
More info for those thinking of upgrading their ReplayTV 4000s using SATA drives. There is a LOT of detail here but the process is simple. You don't need

1: I've now used many $7 SATA-IDE adapters to allow the RTV4K to use a SATA drive. There have been no problems. There is usually one jumper to set either to using a SATA drive on an IDE device, or the other way around to use an IDE drive on a SATA device. If you get it wrong, the drive won't work and you'll never get past the initial red logo screen. If you get it right, you'll hear the drive spin up on power-on.

2: I've now used many SATA drives. The only problematic ones are the WD green ones. The "problem" in detail are described in section g ("Notes") below. Drives that work flawlessly are 80GB WD, 500GB Seagate, 1TB Seagate, 2TB WD Red [formatted to 1TB].

3: I've now used many SATA to USB cables to setup the drive on my laptop. They're all $10-$15. Some correctly negotiate the cache information on the drive, and some don't (according to dmesg). It doesn't have any effect on the success of the process.

4: I've now used two different laptops and three different RTV4Ks to test this in detail. No changes in the process were detected.

My setup: I have a 64-bit Linux laptop. Required items are dd (or dcfldd) which is part of the system, a ReplayTV image file, and the RTVPATCH program.

HOWTO in excruciating detail:
a. Required hardware:
1. A SATA drive to IDE device adapter
2. A SATA drive to USB cable
3. A power supply with a SATA drive adapter
Note: 2&3 can be worked around by plugging the drive into your PC, however this really greatly simplifies the whole process since you don't have to open up your PC, can easily redo the process, and even can use this cable to reformat the drive (or pull shows off of it) even when it's in the RTV4K chassis... without having to unscrew it at all.
4. A ReplayTV device
5. A PC

b. Required software/firmware:
1. Any linux installation including a Live-Linux CD or USB bootable device. (DSL, Knoppix, Ubuntu, whatever on an SDcard)
2. The disk image for the proper RTV device. (I used "520430360_factory_reset.zip")
3. The RTVPATCH program

c. Physical Connection
1. Connect the power supply to the new drive
2. Connect the USB adapter cable to the new drive
3. Plug the USB adapter cable into the PC

d. Initial Imaging
1. Boot the PC using any of the Linux methods mentioned in b.1.
2. Unplug the USB adapter cable for a second, then plug it in, then wait 10 seconds
3. Enter the command "dmesg" to see the latest USB device detected, and the device designation (sdb, sdc, sdd, etc.)
Example:[ 166.573624] sd 6:0:0:0: [sdb] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Note: in this example we see it's device SDB and the size is 976773168 512-byte blocks.
4. If the disk image file is unpackets (ends in .rtv or .img or .dd or the link) then skip directly to step 6
5. Unpack the image. In my case it was a zip file, so I did "unzip 520430360_factory_reset.zip". If it ends with .gz then use gunzip. If you're not sure google the file extension (.zip, .gz, .7z, etc.)
6. Take the name of the image file as well as the device name from step 3 and enter the command "dd if=image_file of=/dev/sdX"
Note: image_file is the name of the image file. sdX is the device name (sdb like above is one such).
Note2: Some people have said you have to add "bs=1024" and/or "count=520000". Neither is required nor will either help unless the image file is significantly larger than 520Mbytes.

e. Patching
1. The RTVPATCH program expects original-style hard drive device names (/dev/hdX) instead of SCSI style device names (/dev/sdX). For this reason we need to fool the system into treating this new drive (/dev/sdb in the above example) as an older drive. Do so by the following command: "sudo rm -f /dev/hdd ; ln /dev/sdX /dev/hdd" <-- replace sdX with your device from d.3. above.
2. Run the RTVPATCH program via the command "./RTVPATCH"
3. Hit "t" to select the target drive. Use the number next to the offered choices to select "hdd". If the size shown is correct skip to step 5
4. Hit "f" to force device size. Answer Y to all questions. Use the number next to the offered drive to select "hdd". Set the size in blocks to the number given to you in step d.3 above. (example: 976773168)
5. Hit "p" to patch the target.
6. Hit "x" to exit

f. Cleanup
1. Enter the command "sudo sync" to flush all disk caches and sync the filesystem
2. Disconnect the cables from the drive
3. ***UNPLUG YOUR RTV4K POWER CORD*** and mount the drive into the RTV4K
4. Plug the RTV4K IDE cable into the SATA/IDE adapter
5. Plug the RTV4K hard-drive power cable into the SATA/IDE adapter
6. Plug the new drive into the SATA/IDE adapter
7. Connect up your RTV4K to the TV and reconnect the power cord

g. Notes:
1. If you use a drive larger than 1.1TB, the RTV4K will only see 1.1TB. To prevent filesystem corruption you should force size to no more than that.
2. When using a WD green 2TB and 1TB drive, the following have been observed:
a. When showing Guide (TV or Replay) the background colors or highlights are not properly displayed
b. In the initial factory default screen it freezes and one can't select "I agree"
c. In the 3rd factory default screen entering the ZIP code freezes
d. At the end of the factory default sequence it errors on formatting a photo partition of any size
e. It freezes
3. All of 2a-2e are resolved by its own watchdog rebooting it 30-60 seconds later. "Resolved" means for now, but it happens again shortly.

That's all I have on this topic. Hopefully it will help someone else considering an upgrade to do it.

markus98 is offline  
Old 08-30-2013, 04:32 PM
Member
 
srfrdan's Avatar
 
Join Date: Mar 2010
Location: jersey shore
Posts: 162
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 4 Post(s)
Liked: 13
i could not get a 1.5tb sata to work in a 5k with the adapter but was able to get a dual drive system using a 750gb+80gb ide drves to work in a 4k just for fun---d
srfrdan is offline  
Old 08-30-2013, 04:38 PM - Thread Starter
Senior Member
 
markus98's Avatar
 
Join Date: Dec 2001
Location: Tucson, Arizona
Posts: 325
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by srfrdan View Post

i could not get a 1.5tb sata to work in a 5k with the adapter but was able to get a dual drive system using a 750gb+80gb ide drves to work in a 4k just for fun---d

If you force-size the 1.5TB to just under 1TB it should work smile.gif I found that no matter what the 4K couldn't see more than 1.1TB,
and above 999GB if I tried to resize the photo partition the 4K would error out... but at 999GB I could factory-default and move
that back and forth... and had no random reboots.

Curious - when you used the 1.5TB what issues did you encounter? Mine were just random freezes/reboots and inability
to format photo partition in factory default screen.

E

markus98 is offline  
 
Thread Tools


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off