[timothee]

PiGS: The CM-7500 DVR+ EP Guide Server Replacement

Introducing PiGS - The Channel Master CM-7500 DVR+ EP Guide Server Replacement, that uses Guide data from Schedules Direct.

Send inquiries to [email protected] if you wish, but just watch this thread for updates. Visit pigs.goihl.com to download the application.

PiGS was approved for use with Schedules Direct data on June 17, 2020.

What is PiGS? PiGS is an application (a program) that allows your DVR+ to use Guide data (the EPG) from Schedules Direct instead of the Channel Master provided guide data. It works by redirecting DVR+'s guide requests from Channel Master's servers to your personal server using a simple DNS redirection technique.

How much does it cost? The PiGS application is FREE, a requirement of the licensing agreement with Schedules Direct. You will need an account with Schedules Direct ($25/yr), less than a cup (a good cup) of coffee a month! SD provides excellent data and has support if you ever have guide data issues. You will also need something to run the PiGS application on, like a $35 Raspberry Pi computer, but PiGS can also run on other platforms, like Windows Desktop computers.

PiGS features
Schedules Direct gets daily updates from GraceNote, so you will get the most current schedule data.
SD has a support email if you have lineup / schedule concerns (and they respond!)
PiGS accomodates up to 4 lineups (multiple zipcodes) so you should be able to get guide data on any channel you can tune
PiGS goes out to CM/Rovi to get the pretty channel Icons (but none of their other garbage data)
PiGS can also use channel Icons from Schedules Direct or you can even create your own Icon artwork
PiGS puts the word (New) in front of the description of any programs that are considered 'new'
(This overcomes a bug in the DVR+ popup guide where 'New Episode' is not displayed on some channels)
PiGS can supply the Guide to older DVR+ firmware versions (1.34R tested, works).
Provides enhanced guide data with Episode, Season, and Cast info, e.g.:

(New)"Raging Bulls." S6, E8. Aired: 1998-12-15. Tension between Sipowicz and Fancy explodes after a white patrolman shoots a black undercover officer. Cast List: Dennis Franz, Rick Schroder, James McDaniel, Nicholas Turturro, Gordon Clapp, Kim Delaney, Andrea Thompson, Sharon Lawrence, Kevin Dillon, Lola Glaudini, Scott Allan Campbell, Christopher Stanley, Sterling Macer, Scott Klace, James McBride, Joe Sabatino, David-Jean Thomas, Leontine Guilliard, Kingston DuCoeur.(469)" ​

PiGS was developed in the Python programming language on a Raspberry Pi computer - the name PiGS stands for Pi Guide Server.

Python code can run on different machines. At present, PiGS is running on a Pi 3B-1GB, a Pi 4, and even Pi Zero's, desktop Linux Machine, and Windows.

The PiGS application was coded in Python 3.7 and javascript (for the configuration webpage). As of June 15, 2020, it consisted of 4142 lines of code and comments.

Research on the project started in November 2019 when Channel Master had a complete EPG outage. Since then the CM guide data has been marginal at best. You can follow the history and progress on this chain of posts.

The research and development information can be found on the Channel Master DVR Owners Thread here on this forum, but please post comments on this thread if your question is about the Channel Master Guide and PiGS, even if you are responding to a post over there, copy the Quote and text here - members are likely following this thread also.

Here is the Shared Google Sheet with the Pi Setup and lots of reference information. Don't be intimidated by the setup. It's step-by-step and not too difficult. It's long because every little step is documented. Over time, the installation and Setup will likely be simplified.

About the CM/Rovi Guide... Did you know:
CM guide data is only updated like 1 time per week!
You will notice that the end date of your guide does not advance each day, then all at once, there will be another week of data.
That means every day your DVR+ gets the same bad data for a whole week.
At the time I write this, the CM/Rovi data for 53593 ends on 6/24 and:

• has 69 programs (1.1%) that do not have titles (unknown events), may of them ABC,NBC type shows
• has 938 programs (15.3%) that do not have program descriptions!

That means, even if a TV Network (eg NBC) sends new data to Rovi, your DVR+ will probably never see it.

 

[frank70]

If you also need to run the Apache web server...

The Raspberry Pi 3 that I used happened to already be running another newtork accessible application, so I had previously installed the Apache web server which normally serves web pages out of /var/www/html on IP port 80.

But the PiGS program needs to also bind to IP port 80 in order to service guide requests from the DVR+. If Apache is running in its default state (port 80), PiGS will fail to start, complaining that it can't bind to port 80.

Because the DVR+ always attempts to talk to epg.channelmastertv.com or roproxy-guide.echostarcdn.com on port 80, PiGS cannot successfully use a different port. Therefore, you must change a couple of Apache configuration files to make it use the commonly accepted alternate HTTP port 8080. Here is how to do that (type the following linux shell commands):

sudo apachectl stop            [ignore any warnings]
cd /etc/apache2/
sudo vi ports.conf             [use your favorite text editor]
   change: Listen 80
   to:     Listen 8080
   and save the file
cd sites-enabled
sudo vi 000-default.conf       [use your favorite text editor]
   change <VirtualHost *:80>
   to     <VirtualHost *:8080>
   and save the file
sudo apachectl start           [ignore any warnings]

From this point on, you will need to access your other (non-PiGS) web pages by appending :8080 to the end of the host name or IP. For example:
192.168.1.191:8080/index.html instead of 192.168.1.191/index.html
The PiGS Configuration page would be accessed without the appended :8080 thusly:
192.168.1.191 (or 192.168.1.191/index.htm)

A tad cumbersome, but the only way to run both servers concurrently.

 

[flowersrj]

great job

Tim,


I am really liking this alternative to replacing the DVR+.


You're going to keeping the price up for a used DVR+.



Especially like the fact that there is more detail under INFO for many shows. Especially when recording all instances of an old show. Now it is easier to root out the duplicates.



Thanks for all your hard work

 

[timothee]

Especially like the fact that there is more detail under INFO for many shows. Especially when recording all instances of an old show. Now it is easier to root out the duplicates.
Thanks for all your hard work

That is my favorite feature also, along with the (New) in the description. I was constantly opening the TVGuide app on my phone to find the Episode and Cast info, and now it's right there on the Guide!

My other favorite feature.... we beat the 'system.'

 

[frank70]

My other favorite feature.... we beat the 'system.'

And if CM tries to get cute and force another firmware update on us to circumvent PiGS (which they could do by bypassing DNS and hard-coding the guide server IP), we can just add the firmware update host to the "hosts" file (which name I apparently dare not say here) and redirect the firmware update check somewhere where it won't register.

 

[pachinko]

And if CM tries to get cute and force another firmware update on us to circumvent PiGS (which they could do by bypassing DNS and hard-coding the guide server IP), we can just add the firmware update host to the "hosts" file (which name I apparently dare not say here) and redirect the firmware update check somewhere where it won't register.

If you ask me, CM should be grateful for PiGS! When I think about it, CM should recommend PiGS when their free guide service finally dies (which could be now for some). Of course, keeping the DVR+ alive makes it less likely owners will be looking for an alternative OTA device. Of course, at the moment, IMHO, they lack anything to temp me (I'd rather the device whose name shall not be mentioned). But a good EPG keeps DVR+ owners happy, and that's a positive for CM. At the very least, it keeps us thinking about other products they sell, and if there's one thing that turns me off to a company, is when they do something to harm their customers.

Of course, I don't believe CM would try to circumvent PiGS, even if they could with E* assistance. Neither one tried to do anything negative to DVR+ Lister. But they likely won't promote PiGS either, although, IMHO, they should.

 

[timothee]

Did I ever mention my other thought? The DVR hardware is incredibly simple. A Broadcom System on a Chip, 2 tuner/demodulator pairs, some RAM, some ROM, and some glue logic for the various ports.

Broadcom did all the work - all E* did was solder the stuff to a pcb. Broadcom supplies a programming library for their video chips which I wouldn't be surprised if it did half the the functions you see on DVR+. It's mostly a programming project. Hmm, what do we want? 4 tuners, multiple hard drives, network streaming, remote programming on a phone app...

ATSC 3.0 has me a little concerned though... the overview I read spent a lot of time talking about DRM, which makes me think they are going to build an OTA PPV cable system and the only free stuff will be QVC and HSN.

 

[timothee]

And if CM tries to get cute and force another firmware update on us to circumvent PiGS (which they could do by bypassing DNS and hard-coding the guide server IP), we can just add the firmware update host to the "hosts" file (which name I apparently dare not say here) and redirect the firmware update check somewhere where it won't register.

I added the the urls you referenced to the install instructions, then connected Wireshark to watch those addresses:

1 16:58:06.444680 192.168.1.201 67.148.153.225 TCP 74 35195→80 [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=1388261828 TSecr=0 WS=128
2 16:58:06.483823 192.168.1.201 67.148.153.225 TCP 66 35195→80 [ACK] Seq=1 Ack=1 Win=14600 Len=0 TSval=1388261867 TSecr=3638788121
3 16:58:06.484597 67.148.153.225 192.168.1.201 TCP 74 80→35195 [SYN, ACK] Seq=0 Ack=1 Win=4380 Len=0 MSS=1460 TSval=3638788121 TSecr=1388261828 SACK_PERM=1
4 16:58:06.487612 192.168.1.201 67.148.153.225 HTTP 222 GET /cm-7500-gb/cm-7500-gb_targets.php?sw_id=135R&rcvr_id=R3484603909 HTTP/1.1
5 16:58:06.520276 67.148.153.225 192.168.1.201 TCP 60 80→35195 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0
6 16:58:06.526602 67.148.153.225 192.168.1.201 TCP 60 80→35195 [RST, ACK] Seq=1 Ack=157 Win=0 Len=0

Lines 1,2,3 The server lets dvr+ connect
Lines 4 Dvr asks for a list of FW. Says I have v135R and my mac id=R is the dec equiv of my dvrs mac address CF-B2-D6-05
Lines 5,6 No Response. The firmware server is Dead.

GET /cm-7500-gb/cm-7500-gb_targets.php?sw_id=135R&rcvr_id=R3484603909 HTTP/1.1
User-Agent: Downloader CM-7500-GB Host: tr50.dishaccess.tv Accept: */*

Update 2020-06-22: My DVR checked again today at 16:58 today also, so it appears that it does a firmware update check once per day at the exact same time.

 

[flowersrj]

... Hmm, what do we want? 4 tuners, multiple hard drives, network streaming, remote programming on a phone app...

If I were to dream ...



- would love to see bubble help showing program details when in the guide
- keep previous guide searches
- share harddrive between 2 or more DVR+ or use a local NAS





-Rich

 

[frank70]

I have 2 DVR+ units using PiGS now

I just set up a 2nd DVR+ (which is actually a spare that I got for a song when CM was closing these out on ebay) to use PiGS, and as soon as I changed its IP from DHCP to static, and put in PiGS as the DNS server, this 2nd unit requested a full guide download from PiGS without me even having to initiate a guide refresh.

In the meantime, I found an "out-of-sight-out-of-mind" place for the Pi3 inside my TV stand right behind the primary DVR+. Since it's on WiFi, it only needs the power cable. I put a little piece of tape on the case to diffuse the power LED, since it was a bit glary (the TV stand has glass door). There's a picture of my setup attached. The Pi3 to the left is the one running PiGS (the Pi2 off to the right has been sitting there happily running OpenElec/Kodi for the past 5 years). The Pi3 is powered off the same UPS as the DVR+, Pi2, and AppleTV. Its CPU runs at about 51 degrees Celsius tucked in there, which is well within spec.

Everything is running just dandy and I love the new guide. Thanks again Tim!

 

[timothee]

Channel Master data was ending tomorrow 6/24. Overnight the data on their server was extended to July 8. There has been no improvement in their data 'quality'
655 programs without titles (Unknown Event), 7234 programs without program descriptions (No description available).
PM me your zip code if you want me to run a report of your lineup - would be interesting to see if its just me or if it's across the board.

Just so everyone understands what this means... CM is getting an update from their upstream data provider once every week or two. The data that is now on their server is what your DVR is going to get until July 8. SD on the other hand gets daily updates from GraceNote. PiGS rebuilds its internal schedule at least once per day with the most current SD data.

==== Summary ====
2020-06-23T00:00:00Z - 2020-07-08T00:00:00Z (16 Days)
Ch Filter: None
Total Airings : 45057
NoTitleKey 1.5% 655
EmptyTitle 0.0% 0
NoCopyKey 16.1% 7234

Starting CM/Rovi Data test at 2020-06-23 16:12(CDT)
SYS args name=pc.py Port=8080 Zip=53593
Checking ServiceDetail Lineup zip(53593), ServiceId(20499)
Channel(41.2) VirtualChannel(17.4) Error!
Channel(41.4) VirtualChannel(49.4) Error!
Scanning the CM/Rovi database for errors. PostalCode(53593)
Totals>> {'TotAirings': 45057, 'NoTitleKey': 655, 'EmptyTitle': 0, 'NoCopyKey': 7234, 'EmptyCopy': 77, 'BadSvcId': 0}

I also scanned CM data for the zips of a few of the PiGS Beta testers:
37135 492/1.1% no title, 6898 / 14.8% no description
70448 478/1.4% no title, 7372 / 22.1% no description
15330 422/1.2% no title, 5450 / 16.8% no description
19053 900/1.5% no title, 9801 / 16.6% no description (this must be a bigger market of channels)

 

[frank70]

19053 900/1.5% no title, 9801 / 16.6% no description (this must be a bigger market of channels)

19053 is in the Philadelphia metropolitan area. There are 110 channels in the SD lineup (I certainly don't use all of those - we are inundated with infomercial channels, plus a few religious and Spanish language channels and a few duplicated channels). But the Rovi/CM guide was (is?) nearly always missing titles/info for several prime-time programs on the major networks per day - it was very frustrating. But since PiGS... no more!

 

[pachinko]

Automatically start pigs.py at boot time

Unless I missed this, or something else, the Pi needs to automatically start pigs.py during boot, or login.

sudo python3.7 pigs.py 80

This is especially necessary when power is lost, and later restored (without our knowledge), or the DVR+ will not receive an EPG update until the program is manually executed.

Five ways to run a Pi program at startup can be found at: https://www.dexterindustries.com/howto/run-a-program-on-your-raspberry-pi-at-startup/

 

[timothee]

Unless I missed this, or something else, the Pi needs to automatically start pigs.py during boot, or login.

sudo python3.7 pigs.py 80

This is especially necessary when power is lost, and later restored (without our knowledge), or the DVR+ will not receive an EPG update until the program is manually executed.

Five ways to run a Pi program at startup can be found at: https://www.dexterindustries.com/howto/run-a-program-on-your-raspberry-pi-at-startup/

Ya... V0.99 coming soon with some fixes. The current method is manual start from the terminal window.

A few impatient beta testers have pushed me along with some good suggestions (tyvm). 1) I added some network delay to allow the network connection to stabilize before going out to SD (was causing a crash during the auto-startup), and 2) pigs now figures out what directory it is started from to avoid an incorrect working directory issue (pigs couldnt find its config page, or config file). Being directory aware greatly simplifies the auto-booting configuration. Both issue have been resolved. I'm looking for a license to stick on the project, and then V0.99 will come out.

You can preview auto-start on tab..... [Auto-start] You'll see I worked thru method 4, but two others were successful with #5 .

 

[pachinko]

Ya... its coming soon....

Great! A little bit at a time.

 

[frank70]

autostarting

Tim, I've been using a different method for a couple years, but that means I don't know whether the current Raspbian supports this. But anyway, for reference, here it is:

Edit the file:
~/.config/lxsession/LXDE-pi/autostart [notice that's .config, not config]
Add a line to the end that runs an executable shell script:
@/home/pi/pigs/startupscript.sh

Now this script will run every time a desktop opens, which happens on the main HDMI display after the Pi boots (assuming you've set it up to boot to desktop); but it also happens if you've installed xrdp (the remote desktop protocol server) and invoke a remote desktop client to bring up a Pi desktop remotely. So since I don't want the script (startupscript.sh in my example) to attempt to start PiGS a second time when I open a remote desktop, the script needs to test whether PiGS is already running and only start it if it is not running (this example assumes that no python3 program other than PiGS runs on this Pi):

[B]#!/bin/sh
if ps -e | grep -q python3
then
        # Already running PiGS - do nothing
        true
else
        cd /home/pi/pigs
        sudo python3 pigs.py 80 >/tmp/pigs.output 2>&1 &
fi[/B]

Anyway, that's the simplistic view of how I do it. My startup script is more complex because I'm running multiple programs (unrelated to PiGS) in terminal windows, and display special windows in the remote desktop, but the concept is the same.

You might want to check if a newer Raspbian OS even has a file ~/.config/lxsession/LXDE-pi/autostart, and if it does, will it execute a script if you add a new line to it (I don't know what the leading @ does, but apparently it needs to be there). It might not if they've changed the desktop to something other than LXDE.

 

[timothee]

Tim, I've been using a different method for a couple years, but that means I don't know whether the current Raspbian supports this. But anyway, for reference, here it is:

Edit the file:
~/.config/lxsession/LXDE-pi/autostart [notice that's .config, not config]
Add a line to the end that runs an executable shell script:
@/home/pi/pigs/startupscript.sh

Now this script will run every time a desktop opens, which happens on the main HDMI display after the Pi boots (assuming you've set it up to boot to desktop); but it also happens if you've installed xrdp (the remote desktop protocol server) and invoke a remote desktop client to bring up a Pi desktop remotely. So since I don't want the script (startupscript.sh in my example) to attempt to start PiGS a second time when I open a remote desktop, the script needs to test whether PiGS is already running and only start it if it is not running (this example assumes that no python3 program other than PiGS runs on this Pi):

[B]#!/bin/sh
if ps -e | grep -q python3
then
        # Already running PiGS - do nothing
        true
else
        cd /home/pi/pigs
        sudo python3 pigs.py 80 >/tmp/pigs.output 2>&1 &
fi[/B]

Anyway, that's the simplistic view of how I do it. My startup script is more complex because I'm running multiple programs (unrelated to PiGS) in terminal windows, and display special windows in the remote desktop, but the concept is the same.

You might want to check if a newer Raspbian OS even has a file ~/.config/lxsession/LXDE-pi/autostart, and if it does, will it execute a script if you add a new line to it (I don't know what the leading @ does, but apparently it needs to be there). It might not if they've changed the desktop to something other than LXDE.

This is what I find:
[email protected]:~ $ ls ~/.config/lxsession
LXDE-pi
[email protected]:~ $ ls ~/.config/lxsession/LXDE-pi
[email protected]:~ $

looks like it is mostly there.

Also, v0.99 eliminates the need for the cd/home/pi/pigs - that was causing lots of headaches. pigs figures out the working directory by itself. your cd wont hurt anything.

also... your script looks a lot like the method 1 rc.local - it wouldn't work for me b/c pigs got started so early the network was not yet up. your method must start pigs later (at desktop) time, and the network was up. v99 fixed that too waiting for the network to become available before checking in with SD.

 

[frank70]

This is what I find:
[email protected]:~ $ ls ~/.config/lxsession
LXDE-pi
[email protected]:~ $ ls ~/.config/lxsession/LXDE-pi
[email protected]:~ $

looks like it is mostly there.

So it may be that newer versions of the OS replaced the LXDE desktop with one called PIXEL, in which case you still may find a desktop "autostart" file in a different directory (probably somewhere else under .config, but not the one in the "autostart" directory directly under .config, which is a different kind of beast.)

In case compatibility with the LXDE autostart may have been retained, you can try creating an ordinary text file in the empty directory ~/.config/lxsession/LXDE-pi called "autostart" and put a test line in there like:
@cp /bin/ls /home/pi
Then reboot and see if after your desktop comes up, there is a copy of ls in /home/pi. If there is, just delete it and replace that line with one that runs a startup script. If there isn't, then this won't work because LXDE is not your desktop.

 

[LenL]

Anyone working on a WIN 10 solution? Seems like someone has it running in a Linux distro?



If we get python running under windows 10 do we do away with the need for the raspberry PI hardware?

 

[flowersrj]

Anyone working on a WIN 10 solution? Seems like someone has it running in a Linux distro?

I believe you might be able to run in Windows in a linux subsystem. See ...

https://docs.microsoft.com/en-us/windows/wsl/install-win10


I think you then install a Linux distro afterwards, just a guess.



flowersrj