Need Help with HSTV3 XMLTV Grabber.

[CFGuy]

The way HSTV works is it reads the xml files and puts all the data into the tvImport table. It doesn't do anything other than just put the data in that table. When that process is done, the third event runs and does the cleanup of the data. During the cleanup, the event checks for new channels, new ratings, etc and fills the tables. Then when all the cleanup is done, it moves the data into the tvlistings table.

We aren't to the point where the database is important. The error from the rating system that I was working on is from the xml file import. We need to first get the import to work with no errors and to import all the data.

So start the import xml files event, go to hstv admin, set the error level to the highest. Wait about 5 seconds and change it to the error only level. Then go to the log and look for any errors. First thing to figure out is why the data is not making it into the import table. Then we can move on to the next steps.

Edit:
I noticed the log in your last post. Missed it the first time. Looks like the code I changed is causing an error with your xml files. I didn't get that error here but was using my normal xml files for testing.

Lets do this. Create about 5 or 6 days of xml files and email them to me. When you post them here they get cut off and are imcomplete.
That way I have several to test with.

[travisdh]

The way HSTV works is it reads the xml files and puts all the data into the tvImport table. It doesn't do anything other than just put the data in that table. When that process is done, the third event runs and does the cleanup of the data. During the cleanup, the event checks for new channels, new ratings, etc and fills the tables. Then when all the cleanup is done, it moves the data into the tvlistings table.

We aren't to the point where the database is important. The error from the rating system that I was working on is from the xml file import. We need to first get the import to work with no errors and to import all the data.

So start the import xml files event, go to hstv admin, set the error level to the highest. Wait about 5 seconds and change it to the error only level. Then go to the log and look for any errors. First thing to figure out is why the data is not making it into the import table. Then we can move on to the next steps.

Edit:
I noticed the log in your last post. Missed it the first time. Looks like the code I changed is causing an error with your xml files. I didn't get that error here but was using my normal xml files for testing.

Lets do this. Create about 5 or 6 days of xml files and email them to me. When you post them here they get cut off and are imcomplete.
That way I have several to test with.

Hello

I sent you an email to the messenger email account (hoping you received it as I had to reformat my media pc and lost the data) If you didn't please let me know and I will save up a couple more days of the xml tv data for Aus and resend it to you.

Cheers - Trav

Thanks

[CFGuy]

I did get it but didn't realize it. It ended up in the junk folder. I found it today and will try to get to it pretty soon.

[travisdh]

I did get it but didn't realize it. It ended up in the junk folder. I found it today and will try to get to it pretty soon.

Thanks for that, no rush , It would be nice to be able to use your system to add an extra level of notification to my house, eg this show is about to start in 10 mins .

I was wondering, its very Sci-Fi but in the show Eureka they have the house that tells jack "I remember you liked Baseball, I have recorded the game for you", I was wondering if technically it would be possible to have the HSTV communicate with other plugins that record TV to have that same concept work, tell the house what categories you like, or what shows you like through HSTV and it intergrates with PVR's or other software eg WebScheduler to record .

[travisdh]

I did get it but didn't realize it. It ended up in the junk folder. I found it today and will try to get to it pretty soon.

Thanks again for all your work on a fantastic component. I am not trying to load you up with work but I was wondering, is it possible to set up in your addon the ability to have a grabber eg XMLTV which allows a filename to be just that, without having a date connotation with it. The reason I ask is whilst I have been renaming the xmltv files that are output by the epgstream.net program, it does not actually put a date in it. The output is basically a file which is called in my case xmltv.xml that actually has more than one day in it but as the one file, so its not xmltv20100310 xmltv20100311 etc but always just xmltv.xml, If it is possible it would be great to be able to have a setting to be able to define a non-variable filename.

The other thing I noticed, is that when I needed to import files, I could add them to the folder and from the status tell it to import all, or import new and it seamed like it would delete them before it started. It would be nice, if the file never changes, to be able to not have to delete it at all. My example is just for example with the epgstream.net program, if you set the location to the folder of your addon, every night it would automatically download the listing and put the xmltv.xml file in that folder. Then each night an event could be run that only imports new listings, or imports all listings but does not need to download it. Then the next day when the event runs again, the software for epgstream.net would have redownloaded the listings, and the event would go again without having to delete anything.

I don't mean to sound needy, I am extreamly appreciative of the work you have put into the component, and would love to see it working in my enviroment , especially in the Australian enviroment as I know there are a few australian users. The epgstream.net software also gets listings for:

• Australia - All States
• Europe - Bosnia, Congo, Germany, Spain, Croatia, Hungary, Italy, Macadonia and more.
• India
• South Africa

I assume they all make the xmltv.xml output, and in the case of me this also has an import guide server for windows media centre so it imports into that but It would be great to see support for non-changing date free filenames , and also the ability to have the files be left and not deleted after import

[travisdh]

For those that are interested, epgstream.net lets you run bat files etc after its finished, so I did come up with a way around the file names, it would need to be modified for your setup not mine, but in this example it copys the xmltv.xml file from the location, to a folder in my docs.

Its not the most elegant way of doing it, and it would still prob be better to be able to avoid dates if we could , but its one less thing to look at now.

My regional settings give date like YYYY-MM-DD

XCOPY "C:\ProgramData\epgStream.net\xmltvDownload\XMLTV.xml" "C:\Users\Travis\Documents\xmltv\" /C
cd "C:\Users\Travis\Documents\xmltv\"
Rename "XMLTV.xml" epg%date:~0,4%%date:~5,2%%date:~8,2%.xml
cd "C:\Users\Travis\Documents"

So then we just need to get it to import , and see if we can have it so it doesen't always need to delete the files (maybe?)

[CFGuy]

Thanks for that, no rush , It would be nice to be able to use your system to add an extra level of notification to my house, eg this show is about to start in 10 mins .

I was wondering, its very Sci-Fi but in the show Eureka they have the house that tells jack "I remember you liked Baseball, I have recorded the game for you", I was wondering if technically it would be possible to have the HSTV communicate with other plugins that record TV to have that same concept work, tell the house what categories you like, or what shows you like through HSTV and it intergrates with PVR's or other software eg WebScheduler to record .

originally HSTV could send the record commands to Beyond TV by sending all the data on a URL line. But Beyond TV changed to not accept URLs so I took out the integration.

If there is a HS plugin that someone has written to control a PVR then HSTV would be able to send commands to the plugin. I designed HSTV to be able to call other plugins commands. I would just have to write a script that adds the plugin info to the correct HSTV tables.

[CFGuy]

For those that are interested, epgstream.net lets you run bat files etc after its finished, so I did come up with a way around the file names, it would need to be modified for your setup not mine, but in this example it copys the xmltv.xml file from the location, to a folder in my docs.

Its not the most elegant way of doing it, and it would still prob be better to be able to avoid dates if we could , but its one less thing to look at now.

My regional settings give date like YYYY-MM-DD

XCOPY "C:\ProgramData\epgStream.net\xmltvDownload\XMLTV.xml" "C:\Users\Travis\Documents\xmltv\" /C
cd "C:\Users\Travis\Documents\xmltv\"
Rename "XMLTV.xml" epg%date:~0,4%%date:~5,2%%date:~8,2%.xml
cd "C:\Users\Travis\Documents"

So then we just need to get it to import , and see if we can have it so it doesen't always need to delete the files (maybe?)

Actually you can set HSTV to not delete the listing files.
If you go to the page where you entered the grabber name, there is an option
Re-download all xml files every night

If that is set to yes, then it deletes the xml file and downloads it again the next night.
Set to no, it does not delete the xml files.

The reason I did this is to allow changes to the listings data. Ex. The grid shows Game 7 of the baseball World Series. The series ends in 5 games and the data is sometimes updated by the channel. If the old xml file was not deleted, then it still shows Game 7. If you do delete the files and download all new every night, then the xml file shows whatever is going to air in place of Game 7. A lot of people probably won't care about a change that is pretty rare. Thats why it is a user settable option.

This also ties into why I put the date in the file name. If you set the option to not delete the files and the event triggers to download the data, it needs to know if that day is already downloaded. The download event always starts with the current day and loops ahead based on how many days ahead you have set in the XMLTV options. I have mine set to download 13 days ahead.
The event runs, the loop starts. Looks to see if the file for today is there. If so, it skips the download and looks for the next day. It skips the downloads until it finds a date with no file.
However, in your case you are not using the download event so I can see why the file name wouldn't need the date in it. But I also use the date in the name to know which files to import so I think ideally, you can use your script to change the name of the file when it copies it to the listings directory.
Perhaps you could make it a HS script and have the script read the ini file for the name you used for the grabber. It can then dynamically make the filename for you. Just run your script after the download each day.

[CFGuy]

Thanks again for all your work on a fantastic component. I am not trying to load you up with work but I was wondering, is it possible to set up in your addon the ability to have a grabber eg XMLTV which allows a filename to be just that, without having a date connotation with it. The reason I ask is whilst I have been renaming the xmltv files that are output by the epgstream.net program, it does not actually put a date in it. The output is basically a file which is called in my case xmltv.xml that actually has more than one day in it but as the one file, so its not xmltv20100310 xmltv20100311 etc but always just xmltv.xml, If it is possible it would be great to be able to have a setting to be able to define a non-variable filename.

The other thing I noticed, is that when I needed to import files, I could add them to the folder and from the status tell it to import all, or import new and it seamed like it would delete them before it started. It would be nice, if the file never changes, to be able to not have to delete it at all. My example is just for example with the epgstream.net program, if you set the location to the folder of your addon, every night it would automatically download the listing and put the xmltv.xml file in that folder. Then each night an event could be run that only imports new listings, or imports all listings but does not need to download it. Then the next day when the event runs again, the software for epgstream.net would have redownloaded the listings, and the event would go again without having to delete anything.

I don't mean to sound needy, I am extreamly appreciative of the work you have put into the component, and would love to see it working in my enviroment , especially in the Australian enviroment as I know there are a few australian users. The epgstream.net software also gets listings for:

• Australia - All States
• Europe - Bosnia, Congo, Germany, Spain, Croatia, Hungary, Italy, Macadonia and more.
• India
• South Africa

I assume they all make the xmltv.xml output, and in the case of me this also has an import guide server for windows media centre so it imports into that but It would be great to see support for non-changing date free filenames , and also the ability to have the files be left and not deleted after import

See my comments above about the file name with dates in it and how to set HSTV to not delete the file.

For the short term, I want to get the import working. Anything, more than that will need to wait until July. I have 4 weeks left in my current term at school, and taking 2 classes. After that I go back to 1 class for 10 weeks and then I am done on June 29th. I will have a lot more time after that. Only 110 days to go. Not that I am counting or anything.

[travisdh]

See my comments above about the file name with dates in it and how to set HSTV to not delete the file.

For the short term, I want to get the import working. Anything, more than that will need to wait until July. I have 4 weeks left in my current term at school, and taking 2 classes. After that I go back to 1 class for 10 weeks and then I am done on June 29th. I will have a lot more time after that. Only 110 days to go. Not that I am counting or anything.

Thats not a problem at all The work you have put into the addon is fantastic and I look forward to seeing it working in my setup . Enjoy your free time because as you see it never lasts very long at all lol

[travisdh]

Hello ,

Just wanted to get an update, and see if you had any luck getting the import going with the epgstream data?

And also how the studies are going

I heard mention of being able to interface HSTV with snapstream to use HSTV to tell a snapstream PC to record a channel at a time, is this functionality still in the system?, And if so with HSTV could I have it detect a certian genre or title of a show and it automatically record it via snapstream?

Thanks Heaps

[travisdh]

Just for your info, I tried about 3 additional sources of xmltv data. Still all melbourne sources but it normally comes up with similar errors and will not import the data. I thought id even try a clean install of hstv3 to see but no luck.

http://guides.byremote.com.au/guides...oMelbourne.zip

That was another example of one of the data sources I used, renamed and also tried different grabber names and not much luck, the constant error recieved was either:

Error Import XML: Conversion from type 'DBNull' to type 'String' is not valid.
Error Import XML: Public member 'Length' on type 'DBNull' not found.

4/2/2010 5:10:09 PM ~!~Event~!~Event Trigger "HSTV Import XML FIles"
4/2/2010 5:10:09 PM ~!~Event~!~Running script statement immediately: &hs.Plugin("HSTV").dataImport()
4/2/2010 5:10:09 PM ~!~HSTV~!~Data Import: Importing XML files.
4/2/2010 5:10:09 PM ~!~HSTV~!~Deleting records from tvImport table.
4/2/2010 5:10:09 PM ~!~HSTV~!~sql=DELETE  *  FROM tvImport
4/2/2010 5:10:09 PM ~!~HSTV~!~Starting import of file: C:\Program Files\HomeSeer HSPRO\data\hstv\listings\epg20100402.xml
4/2/2010 5:10:10 PM ~!~HSTV~!~SELECT rating, ratingID FROM tvRatings WHERE ratingID = 1
4/2/2010 5:10:11 PM ~!~HSTV~!~Importing 7TWO (0) from C:\Program Files\HomeSeer HSPRO\data\hstv\listings\epg20100402.xml
4/2/2010 5:10:11 PM ~!~HSTV~!~Error Import XML: Public member 'Length' on type 'DBNull' not found.
4/2/2010 5:10:11 PM ~!~HSTV~!~SQL: INSERT INTO tvImport (channelID, channelname, channelnum, starttime, endtime, title, vrTitle, episode, description, category, episodenum, audio, subtitles, ratingsystem, rating, ratingadvisory, videoaspect, videocolor, starrating, actors, guests, director, producers, hosts, repeatshowing, programyear, channelDownload, configfile, viewhide) VALUES ('7TWO','7TWO','0',#2010/04/01 00:10:00#,#2010/04/01 01:00:00#,'Dirty Sexy Money','Dirty Sexy Money','The Birthday Present','Six months have passed and the Darlings are throwing Nick George a fantastic yacht party - anyone who is anyone in New York is there... and soon so are the NYPD.','Drama','1.0.',Null,Null,

or the other one was

4/2/2010 8:05:03 PM ~!~HSTV~!~PagePut: name: startXMLImport = value:Import Data
4/2/2010 8:05:03 PM ~!~HSTV~!~PagePut: name: ref_page = value:hstv
4/2/2010 8:05:03 PM ~!~HSTV~!~PagePut: name: pg = value:hstvInstall
4/2/2010 8:05:03 PM ~!~HSTV~!~Deleting records from tvImport table.
4/2/2010 8:05:03 PM ~!~HSTV~!~sql=DELETE  *  FROM tvImport
4/2/2010 8:05:03 PM ~!~HSTV~!~Starting import of file: C:\Program Files\HomeSeer HSPRO\data\hstv\listings\epg20100402.xml
4/2/2010 8:05:03 PM ~!~HSTV~!~SELECT rating, ratingID FROM tvRatings WHERE ratingID = 0
4/2/2010 8:05:04 PM ~!~HSTV~!~Importing ABC1 (0) from C:\Program Files\HomeSeer HSPRO\data\hstv\listings\epg20100402.xml
4/2/2010 8:05:04 PM ~!~HSTV~!~Error Import XML: Conversion from type 'DBNull' to type 'String' is not valid.

[travisdh]

Update?

Hi,

Just wondering if your still looking into this?, I havent seen you around the forums in like a month so am unsure of if you are still around or if HSTV is still being maintained?

I know you are / were busy with school and the likes, but just wanted to see if any progress had been made?

Once again, Thanks for all the wonderful work into HSTV .

[CFGuy]

Sorry, I haven't been around much.
Just too much going on with work, school and life.

At this point I doubt I will be able to get to it until I finish school at the end of June.

[travisdh]

Hi, Just wondering if you have had any luck looking at the format of the epgstream data and using it to re-configure HSTV3 to allow it to import that data?

I am really really keen to be able to use HSTV for upcoming movie announcements and all that information.

Thanks