Jon00 DataScraper/JSON Parser Script For Homeseer 3 and Homeseer 4

[Blob]

Yes, that can be easily added. Just awaiting a response from Malosa regarding his issues.

Awesome

[jon00]

thanks, yeah maybe it wil solve the problem then.. but i still not understand why it keep the settings in homeseer.. maybe a bug in homeseer?
running .500

Awesome

Please try V1.0.15 on my site.

Path entry now supports replacement variables
When using UseIE, the orphan instances of IE are now killed after 3 seconds (definable in the ini file)

[Malosa]

Please try V1.0.15 on my site.

Path entry now supports replacement variables
When using UseIE, the orphan instances of IE are now killed after 3 seconds (definable in the ini file)

Hi jon,

many thanks for the update, but it still won't work.
when i use your new setting it scrapes wrong i get only (30kb of the scrape instead of 350 kb)
even when i set it up to 8 seconds..

So i always need to restart homeseer and than rweplace the older scraper file and it works again..
i think it has to do something with killing the iexplorer in homeseer.. when i use a bat file it does not happen.

regards

[jon00]

So if you use UseIE=2 does it work normally (Kill is disabled - you kill it via your bat file)?

[Wade]

Hi jon,

many thanks for the update, but it still won't work.
when i use your new setting it scrapes wrong i get only (30kb of the scrape instead of 350 kb)
even when i set it up to 8 seconds..

So i always need to restart homeseer and than rweplace the older scraper file and it works again..
i think it has to do something with killing the iexplorer in homeseer.. when i use a bat file it does not happen.

regards

Have you gotten the right data with either of the recent versions using GetURLIE? I had to change my patterns to get it to work because GetURLIE apparently processes the source code differently. Got one pattern working again but haven't solved the other yet.

[Malosa]

So if you use UseIE=2 does it work normally (Kill is disabled - you kill it via your bat file)?

ok, with the last version on UseIE=2 and even 1 scrape on UseIE=1 doesnt work at all.. but jon if u have time try yourself my url, i send u pm with the scrape code.
i really want to know if u having the same problems.. i try it on 2 seperate pc's and have both same issue.

[Malosa]

Have you gotten the right data with either of the recent versions using GetURLIE? I had to change my patterns to get it to work because GetURLIE apparently processes the source code differently. Got one pattern working again but haven't solved the other yet.

For me both scrapes work with the version .13 but only when i use the bat file to kill iexplorer.. if i gonna use the build-in kill in the new scraper it scrapes wrong data somehow..
so then i need to restart homeseer and the scrapes work again with .13

[jon00]

I found the issue. Re-download and all should be good!

[Malosa]

I found the issue. Re-download and all should be good!

Yep i can confirm it scrapes everything right now.
i let it scrape for this whole day and see how it goes

but there now is an other thing what came up, thats the scrape with a login and pass, its asking the credentials again, even if i pressed remember.. it keeps coming up everytime the scrape starts, and it doesn't scrape that one.

thanks for fixing it

[Blob]

Please try V1.0.15 on my site.

Path entry now supports replacement variables

Thanks Jon. Just what i needed

[Malosa]

The password thing is fixed.. forget to enable an option.

it still works jon,
great job!

But when i scrape from a password protected website, it scrapes but old data.. so not the data from now only the data if i manually open the website and close it then it wil scrape the new data only.. any idea ?

[Malosa]

Hi jon,

actually this is one more thing i don't get why it does not scrape.
like i have an device (owermeter) it uses a username and pass..
so i did fill it in and it gets the data normal the first time..

but after 10 minutes it gets the same data over and over again.. so if i open up internet explorer and load up the page and close it and i start scraping i get the new data.. any idea why ?

regards

[jon00]

Sounds like a cache is not cleared because IE is killed. As a test, you could try that by setting UseIE=2

What happens if you try another dummy user/pass first?

[Malosa]

hi jon,

i did change it to 2 and used wrong credentials and i deleted the data file ..
what happens is it still scrapes the old info and there is no message with the wrong credentials.

no idea why

[Malosa]

i did try again by restarting hs, and now it scrapes something (i see it in the db file) but without info, so i think it can't login.
when i press it 4x after eachother a window pops up from internet explorer with a pre filled in pass and username and i can click ok, but nothing happens as it closes to quick.

i did also try http://username : password @192.168.1.13/ it won't work aswell
when i go to iexplore and i paste that link it works normal.

regards