Announcement

Collapse
No announcement yet.

Jon00 DataScraper Script

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Originally posted by jon00 View Post
    Yes, that can be easily added. Just awaiting a response from Malosa regarding his issues.
    Awesome

    Comment


    • Originally posted by Malosa View Post


      thanks, yeah maybe it wil solve the problem then.. but i still not understand why it keep the settings in homeseer.. maybe a bug in homeseer?
      running .500
      Originally posted by Blob View Post

      Awesome
      Please try V1.0.15 on my site.

      Path entry now supports replacement variables
      When using UseIE, the orphan instances of IE are now killed after 3 seconds (definable in the ini file)
      Jon

      Comment


      • Originally posted by jon00 View Post



        Please try V1.0.15 on my site.

        Path entry now supports replacement variables
        When using UseIE, the orphan instances of IE are now killed after 3 seconds (definable in the ini file)
        Hi jon,

        many thanks for the update, but it still won't work.
        when i use your new setting it scrapes wrong i get only (30kb of the scrape instead of 350 kb)
        even when i set it up to 8 seconds..

        So i always need to restart homeseer and than rweplace the older scraper file and it works again..
        i think it has to do something with killing the iexplorer in homeseer.. when i use a bat file it does not happen.

        regards
        Preferred -> Jon's Plugins, Pushover, Phlocation, Easy-trigger,
        Rfxcom, Blade Plugins, Pushbullet, homekit, Malosa Scripts




        HS3Pro 3.0.0.523 on windows 7 ultimate X64 on hp quadcore laptop 8 GB.

        Comment


        • So if you use UseIE=2 does it work normally (Kill is disabled - you kill it via your bat file)?
          Jon

          Comment


          • Originally posted by Malosa View Post

            Hi jon,

            many thanks for the update, but it still won't work.
            when i use your new setting it scrapes wrong i get only (30kb of the scrape instead of 350 kb)
            even when i set it up to 8 seconds..

            So i always need to restart homeseer and than rweplace the older scraper file and it works again..
            i think it has to do something with killing the iexplorer in homeseer.. when i use a bat file it does not happen.

            regards
            Have you gotten the right data with either of the recent versions using GetURLIE? I had to change my patterns to get it to work because GetURLIE apparently processes the source code differently. Got one pattern working again but haven't solved the other yet.
            -Wade

            Comment


            • Originally posted by jon00 View Post
              So if you use UseIE=2 does it work normally (Kill is disabled - you kill it via your bat file)?
              ok, with the last version on UseIE=2 and even 1 scrape on UseIE=1 doesnt work at all.. but jon if u have time try yourself my url, i send u pm with the scrape code.
              i really want to know if u having the same problems.. i try it on 2 seperate pc's and have both same issue.
              Preferred -> Jon's Plugins, Pushover, Phlocation, Easy-trigger,
              Rfxcom, Blade Plugins, Pushbullet, homekit, Malosa Scripts




              HS3Pro 3.0.0.523 on windows 7 ultimate X64 on hp quadcore laptop 8 GB.

              Comment


              • Originally posted by cc4005 View Post

                Have you gotten the right data with either of the recent versions using GetURLIE? I had to change my patterns to get it to work because GetURLIE apparently processes the source code differently. Got one pattern working again but haven't solved the other yet.
                For me both scrapes work with the version .13 but only when i use the bat file to kill iexplorer.. if i gonna use the build-in kill in the new scraper it scrapes wrong data somehow..
                so then i need to restart homeseer and the scrapes work again with .13
                Preferred -> Jon's Plugins, Pushover, Phlocation, Easy-trigger,
                Rfxcom, Blade Plugins, Pushbullet, homekit, Malosa Scripts




                HS3Pro 3.0.0.523 on windows 7 ultimate X64 on hp quadcore laptop 8 GB.

                Comment


                • I found the issue. Re-download and all should be good!
                  Jon

                  Comment


                  • Originally posted by jon00 View Post
                    I found the issue. Re-download and all should be good!
                    Yep i can confirm it scrapes everything right now.
                    i let it scrape for this whole day and see how it goes

                    but there now is an other thing what came up, thats the scrape with a login and pass, its asking the credentials again, even if i pressed remember.. it keeps coming up everytime the scrape starts, and it doesn't scrape that one.

                    thanks for fixing it
                    Preferred -> Jon's Plugins, Pushover, Phlocation, Easy-trigger,
                    Rfxcom, Blade Plugins, Pushbullet, homekit, Malosa Scripts




                    HS3Pro 3.0.0.523 on windows 7 ultimate X64 on hp quadcore laptop 8 GB.

                    Comment


                    • Originally posted by jon00 View Post



                      Please try V1.0.15 on my site.

                      Path entry now supports replacement variables
                      Thanks Jon. Just what i needed

                      Comment


                      • The password thing is fixed.. forget to enable an option.

                        it still works jon,
                        great job!

                        But when i scrape from a password protected website, it scrapes but old data.. so not the data from now only the data if i manually open the website and close it then it wil scrape the new data only.. any idea ?

                        Preferred -> Jon's Plugins, Pushover, Phlocation, Easy-trigger,
                        Rfxcom, Blade Plugins, Pushbullet, homekit, Malosa Scripts




                        HS3Pro 3.0.0.523 on windows 7 ultimate X64 on hp quadcore laptop 8 GB.

                        Comment


                        • Hi jon,

                          actually this is one more thing i don't get why it does not scrape.
                          like i have an device (owermeter) it uses a username and pass..
                          so i did fill it in and it gets the data normal the first time..

                          but after 10 minutes it gets the same data over and over again.. so if i open up internet explorer and load up the page and close it and i start scraping i get the new data.. any idea why ?

                          regards
                          Preferred -> Jon's Plugins, Pushover, Phlocation, Easy-trigger,
                          Rfxcom, Blade Plugins, Pushbullet, homekit, Malosa Scripts




                          HS3Pro 3.0.0.523 on windows 7 ultimate X64 on hp quadcore laptop 8 GB.

                          Comment


                          • Sounds like a cache is not cleared because IE is killed. As a test, you could try that by setting UseIE=2

                            What happens if you try another dummy user/pass first?
                            Jon

                            Comment


                            • hi jon,

                              i did change it to 2 and used wrong credentials and i deleted the data file ..
                              what happens is it still scrapes the old info and there is no message with the wrong credentials.

                              no idea why
                              Preferred -> Jon's Plugins, Pushover, Phlocation, Easy-trigger,
                              Rfxcom, Blade Plugins, Pushbullet, homekit, Malosa Scripts




                              HS3Pro 3.0.0.523 on windows 7 ultimate X64 on hp quadcore laptop 8 GB.

                              Comment


                              • i did try again by restarting hs, and now it scrapes something (i see it in the db file) but without info, so i think it can't login.
                                when i press it 4x after eachother a window pops up from internet explorer with a pre filled in pass and username and i can click ok, but nothing happens as it closes to quick.

                                i did also try http://username : password @192.168.1.13/ it won't work aswell
                                when i go to iexplore and i paste that link it works normal.

                                regards
                                Preferred -> Jon's Plugins, Pushover, Phlocation, Easy-trigger,
                                Rfxcom, Blade Plugins, Pushbullet, homekit, Malosa Scripts




                                HS3Pro 3.0.0.523 on windows 7 ultimate X64 on hp quadcore laptop 8 GB.

                                Comment

                                Working...
                                X