Announcement

Collapse
No announcement yet.

Jon00 DataScraper Script

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Give this pattern a go:

    Code:
    Pattern1=(?s)instantaneous value: .*? (.*?)  
    Jon

    Comment


    • Originally posted by jon00 View Post
      Give this pattern a go:

      Code:
      Pattern1=(?s)instantaneous value: .*? (.*?)  
      Yes, of course, match on the blank spaces! Works perfectly, thank you!
      -Wade

      Comment


      • Sorry if this is a basic question; Can the DataScraper script be used with javascript based web pages?

        I'm trying to pull temperature from a Control-by-Web temperature sensor (https://www.controlbyweb.com/xw110/) that has a http interface. The page is simple, and at first it looked like an easy task.

        Click image for larger version

Name:	Screenshot (27).png
Views:	1
Size:	5.3 KB
ID:	1264542


        However when I look at the source for the interface page, i see:

        Code:
        <html><head><title>XW-110</title><style type="text/css"> * { margin: 0px; padding: 0px; } .main { padding: 5px 5px 5px 5px; background-color: #737BA0;border-radius: 4px;-webkit-border-radius: 4px;-moz-border-radius: 4px; } div.left { width: auto; float: left; padding: 5px; } div.main { width: auto; float: left; padding: 10px; margin: 14px; border: solid 1px black; } span.left { float: left; display: block; text-align: right; width: 8em; } span.right { float: left; display: block; text-align: left; } table { background: #929dcc; border-style: solid; border-width: 0px 1px 1px 0px; border-color: #000; font: 14px verdana, arial, sans-serif; color: #fff;width: 352px; } td { padding: 5px 10px 5px 10px; text-align: left; background: #484e65; border-style: solid; border-width: 1px 0px 0px 1px; border-color: #000; } td.header {padding: 10px 30px 10px 30px;text-align: center;background: #f8f8f8;font: bold 20px verdana, arial, sans-serif;color: #484e65 !important; } td.center { text-align: center; } input { padding: 2px 8px 2px 8px; } </style><script type="text/javascript">var rID=null;function gId(a){return document.getElementById(a)}function refresh(a){var b=(new Date).getTime(),c="state.xml";ajaxRequest(a?c+"?"+a+"&updtTemp=1&time="+b:c+"?updtTemp=1&time="+b)}function resetRefresh(){rID&&clearTimeout(rID);rID=setTimeout("refresh()",3E3)}function ajaxRequest(a){if(ax="Microsoft Internet Explorer"==navigator.appName?new ActiveXObject("Microsoft.XMLHTTP"):new XMLHttpRequest){ax.onreadystatechange=ajaxRequestFinished;try{ax.open("GET",a,!0),ax.send(null)}catch(b){}}}function ajaxRequestFinished(){switch(ax.readyState){case 4:resetRefresh(),updateStatus()}}function fV(a){var b="<"+a+">";a="</"+a+">";index=ax.responseText.indexOf(b);startindex=index+b.length;endindex=ax.responseText.indexOf(a);return ax.responseText.slice(startindex,endindex)}function updateStatus(){var a,b;a=fV("sensor1");b=gId("sv1");a="C"==fV("units")?a+" &deg;C":a+" &deg;F";b.innerHTML=a;a=fV("s1Alrm");"0"!=a?(b.style.backgroundColor="rgb(255, 0, 0)",b.style.color="rgb(255, 255, 255)"):(b.style.backgroundColor="",b.style.color="");a=fV("battery");b=gId("sv2");"External"===fV("powersource")?(b.innerHTML="External",b.style.backgroundColor="",b.style.color=""):15>=a?(b.innerHTML=a+" %",b.style.backgroundColor="rgb(255, 0, 0)",b.style.color="rgb(255, 255, 255)"):(b.innerHTML=a+" %",b.style.backgroundColor="",b.style.color="")}function init(){refresh()};</script></head><body onload="javascript:init()"><div class="main"><div class="left"><table id="t0"><tbody><tr><td class="header" id="hd">XW-110</td></tr></tbody></table></div><div style="clear: both;"></div><div class="left" id="d1"><table id="t1"><tbody><tr id="s1"><td>Temperature</td><td id="sv1" style="color: rgb(255, 255, 255); background-color: rgb(255, 0, 0);">13.0 °C</td></tr><tr id="s2"><td>Battery</td><td id="sv2">External</td></tr></tbody></table></div><div class="left" id="d2"></div><div style="clear: both;"></div><div class="left" id="d3"></div><div style="clear: both;"></div></div></body></html>
        If I look at the temperature element as displayed on the on the page, I can see:

        Code:
        <td id="sv1" style="background-color: rgb(255, 0, 0); color: rgb(255, 255, 255);">13.0 °C</td>
        And attempted this, but no luck:

        Code:
        Pattern1= (?s) <td id="sv1" style="background-color: rgb(255, 0, 0); color: rgb(255, 255, 255);">(.*?)</td>
        Am I on the right track, or out to lunch? Thanks for any help!

        Comment


        • Out to lunch unfortunately. The temperature is being updated via AJAX calls hence the reason you cannot see it in the source code.
          Jon

          Comment


          • Hi,
            I have a setup running without problems for several months.
            After upgrading mono from version 4.5.1 to 5.16.0.220 the script stops working. A reboot fixes the problem, but after a few days it stops. The script is running every five minutes to update status from a webpage.

            Is there anything I can to to troubleshoot? Any logging that I can enable?
            In my HS -log it looks like everything is normal and the script is started, but its not producing any outputs anymore..

            Comment


            • I suspect it is related to a recent mono bug: https://github.com/mono/mono/issues/8775

              All you can do is revert until the bug is fixed.
              Jon

              Comment


              • Hello Jon,
                I have tried setting my Path statement in the Jon00DataScraper.ini file to a mapped drive X:\ as well as a network share UNC path but I get the "cannot access path" errors you see below. All permissions are correct on the network share and the share folder and file are accessible in File Explorer as you can see from the screenshots. The local drive path i.e. C:\cox_usage.txt works fine.
                Any ideas as to what I am missing?
                Thank you.

                Mapped Dive X:\






                Network Share UNC path:








                Comment


                • This is an OS permissions issue.

                  Have you tried opening the same file with a text editor on your Homeseer PC?

                  Jon

                  Comment


                  • Yes, I can open the file locally on the Homeseer PC with Notepad.

                    By OS permissions, do you mean HS3 System OS i.e. Windows 10 or shared folder OS?

                    Comment


                    • Well I meant to say file permissions issue. If you can map this to a local drive, I cannot see how it is not working.

                      Would you post your path entry in the ini file.
                      Jon

                      Comment


                      • I have tried both of the paths shown below:



                        Comment


                        • Also, I can open the cox_usage.txt file in Notepad on the Homeseeer PC from the shared drive directly, but it seems like the script cannot access the file.

                          Comment


                          • Would you please run the following script and tell me the result in the Homeseer log:

                            Code:
                            Sub Main(ByVal Parm As Object)
                            
                            Dim Path As String = "X:\cox_usage.txt"
                            
                            If System.IO.File.Exists(Path) Then
                                hs.writelog("Test","File exists")
                            Else
                                hs.writelog("Test","File does not exist")
                            End if
                            
                            End sub
                            Jon

                            Comment


                            • do i have to name the script something specific? I created a text file called jon00_test.vben and put it in the HS3 scripts directory...then created an event to run it. But I get this error when I run it:
                              Dec-06 1:04:29 PM Error Loading script C:\Program Files (x86)\HomeSeer HS3\scripts\jon00_test.vben :File not found: C:\Program Files (x86)\HomeSeer HS3\scripts\jon00_test.vben
                              Dec-06 1:04:29 PM Error Reading script: C:\Program Files (x86)\HomeSeer HS3\scripts\jon00_test.vben Script does not appear to be encrypted properly.

                              Comment


                              • Sorry, you can call it anything but it must have the extension .vb
                                Jon

                                Comment

                                Working...
                                X