Jon00 DataScraper/JSON Parser Script For Homeseer 3 and Homeseer 4

[michaeldecharon]

Write data to existing device

Is it possible to write the data to an existing device?

And if so, how can i do that?

[jon00]

You mean an existing virtual device? No, that is not possible.

[michaeldecharon]

You mean an existing virtual device? No, that is not possible.

That's a shame, but thank you for youre answer

[n8pbm]

Just installed the script and want to use it to pull some xml data. I have it creating a virtual devise but i cannot get it to populate the value i need. The documentation talks about a Jon00DataScraperData.ini file but I cannot find it.

Here is the xml data i am using (Local) and it does create a Grab1 file with the information. I was trying to grab the 78.8 from the temp tags. I changed the pattern to the simple one i have below.

<status software="4.40_3A16" hardware="1.0">
<hostname>apex</hostname>
<serial>AC4:56381</serial>
<timezone>-5.00</timezone>
<date>01/01/2018 09:46:15</date>
<power>
<failed>12/30/2017 15:25:04</failed>
<restored>12/30/2017 15:27:52</restored>
</power>
<probes>
<probe>
<name>Temp</name>
<value>78.8</value>
<type>Temp</type>
</probe>
<probe>
<name>pH</name>
<value>8.24</value>
<type>pH</type>
</probe>
<probe>
<name>ORP</name>
<value>447</value>
<type>ORP</type>
</probe>
<probe>
<name>Amp_3</name>
<value>4.2</value>
<type>Amps</type>
</probe>
<probe>
<name>Tmpx4</name>
<value>23.4</value>
<type>Temp</type>
</probe>
<probe>
<name>Condx4</name>
<value>33.4</value>
<type>Cond</type>
</probe>
</probes>
<outlets>
<outlet>
<name>VarSpd1_I1</name>
<outputID>0</outputID>
<state>OFF</state>
<deviceID>base_Var1</deviceID>
</outlet>
<outlet>
<name>VarSpd2_I2</name>
<outputID>1</outputID>
<state>OFF</state>
<deviceID>base_Var2</deviceID>
</outlet>
<outlet>
<name>VarSpd3_I3</name>
<outputID>2</outputID>
<state>OFF</state>
<deviceID>base_Var3</deviceID>
</outlet>
<outlet>
<name>VarSpd4_I4</name>
<outputID>3</outputID>
<state>OFF</state>
<deviceID>base_Var4</deviceID>
</outlet>
<outlet>
<name>SndAlm_I6</name>
<outputID>4</outputID>
<state>OFF</state>
<deviceID>base_Alarm</deviceID>
</outlet>
<outlet>
<name>SndWrn_I7</name>
<outputID>5</outputID>
<state>OFF</state>
<deviceID>base_Warn</deviceID>
</outlet>
<outlet>
<name>EmailAlm_I5</name>
<outputID>6</outputID>
<state>OFF</state>
<deviceID>base_email</deviceID>
</outlet>
<outlet>
<name>Heater</name>
<outputID>7</outputID>
<state>AON</state>
<deviceID>3_1</deviceID>
</outlet>
<outlet>
<name>Hold_Heater</name>
<outputID>8</outputID>
<state>ON</state>
<deviceID>3_2</deviceID>
</outlet>
<outlet>
<name>Hold_Pump</name>
<outputID>9</outputID>
<state>ON</state>
<deviceID>3_3</deviceID>
</outlet>
<outlet>
<name>Skimmer</name>
<outputID>10</outputID>
<state>AON</state>
<deviceID>3_4</deviceID>
</outlet>
<outlet>
<name>WC_Pump</name>
<outputID>11</outputID>
<state>OFF</state>
<deviceID>3_5</deviceID>
</outlet>
<outlet>
<name>Sump_Light</name>
<outputID>12</outputID>
<state>AOF</state>
<deviceID>3_6</deviceID>
</outlet>
<outlet>
<name>CO2_3_7</name>
<outputID>13</outputID>
<state>OFF</state>
<deviceID>3_7</deviceID>
</outlet>
<outlet>
<name>Return_Pump</name>
<outputID>14</outputID>
<state>AON</state>
<deviceID>3_8</deviceID>
</outlet>
</outlets>
</status>

Here is my Jon00DataScraper.ini

Path=http://xxxxxxxxxxx/cgi-bin/status.xml
TextFile=1
Encoding=UTF-8
Username=
Password=
Options=
UserAgent=
Devicemode=0
StripHTML=0

Pattern1=(?s)<value>(.*?)</value>

DeviceName1=TankTemp
DeviceText1=[0]
DeviceValue1=[0]
DeviceImage1=
Speakbutton1=
TriggerString1=
SearchMode1=
TriggerEvent1=

Any ideas?

Thanks

[n8pbm]

I got it working. Not really sure what I did.

[travisdh]

Hi Jon,

Hoping you might be able to help, I am on my graph everything quest lol and wanted to try and graph some social media stuff. In theory my code works on the regex tester sites, but for the life of me I cant get it to work on this.

Any chance you could take a peek at it and let me know where I am going wrong, I had tried with and without the multiline regex prefix but no number (41) shows up which it should.

Many thanks!

[Grab1]
Path=https://plus.google.com/+TravHale
TextFile=0
Encoding=
Username=
Password=
Options=
UserAgent=
Devicemode=2
StripHTML=1

Pattern1=\d+(?= followers)

DeviceName1=Google+ Followers
DeviceText1=[0]
DeviceValue1=[0]
DeviceImage1=
Speakbutton1=
TriggerString1=
SearchMode1=
TriggerEvent1= 

[jon00]

Try:

Pattern1=(?s)<br>(.*?)followers</div>

[travisdh]

Try:

Pattern1=(?s)<br>(.*?)followers</div>

Thanks Jon,

Sent from my SM-T815Y using Tapatalk

[travisdh]

Thanks Jon,

Sent from my SM-T815Y using Tapatalk

Hi Jon,

I gave it a go, but had no luck. Looks like it is picking up html or javascript code from the page, with and without strip html

Thanks.

Sent from my SM-T815Y using Tapatalk

[jon00]

Set TextFile=1 and send me the file via email.

[deanrparry]

Struggling!

What are the chances of this being able to open a url that needs basic authentication type/json with accept header? so i can parse the result?

[jon00]

I can easily add the accept header and contentType to the settings. Is that what you want?

[deanrparry]

You would be a star I'd you could ;-) much appreciated

[deanrparry]

https://www.traccar.org/traccar-api/

That's my aim to scrape the results...

[jon00]

I've sent you a PM.