Jon00 DataScraper/JSON Parser Script For Homeseer 3 and Homeseer 4

[ldragon99]

Do I need to put something in the device text1 and such. I get a blank device string.



-
Utility Jon00 DataScraper Bus Alerts Today
12:48:10 PM

[jon00]

Yes, you have to assign the data to the device.

Rather than going into the details, just copy the text below to the existing settings:

DeviceName1=Bus Alerts Today
DeviceText1=[0]

I would suggest you delete the device that has already been created and then it will be recreated with the name "Bus Alertts Today"

[ldragon99]

That did it. Of course I had to call the array, I was just not sure how it was labled. Thaks for pointing me int he right direction. I can play with it from there and get it to do the rest. Thanks for the help.

[Plane_Guy]

I'm trying to get rainfall for the day and the value comes out as zero. I'm able to get temp, wind and other values from the page without a problem. Here is the value I'm after:

<precip_today_in>1.73</precip_today_in>

Here are the values it's pulling:

[Grab1Data]
0=Last Updated on December 22, 10:03 AM EST
100=60.1
200=Calm
300=37
400=0.00

And the code I'm using:

Pattern1=(?s)observation_time>(.*?)</observation_time>
Pattern2=(?s)<temp_f>(.*?)</temp_f>
Pattern3=(?s)<wind_string>(.*?)</wind_string>
Pattern4=(?s)<solar_radiation>(.*?)</solar_radiation>
Pattern5=(?s)<precip_today_in>(.*?)</precip_today_in>

Any suggestions?

Here is the full url:
http://api.wunderground.com/weathers...p?ID=KNCINDIA5

[jon00]

I just tried it here with your settings and it works fine with the URL you posted. There is no solar but there is no solar in the XML file either.

[Plane_Guy]

I noticed I had the wrong URL.

Try this one and let me know:

http://api.wunderground.com/weathers...p?ID=KNCMINTH8

[Plane_Guy]

Like you said it may be working? I just pulled up the URL and the rain is now showing zero? Don't know what is going on with the station!

[jon00]

I've just tried it with the new URL and it is working here.

Do you intend to pull more than 5 metrics from this web page?

[Plane_Guy]

I probably will when I get mine up and running. Just playing around for now. Another question for you. Is there a way to scale an Icon to match others on the page? I tried adding an icon on a couple of them but they are 3 times the size.

[jon00]

If you do, let me know.

You will have to resize the images manually to 32 X 32. There are plenty on online image resize sites if you don't have a graphics editor yourself.

Example: http://www.picresize.com/

[Plane_Guy]

Thanks, I was in the process of re-sizing them but thought I would ask. I thought I read there was a max of 100 patterns. Is there something else I need to do if I want to add more?

[jon00]

There is a maximum of 5 patterns and 100 devices.

To obtain more metrics, you can do something like:

Pattern1=(?s)observation_time>(.*?)</observation_time>.*?<temp_f>(.*?)</temp_f>.*?<wind_string>(.*?)</wind_string> 

That said, you must select entries which are further down the page from the previous metric you are extracting.

[rmies1]

Not quite Sure How I woudl scrape thsi one Your guidance is appreciated

Jon, Merry Christmas

Trying to scrape Status from an APC Internal Website any guidance in how I would do this is appreciated.
The parts I am looking for are as follows:
<td width="34%" class="dataName"><span id="langLastBatteryXfer">Last Battery Transfer</span>:</td> <td colspan="3">Detection of distorted input</td> </tr> <tr> <td class="dataName"><span id="langInternalTemp">Internal Temperature</span>:</td> <td colspan="3">82.9&deg;F</td> </tr> <tr> <td class="dataName"><span id="langRuntimeRemaining">Runtime Remaining</span>:</td> <td colspan="3"> 2hr 48min</td> </tr> <tr> <td class="dataSubSection"><span id="langUPSInput">UPS Input</span></td> <td width="17%">&nbsp;</td> <td colspan="2">&nbsp;</td>

and I have
modified the INI to look like

Sample Grab1

[Grab1]
Path=http://10.0.1.7/NMC/PeO6rBiT89MOwmw+Mzh9vQ/ulstat.htm
TextFile=0
Encoding=
Username=apc
Password=apc
Options=
UserAgent=
Devicemode=0
StripHTML=1

Pattern1=(?s)<td width="34%" class="dataName"><span id="langLastBatteryXfer">Last Battery Transfer</span>:</td><td colspan="3">.*?</td>(.*?)</td>
Pattern2=(?s)<td class="dataName"><span id="langInternalTemp">Internal Temperature</span>:</td><td colspan="3">.*?F</td>(.*?);F</td>
Pattern3=(?s)<td class="dataName"><span id="langRuntimeRemaining">Runtime Remaining</span>:</td><td colspan="3">.*?</td>(.*?)</td>
Pattern4=(?s)<td class="dataName"><span id="langInputValtage">Input Voltage</span>:</td><td colspan="2">.*?;<span id="langVAC">VAC</span></td>(.*?);<span id="langVAC">VAC</span></td>
Pattern5=(?s)<td class="dataName"><span id="langOutputVoltage">Output Voltage</span>:</td><td colspan="2">.*?;<span id="langVAC">VAC</span></td>(.*?);<span id="langVAC">VAC</span></td>
DeviceName1=Wayne Hills UPS 1 Last Alarm Reported
DeviceText1=
DeviceValue1=[0]
DeviceImage1=
Speakbutton1=1
TriggerString1=
SearchMode1=1
TriggerEvent1=

DeviceName2=UPS Operating Temperature
DeviceText2=
DeviceValue2=[0]
DeviceImage2=
Speakbutton2=1
TriggerString2=
SearchMode2=1
TriggerEvent2=

DeviceName3=UPS Battery Run Time Remaing
DeviceText3=
DeviceValue3=[0]
DeviceImage3=
Speakbutton3=1
TriggerString3=
SearchMode3=1
TriggerEvent3=

DeviceName4=UPS Utility Voltage Reading
DeviceText4=
DeviceValue4=[0]
DeviceImage4=
Speakbutton4=1
TriggerString4=
SearchMode4=1
TriggerEvent4=

DeviceName5=UPS Output Voltage Reading
DeviceText5=
DeviceValue5=[0]
DeviceImage5=
Speakbutton5=1
TriggerString5=
SearchMode5=1
TriggerEvent5=

[jon00]

Hi,

Happy Christmas to you too!

I'm not quite sure what your question is?

For the first 5 patterns you have defined, do you see the data under [Grab1Data] in the Jon00DataScraperData.ini file?

[Malosa]

Hi all , and merry christmass !

i maybe can help also ,

i think why it doesnt scrape is because its to long , thy this one

Pattern1=(?s)<td width="34%" class="dataName"><span id="langLastBatteryXfer">.*?</td>(.*?)</td>
Pattern2=(?s)<td class="dataName"><span id="langInternalTemp">.*?</td>(.*?);F</td>
Pattern3=(?s)<td class="dataName"><span id="langRuntimeRemaining">.*?</td>(.*?)</td>
Pattern4=(?s)<td class="dataName"><span id="langInputValtage">.*?</td>(.*?);<span id="langVAC">VAC</span></td>
Pattern5=(?s)<td class="dataName"><span id="langOutputVoltage">.*?</td>(.*?);<span id="langVAC">VAC</span></td>