A script to log device changes to InfluxDB for use with Grafana

[Guest]

I get this error for every device, it worked for 1 device, the first time, then this:
Any idea?

Error: System.Net.WebException: The operation has timed out at Microsoft.VisualBasic.CompilerServices.Symbols.Container.Inv okeMethod(Method TargetProcedure, Object[] Arguments, Boolean[] CopyBack, BindingFlags Flags) at Microsoft.VisualBasic.CompilerServices.NewLateBinding.Object LateGet(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack) at Microsoft.VisualBasic.CompilerServices.NewLateBinding.LateGe t(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack) at scriptcode9.VBWrapper.Main(Object Parms)

[Rob2791]

I get this error for every device, it worked for 1 device, the first time, then this:
Any idea?

Error: System.Net.WebException: The operation has timed out at Microsoft.VisualBasic.CompilerServices.Symbols.Container.Inv okeMethod(Method TargetProcedure, Object[] Arguments, Boolean[] CopyBack, BindingFlags Flags) at Microsoft.VisualBasic.CompilerServices.NewLateBinding.Object LateGet(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack) at Microsoft.VisualBasic.CompilerServices.NewLateBinding.LateGe t(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack) at scriptcode9.VBWrapper.Main(Object Parms)

I'm getting the same thing, it timing out after 2 mintues. It works like once every 100 tries. I can't find any rhyme or reason. All of a sudden it seems to start working for a couple hours then quit again. If it works i get one error in the log, if it fails i get two error lines in the log.

[lveatch]

Error: System.Net.WebException: The operation has timed out at Microsoft.VisualBasic.CompilerServices.Symbols.Container.Inv okeMethod(Method TargetProcedure, Object[] Arguments, Boolean[] CopyBack, BindingFlags Flags) at Microsoft.VisualBasic.CompilerServices.NewLateBinding.Object LateGet(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack) at Microsoft.VisualBasic.CompilerServices.NewLateBinding.LateGe t(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack) at scriptcode9.VBWrapper.Main(Object Parms)

I don't use influx for homeseer, rather graylog, but I do use influx/telegraf/chronograf for my home infrastructure monitoring.

My thought is to remove the code from the script to call influxdb server. Instead, write the device information to a log file and let telegraf's tail or logparser plugins handle the communication to the influxdb.


I'm assuming you can write influx's line protocol to the log file and process it without much telegraf input configuration.

[mminehan]

Can someone post a copy of their working windows script (entire script) with the "allow" list instead of the "skip" list?

I can't seem to get mine working. Every time I put a device number in the list I get a series of errors.

Thanks, Marty.

[OliP]

Thank you for this script - been wanting to send data onto InfluxDB/Grafana for years.

Sadly im yet to get it working. Im getting a 404 error, suggesting it cant find the db. The db exists, and the servername is localhost. Other agents on my network are writing to it fine.

Any ideas?

Error: System.Net.WebException: The remote server returned an error: (404) Not Found. at Microsoft.VisualBasic.CompilerServices.Symbols.Container.InvokeMethod(Method TargetProcedure, Object[] Arguments, Boolean[] CopyBack, BindingFlags Flags) at Microsoft.VisualBasic.CompilerServices.NewLateBinding.ObjectLateGet(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack) at Microsoft.VisualBasic.CompilerServices.NewLateBinding.LateGet(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack) at scriptcode5.VBWrapper.Main(Object Parms)

[OliP]

OKay so the 404 was my fault, was writing to a db that didnt exist - server had ignored the config file on last boot.

I can now see metrics in my db - but my log is still full of errors.

Error: System.IO.FileNotFoundException: Could not find file 'C:\Program Files (x86)\HomeSeer HS3\System.Net.ConnectStream'. File name: 'C:\Program Files (x86)\HomeSeer HS3\System.Net.ConnectStream' at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) at System.IO.StreamReader..ctor(String path, Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize, Boolean checkHost) at System.IO.StreamReader..ctor(String path) at scriptcode4.VBWrapper.Main(Object Parms)

for example.
Going to let it run for a few hours, make my dashboards and see if errors continue.

Thanks again for an awesome script!

[mminehan]

Ok....for those of us less apt at coding, like me who fumbles their way through it, here is the section of code for the 'allow list' section:

Dim Allowed As Boolean = False

If ALLOW_LIST <> "" Then

Dim Allows As String() = ALLOW_LIST.Split(New Char() {","c})
If Allows.Contains(devRef) = True Then

Allowed = True
hs.WriteLog("L2DB-InfluxDB", "Allowed: " & devRef)

End If

End If
If Not Allowed Then Exit Sub

[ando1]

Thank you to all in this thread as I was able to get a working script and Grafana dashboard. It still seems I have an issue though as most of the things I am tracking like door sensors and temps from my ecobee thermostat stop working after some period of time. I looked in the log and this warning seems to start after this happens:

Dropping event callbacks due to full queue (Type: 2048) (500 entries), system may be too busy, plugins and HSTouch may not receive all device updates

I am using the allowed list and only have about 10 things I am monitoring. Anyone else seeing this?


UPDATE: I tried watching the logs to see how long this would work before breaking and it seems to only work for about an hour. I am monitoring 15 devices (motion sensors, ecobee temps, and door sensors). I traced back into the logs and found this error:

Error: System.Net.WebException: The operation has timed out at Microsoft.VisualBasic.CompilerServices.Symbols.Container.Inv okeMethod(Method TargetProcedure, Object[] Arguments, Boolean[] CopyBack, BindingFlags Flags) at Microsoft.VisualBasic.CompilerServices.NewLateBinding.Object LateGet(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack) at Microsoft.VisualBasic.CompilerServices.NewLateBinding.LateGe t(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack) at scriptcode2.VBWrapper.Main(Object Parms)


UPDATE 2: I was able to get this working by installing influxdb on my HS3 box and pointing the script to localhost instead of my remote Grafana/influx server. Has been working consistently now for 2 days!

[Rob2791]

Has it been stable?? I can't get mine stable. I had Influxdb on a separate Ubuntu VM. My Homeseer is Win7.

Thank you to all in this thread as I was able to get a working script and Grafana dashboard. It still seems I have an issue though as most of the things I am tracking like door sensors and temps from my ecobee thermostat stop working after some period of time. I looked in the log and this warning seems to start after this happens:

Dropping event callbacks due to full queue (Type: 2048) (500 entries), system may be too busy, plugins and HSTouch may not receive all device updates

I am using the allowed list and only have about 10 things I am monitoring. Anyone else seeing this?


UPDATE: I tried watching the logs to see how long this would work before breaking and it seems to only work for about an hour. I am monitoring 15 devices (motion sensors, ecobee temps, and door sensors). I traced back into the logs and found this error:

Error: System.Net.WebException: The operation has timed out at Microsoft.VisualBasic.CompilerServices.Symbols.Container.Inv okeMethod(Method TargetProcedure, Object[] Arguments, Boolean[] CopyBack, BindingFlags Flags) at Microsoft.VisualBasic.CompilerServices.NewLateBinding.Object LateGet(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack) at Microsoft.VisualBasic.CompilerServices.NewLateBinding.LateGe t(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack) at scriptcode2.VBWrapper.Main(Object Parms)


UPDATE 2: I was able to get this working by installing influxdb on my HS3 box and pointing the script to localhost instead of my remote Grafana/influx server. Has been working consistently now for 2 days!

[ando1]

@Rob2791 Yes it has been stable for the most part. I have added some more things like Humidity and HVAC status (shows when my AC runs) and it has all been running for a week solid. The only glitch I have seen is sometimes the humidity and/or the temps will stop reporting for a couple of hours but then restart on their own. I am not sure that is any fault of the script and my HS3 VM has plenty of horsepower.

Here is an updated screen cap of my dashboard:

The high CPU you see in the graph is due to the MeiHarmonyHub plugin. I just found this while getting the screen cap for this post!

[ando1]

And after some further investigation using Grafana, I was able to see that the CPU spike started yesterday around 4:00PM. Checked my logs as well and I have some errors. Time to head over to the HS forums and see if I can get some answers

[Rob2791]

What are you running HS3 on?

[ando1]

Rob. HS3 is running on a Win10 VM with 12GB RAM and quad core CPU. I had a similar setup you described earlier where I was running influx on a remote Ubuntu VM and sending the logs to it via the HS3 script. I had issues with the data just stopping and my fix was to install influx on the HS3 box.

[Rob2791]

I think i figured it out! By default there is a connection limit of 2. I up'd the connection limit to 20 and it's been working fine for 12 hours!
I added the following right under the httpWebRequest statements in the middle of the script.
"ServicePointManager.DefaultConnectionLimit = 20"

[mminehan]

I added the following right under the httpWebRequest statements in the middle of the script.
"ServicePointManager.DefaultConnectionLimit = 20"

Thanks Rob. I just did the same as per your suggestion and it seems to have done the trick!

My InfluxDB and Grafana are hosted on a Raspberry Pi and I was seeing the "full queue" error a couple of times a day.

This also reminded me of the TCP limitations in Windows (e.g. maximum of 20 external TCP connections for Windows 10). I then used SG TCP Optimizer from www.speedguide.net on my HS3 PC. It seems to have improved the responsiveness of the system. The device viewer loads much quicker for example when viewing remotely, and HS Touch is way less laggy. It has also speed up the remote access of my Blue Iris PC. It might be worth a look. You can backup all the settings before applying changes.

Cheers, Marty.