General Health Monitoring

[SteveMSJ]

I have had many requests to extend SDJ-Health beyond battery device monitoring which I have resisted up until now. I've always seen this as a separate issue and I didn't want to dilute or interfere with the fundamental philosophy of what started out as a fairly simple plug-in, albeit one that has become considerably more complex over the years.

I have now produced a 'Proof of Concept' update to SDJ-Health as a Beta. This tests out General Health monitoring without interfering with the Battery Device monitoring which was the pi's primary role.

If anybody wants to give it a go and let me have their feedback, it is version 3.0.7.6 in the Beta section of the updater. The General Health monitoring is kept separate to the Battery Health monitoring so running the beta should not interfere in any way with the current operation of the plug-in, however it is a beta so you are warned. Currently it doesn't create any devices or alerts, so is for testing the way it operates rather than an implementation.

The main guide has not been updated but there is a document attached to this thread which explains the basics of how it works. Respond to this thread with any observations, suggestions.

EDIT Document removed as now superseded

Steve

[The Profit]

Hi Steve, I read through the documentation and understand that the plugin will now Scan general interfaces/devices but I’m confused, is the purpose of doing this to determine if a plugin has stopped working? If so, will the PI eventually shutdown and restart the plugin via events?

I’m curious about the polling feature, I’ve read many posts about setting up z-wave devices to manually poll (HH/mm/ss) and that it can be problematic. I can see that you have many user defined options to control/limit/prevent polling but I’m wondering how a z-wave poll could potentially affect a z-wave system if all powered z-wave devices on a network were polled at once?

If I just happened to arrive home right at the time that I had set the plugin to perform a z-wave poll and a few of my “Arriving home” events trigger and send out z-wave commands, is it possible that a poll of all z-wave powered devices could cause a delay?


Sent from my iPad using Tapatalk

[SteveMSJ]

Hi Steve, I read through the documentation and understand that the plugin will now Scan general interfaces/devices but I’m confused, is the purpose of doing this to determine if a plugin has stopped working? If so, will the PI eventually shutdown and restart the plugin via events?

I’m curious about the polling feature, I’ve read many posts about setting up z-wave devices to manually poll (HH/mm/ss) and that it can be problematic. I can see that you have many user defined options to control/limit/prevent polling but I’m wondering how a z-wave poll could potentially affect a z-wave system if all powered z-wave devices on a network were polled at once?

If I just happened to arrive home right at the time that I had set the plugin to perform a z-wave poll and a few of my “Arriving home” events trigger and send out z-wave commands, is it possible that a poll of all z-wave powered devices could cause a delay?

Well, the purpose is more to determine if devices have stopped working rather than a plug-in, although obviously it would show if a plug-in had stopped. An option of automatically re-starting a plug-in in that situation might be a nice idea. I'll put that on the list of things to consider.

Personally I don't generally have problems with powered devices or plug-ins stopping working but others apparently do, hence the requests to expand the scope of the pi beyond battery devices. I can see the use of it as long as it doesn't impose any significant overheads on the system. This is the reason for testing out the concept.

The activity scan imposes very little demands but, as you say, Z-Wave polling can impact a system. Polling is optional and anyway it checks activity first and only polls devices that have not passed the activity scan so usually only a few devices. Setting a pause between polls also reduces the impact as it help prevent other commands queuing. Also setting the scan to happen at say 2 o'clock in the morning is unlikely to cause any issues.

You can start the scan manually so it would be a useful test to set it running and then trigger an intensive event and see what impact it has.

The polling of inactive devices can be aborted part way through. This is a button at the moment but one option might be to have it abort automatically under certain conditions.

Anyway, the purpose of this beta is to see whether it is of any use, whether it causes any issues and what suggestions people have for how it should work.

Steve

[SteveMSJ]

I have had many requests to extend SDJ-Health beyond battery device monitoring which I have resisted up until now. I've always seen this as a separate issue and I didn't want to dilute or interfere with the fundamental philosophy of what started out as a fairly simple plug-in, albeit one that has become considerably more complex over the years.

I've developed this Beta version of the plug-in quite a bit over recent weeks and have now released an updated version 3.0.7.7 in the beta section of the updater. Again the changes are to the new General Health Monitoring which should not interfere with the Battery Health Monitoring which has always been, and will remain, the main focus of the plug-in.

This is now pretty much feature complete and works the way I intend it to. It is not intended for the casual user as it requires a reasonable knowledge of devices in HS to be able to interpret the results. The main changes since the last beta, where I introduced the proof of concept, are:

• Scanning for Invalid State devices has been added in addition to Inactivity and Polling.
• A new group of monitoring devices can be created to represent the Interfaces on your system that you whish to monitor.
• Reports are created in html format for better presentation and so that they can be viewed directly from the HS web interface as well as being emailed or pushed by other notification systems.
• The guide document for the plug-in has been updated to cover the new monitoring of General Devices in addition to Battery Devices.

The guide is installed in the /docs folder of HS3 and is also available for download in the following thread:
https://forums.homeseer.com/forum/ul...j/104921-guide

Scans of devices in HS can be setup to run automatically and can trigger events in HS. Devices are grouped and configured by the interface (plug-in) they belong to and the type of scanning configured per interface. Two reports are produced, one showing a full results with passes and failures, the other just showing summaries and failures.

A couple of partial screenshots from a report are shown below for example:






Screenshot of General Health Group of monitoring devices:


Any questions or comments please post here.

Steve

[SteveMSJ]

Beta 3.0.8.1 adds a new option with polling called PollDepth.


At the default of 1 the pi just polls one device in each group. If set to greater than 1 then if the first device fails a poll it will continue to poll other devices in the group, representing a physical device, until a successful result is received or the PollDepth is exceeded. Remember that, only inactive devices are polled and only if Polling is selected as an option.

The reason for this is that I have found very occasionally some Z-Wave devices don't respond to the first poll but do to a subsequent one. Almost as if the need a nudge into action if they have been silent for a while. It doesn't normally add any extra overhead because as long as an inactive device responds to the first poll attempt the pi will move onto the next physical device. If a device is dead then the polling will take longer as it will retry other devices in the group until the PollDepth is exceeded.

Steve

[SteveMSJ]

I've unstuck this thread as the General Devices features are now part of the main plug-in and the guide has been updated to incorporate them.

Steve

[alphatech]

Steve, great addition to your plugin. I have a request if possible. Right now I have hundred of events that check and monitor almost of my devices for time last update. These events check for all devices including non battery operated. I would normally check if the weather was updated, if the temperature sensor was updated and etc. How difficult would it be to implement into your plugin to select whatever devices you want to monitor and the ability to send an email?

Thanks

[SteveMSJ]

Steve, great addition to your plugin. I have a request if possible. Right now I have hundred of events that check and monitor almost of my devices for time last update. These events check for all devices including non battery operated. I would normally check if the weather was updated, if the temperature sensor was updated and etc. How difficult would it be to implement into your plugin to select whatever devices you want to monitor and the ability to send an email?

Thanks

The way the General Devices (which can include battery devices) features work is by Interface/Plug-in. So you can monitor all devices that are owned by a particular plug-in and configure the inactivity period for that interface. The plug-in can create a monitoring device for each interface and you can trigger events off that monitoring child and/or off the parent monitoring device if you so desire. After a scan the value of each monitoring device is set to the number of failures and the device string shows a brief summary. You can trigger an event off the value of the monitoring device. You can set the plug-in to run a full scan at a specific time each day or you can scan individual interfaces from events using the Scan action on the monitoring device for the interface at any time or interval you want.

You can't specifically set it to only scan particular devices, but you can set it to scan particular interfaces and then you can exclude individual devices from that interface. This way you don't have to reconfigure monitoring as you add or remove devices from HS unless you add or remove a plug-in. This works well for me. You could easily set up an event to send you an email and you could attached the html reports to the email.

The way I use it is to run a full scan at 2 o'clock in the morning and send me a report via pushover. At any time, from HSTouch or from the HS web pages, I can view a Full or Failure report showing me the status of all the devices scanned with any problem devices highlighted. I can also trigger a new scan if I want.

I don't know if you have got as far as generating the html reports, but they give you a pretty good indication of the state of the devices on your system. However, if you have some specific issues that you are trying to pick up I'm open to ideas.

Of course this is all in addition to monitoring and alerting on the state of battery devices which is more detailed and operates as previously for the plug-in.

Steve

[alphatech]

Thanks for the help Steve. Few observations and minor issues.
I have several devices that do not use a plugin. I use a script to update those value. I would like to monitor those values.
Under Select Devices to Exclude, it does not show any devices.
Under Display reports: I get this error Apr-15 11:33:33 AM Warning File does not exist: C:\HomeSeer3\html\SDJ-Health\SDJ-Health General Report-1.html
with a bad request: same with the other report. 404 Bad Request: /SDJ-Health/SDJ-Health General Report-0.html

Thanks

[SteveMSJ]

Thanks for the help Steve. Few observations and minor issues.
I have several devices that do not use a plugin. I use a script to update those value. I would like to monitor those values.

Hi alphatech,

It was a conscious decision to exclude non plug-in devices but maybe I should change that. Out of interest in what circumstances do you see a virtual device being dormant indicating an issue?

What I could do would be to have a 'No Interface' section that would pick up all devices not owned by a plug-in. On my system this would be a few hundred devices, but that doesn't really matter as I would select them not to be monitored. I'll look into this.

Another thought would be that if you create your virtual devices with a script you could give them your own interface name in which case they would show up in their own section.

Under Select Devices to Exclude, it does not show any devices.

After each scan any devices that failed are added to this drop down multi select list, so you have the option to exclude them from future scans. If you have run a scan and picked up some failed devices and this drop list is empty then there is something wrong. Try turning debug on by setting LogLevel=2 then run a scan and see what SDJ-Health messages show up in the log.

You can actually manually add devices to be excluded by editing the SDJ-Health.ini file. In the [General Health Section] you can add device ref numbers to the ExclusionList parameter as per the example below. You would need to do this with the SDJ-Health plug-in disabled otherwise it will overwrite it.

[General Health]
ExclusionList=868,2219,2124,1784

They will then be added to the drop down list on startup and remain in the list, unless you deselect them.

Under Display reports: I get this error Apr-15 11:33:33 AM Warning File does not exist: C:\HomeSeer3\html\SDJ-Health\SDJ-Health General Report-1.html
with a bad request: same with the other report. 404 Bad Request: /SDJ-Health/SDJ-Health General Report-0.html

Thanks

Are you on Linux? Looks like I've hard coded back slashes into the path.

If not then can you have a look in that folder on your system and see if there are any report files?
If still nothing can you run a scan after setting LogLevel=2, filter your log for SDJ-Health messages and paste it here.

Thanks,
Steve

[alphatech]

Steve, either one would work, ideally the selection to get all the devices would be great. If is too much work, please let me know how would I add my own interface in a script, I normally use scripts that are already built since I'm not good at it


Running Windows, I do not see any reports. Not sure if this helps, my HS is on C Homeseer3 directory

[SteveMSJ]

Running Windows, I do not see any reports. Not sure if this helps, my HS is on C Homeseer3 directory

Can you run a scan with LogLevel=2 and see if it gives a clue as to why no reports are produced.

[alphatech]

Good morning Steve, I went and check this morning and the reports are there.
Thanks for all your help.

[SteveMSJ]

Good morning Steve, I went and check this morning and the reports are there.
Thanks for all your help.

Good. Thanks for the update.
You are aware that you can trigger a scan at any time by using the Run button in the Now column on the General Devices Config page, or indeed from an event using the control on the General Health Root device. You don't have to wait for the timed scan.

Steve

[SteveMSJ]

Steve, either one would work, ideally the selection to get all the devices would be great. If is too much work, please let me know how would I add my own interface in a script, I normally use scripts that are already built since I'm not good at it

I've added an interface called 'No Interface' which includes all the devices not owned by a plug-in. Version 3.0.8.4. in the beta section of the updater.

Steve