All events stop running, need help narrowing down problem for bug report

[mikee123]

I have been battling with this for a couple of weeks now. For no apparent reason, all events stop working. Manually triggering a event does not result in any action either.
Everything else still works.

I can control Zwave devices, status updates, also Nest, Rfxcom, Shelly, Rachio, Tasker, all work. Devices update and can be controlled.
I thought a update from 4.2.11.0 to 4.2.12.0 and subsequently 4.2.13.0 was the probelm.
So I uninstalled HS4, kept data, and reinstalled 4.2.11.0. A couple of days after starting HS4 events stopped again. I then updated to 4.2.13.0 again. And again, after a couple of days events stopped working. Most of the time after a couple of hours, everything is running again. There is nothing in the log, no errors. ll I can see in the log at the time is there is no events showing.
This is causing major problems, so I need to try to find the problem, and at the same time need more information so I can try and file a bug report, and hope for some help from HS...
I thought of triggering a automatic restart when this happens, but as no events will run, that is not an option...
Not sure what else I can do to narrow down the problem


Date/time=18/06/2022 11:12:36 GMT Summer Time
Version=HS4 Pro Edition 4.2.13.0 (Windows)
License=Registered
MyHS Subscription=Basic
MyHS Subscription Backup Interval=No Backup
MyHS Subscription Max Cameras=1
Confguration File=C:\Program Files (x86)\HomeSeer HS3\Data\HomeSeerData.json
In Virtual Machine=No MFG: lenovo
Antivirus=Windows Defender
OS=Microsoft Windows 10 Pro - Work Station
OS Version=10.0.19043
Uptime=0 Days 0 Hours 31 Minutes 53 Seconds
User/Access=xxx (Admin)
Device Count=1224
Event Count=658
Last Event Ran=System : Multi Watts display, Ran at 18/06/2022 11:12:38
# of unique event scripts=90
Processor:=AMD64 Family 23 Model 24 Stepping 1 at 1.30 GHz
Modules/Threads=136 Modules, 200 Threads
Available Threads=1023
System Load=264 Processes, 31% Load
Free/Total Memory=2.05 GBytes / 5.93 GBytes (35% free)
Free/Total Virtual Memory=4.86 GBytes / 12.44 GBytes (39% free)
HomeSeer Memory Used=151 Mbytes
Plugin Memory Used=24 EXE Plug-Ins using 766 Mbytes
Plugins Installed=AK Shelly 3.0.0.97,BLGData 3.0.55.0,BLLAN 2.0.45.0,BLShutdown 1.0.6.0,BLStat 2.0.87.0,Blue-Iris 3.1.3.33206,drhsIpPlugIn 0.0.0.25,EasyTrigger 3.0.0.74,Global Cache Pro 5.0.0.59,HSBuddy 3.28.1222.5,JowiHue 4.0.7.3,Kodi 3.0.1.14,Linktap 3.0.1.0,mcsMQTT 5.24.10.0,mcsShelly 5.24.10.0,Neato Botvac 4.0.9.1,Nest 3.0.0.34,NetCAM 3.0.0.14,PHLocation2 3.0.0.64,Pushover 3P 0.0.0.54,Rachio 4.0.2.0,RFXCOM 30.0.0.41,Sonos 3.1.0.57,Tasker Plugin 3.0.4.9,UltraLog3 3.0.6644.26753,Z-Wave 3.0.10.0

[pistacheL0]

I am having the same issue since 1 year, BUT on LINUX****

This was me with last account https://forums.homeseer.com/forum/ho...ems-to-be-down

[pistacheL0]

Here, I have narowed down the issue to hs4 event system.

I have tested this with only easy trigger enabled. There problem didn’t appear. But easytrigger plugin was to much used in my events.

I recently transformed all the logic to native hs /device/events….so no more easytrigger dependency to run my automation. (That was a long task).

But the problem came back. There is something in homeseer code that is causing this. Some threads become locked for wtv reasons I don’t understand.


rjh




What can/could cause (or maybe interact with) the event engine to lock?

[rjh]

I would love to get this fixed for you, but I cannot reproduce it. I think I have mentioned this before, but please turn on event logging and post the log when things stop working and the event messages stop. To turn on event logging go to the control panel from the tools menu and enter:

hs.debugmode=2

Check the log and you should see a message about event logging being enabled.

The issue must be related to either a plugin or one of your event configurations.

I assume you don't see any errors in the log?

Here, I have narowed down the issue to hs4 event system.

I have tested this with only easy trigger enabled. There problem didn’t appear. But easytrigger plugin was to much used in my events.

I recently transformed all the logic to native hs /device/events….so no more easytrigger dependency to run my automation. (That was a long task).

But the problem came back. There is something in homeseer code that is causing this. Some threads become locked for wtv reasons I don’t understand.


rjh




What can/could cause (or maybe interact with) the event engine to lock?

[mikee123]

For me, the issue might be fixed. I sent a bug report HSCS-14870 and Matt had a look at my system. I had events which ran every few seconds to update a few virtual devices. He said HS4 was not meant to do that and reduce it to every 5 minutes instead of every few seconds. Although I did not think that would be the issue, as I had this running for years without any issue, I added a do not re-run for 20 seconds to those events. I had 7 days running now without this issue. Very strange that the system managed to run for years without issue, but having the events to update the virtual devices less might have done it.
When I have time I might get rid of the do not re-run for 20 seconds and enable event logging to see if I get any errors.

[John245]

I have events run with a interval of 10 seconds without issues.

---
John

[mikee123]

I have events run with a interval of 10 seconds without issues.

---
John

Well mine ran for years without issues. I have 2 virtual device which displays energy consumption of a few devices. These update every couple of seconds (whenever a value of any of theses underlying devices change). The second one is in screenshot 2. Temperature devices, TRV setpoints, again only update when there is a change, so maybe every minute. The weather device at the bottom, that changes every minute or maybe a little more often. I did not think that was excessive...

[rjh]

I am posting beta 4.2.13.4 shortly. I will check all the event related threads and add some logging in the case they exit unexpectedly. (and make sure they restart)