Out of Memory - Solved!

[PhilCamH]

We'll try this again:

After a record 7 days of uptime, HS3 crashed again today. Here's how it starts in the log (oldest at bottom):

May-27 12:18:20 Warning Starting application regsvr32.exe :Exception of type 'System.OutOfMemoryException' was thrown.
May-27 12:18:20 System Registering C:\WINDOWS\sysWOW64\msscript.ocx with Windows
May-27 12:18:18 Error Launching application: regsvr32.exe->Exception of type 'System.OutOfMemoryException' was thrown.
May-27 12:18:18 System Registering C:\WINDOWS\system32\msscript.ocx with Windows
May-27 12:18:12 Error Running script: Exception of type 'System.OutOfMemoryException' was thrown.-> at System.Threading.Thread.StartInternal(IPrincipal principal, StackCrawlMark& stackMark) at System.Threading.Thread.Start(StackCrawlMark& stackMark) at System.Threading.Thread.Start() at A.c1b0f37609f28fb7afed8d440680e6db4.ce86a0a55d3661fa218b566b 5ba926ee0(String ceefe476d1f77adc790c7782141798e1f, Boolean ce791fe9d8290e0d18cd0af1e42db139a, Int32 c3fda549ce0b74c9ad9fc210b529b2226, String c0cf6c15fc505ce256a93afdc5f2268a9, Boolean cc5b1dd80b9769b83d5de72520f2500e0, String c18f60d71139b909e1bc745dcca77e3f9, String cc463bfced87bf099c3125a86fe30843e, Int32 c135c28f517be83dec993779636acd860, String cdba5e903fed69597b9572ece7ed2b748, Object c70fb72fe0bbd8641a8134e71b176a648, Int32 caafe238b1576633e332c6c566f5707f4)
May-27 12:18:09 Error Running script '&hs.writelog "Motion Event", "Motion in Living Room..."', cannot access the scripting engine 'msscript.ocx', scripting will not function: Creating an instance of the COM component with CLSID {0E59F1D5-1FBE-11D0-8FF2-00A0D10038BC} from the IClassFactory failed due to the following error: 8007000e Not enough storage is available to complete this operation. (Exception from HRESULT: 0x8007000E (E_OUTOFMEMORY)).
May-27 12:18:07 Error Launching application: regsvr32.exe->Exception of type 'System.OutOfMemoryException' was thrown.
May-27 12:18:07 System Registering C:\WINDOWS\sysWOW64\msscript.ocx with Windows
May-27 12:18:05 Error Launching application: regsvr32.exe->Exception of type 'System.OutOfMemoryException' was thrown.
May-27 12:18:05 System Registering C:\WINDOWS\system32\msscript.ocx with Windows
May-27 12:17:51 Motion Event ...motion stopped in Front Yard.
May-27 12:17:39 Error Running script: Exception of type 'System.OutOfMemoryException' was thrown.-> at System.Threading.Thread.StartInternal(IPrincipal principal, StackCrawlMark& stackMark) at System.Threading.Thread.Start(StackCrawlMark& stackMark) at System.Threading.Thread.Start() at A.c1b0f37609f28fb7afed8d440680e6db4.ce86a0a55d3661fa218b566b 5ba926ee0(String ceefe476d1f77adc790c7782141798e1f, Boolean ce791fe9d8290e0d18cd0af1e42db139a, Int32 c3fda549ce0b74c9ad9fc210b529b2226, String c0cf6c15fc505ce256a93afdc5f2268a9, Boolean cc5b1dd80b9769b83d5de72520f2500e0, String c18f60d71139b909e1bc745dcca77e3f9, String cc463bfced87bf099c3125a86fe30843e, Int32 c135c28f517be83dec993779636acd860, String cdba5e903fed69597b9572ece7ed2b748, Object c70fb72fe0bbd8641a8134e71b176a648, Int32 caafe238b1576633e332c6c566f5707f4)


Past this, rinse and repeat until HS is restarted, an ocean of red in the log. This is a fast machine, working well for all other than HS, 3.3 GB RAM used, 3.6 GB available, 183 GB free disc space.

Ideas?

[sparkman]

Can you post the support profile from the help page? This will provide version numbers, etc. Also, on recent versions of HS3, the help page will show Available Threads. Does that number get smaller over the week that HS runs?

Cheers
Al

[sonypoolplr]

Also make sure none of your plugins are in debug mode because the log from these will fill up the computer fast.

[PhilCamH]

Thanks, guys. Sure, here's the info:

Current Date/Time: 2016-05-28 3:21:06 PM
HomeSeer Version: HS3 Standard Edition 3.0.0.280
Operating System: Microsoft Windows 10 Home - Work Station
System Uptime: 0 Days 22 Hours 21 Minutes 24 Seconds
IP Address: 172.16.1.71
Number of Devices: 96
Number of Events: 250
Available Threads: 400

Enabled Plug-Ins
3.0.0.75: HSTouch Server
1.1.0.3: JowiHue
3.0.0.50: weatherXML
3.0.0.36: X10

I'll keep an eye on it to see if the available threads decrease.

As to logging, it did appear that log errors only was enabled in weatherXML, and one option in JowiHue (Send log to console) was selected, though the other three options there were not. I've de-selected all.

[PhilCamH]

Again, just over four days this time. As always, gigabytes of available ram and disc space, computer running cool and fast. Checking the help file shows everything the same as last time, and still 400 threads available.

HS3 simply doesn't seem to run unattended for very long, meaning we'll likely need to rollback to version 1.7 before we go on vacation. Does anyone have any idea what the problem could be? I can't believe no one else has encountered this.

Also, is there anywhere to download previous versions of HS3, or of the plugins, in case of of these is causing the problem? After all these months, I'm willing to try anything.

[cerjzc]

Can you check the size of the database .hsd file and also the size of your Energy.hsd. Another person had an issue with HS3 running out of memory after a number of days and we found that their Energy.hsd had become corrupted and was several gb in size. They disabled it for the time being and the issues stopped.

jeff

[gd1210]

Are you using any scripts in HS?



Sent from my iPhone using Tapatalk

[jon00]

Again, just over four days this time. As always, gigabytes of available ram and disc space, computer running cool and fast. Checking the help file shows everything the same as last time, and still 400 threads available.

HS3 simply doesn't seem to run unattended for very long, meaning we'll likely need to rollback to version 1.7 before we go on vacation. Does anyone have any idea what the problem could be? I can't believe no one else has encountered this.

Also, is there anywhere to download previous versions of HS3, or of the plugins, in case of of these is causing the problem? After all these months, I'm willing to try anything.

You could try installing my Performance monitor for HS3 to see if that picks up anything unusual.

Also how many .txt (VBScript) scripts do you run?

[Rupp]

Check the Windows event viewer for any errors when this error occurs.

[PhilCamH]

Thanks again, so much, for all the help. Currently HS has not hit that snag yet, having been restarted yesterday, but here's the scoop so far. (I'll post the numbers here now, and again when it next starts to lock up.)

cerjzc -- The database HSD file is currently 1.64 MB in size, after running for almost 23 hours. Energy.hsd is 23 KB (I haven't started to explore this feature yet). HomeSeerLog.hsd is 9.51 MB.

gd1210, jon00 -- Yes, I have 68 scripts in my Scripts folder. 18 have the .cs extension, there are a couple of legacy .txt ones, the remaining 47 are .vb. (Of these last, most could be probably served by executing a single line of code in events, eg, &hs.writelog("Motion Event","Motion in Kitchen...").) Since switching to HS3 for 1.7, I've found events to be just as easy to use for a lot of what I want to do.

I've not yet used Performance Monitor for HS, but I'll definitely give it a try.

Rupp -- I've checked all the Event viewers (System, Security, etc) for clues each time this has happened. They all remain stubbornly adamant that nothing much at all was happening at the time. I did think at one point that it might have something with MS's "Oh yes, you will" involuntary Win 10 updates, but experimentation seems to have disproved that theory.

Finally, here's the current situation so far in this restart. Note that I've discontinued HSTouch plugin now, again, as part of my ongoing process of elimination:


Current Date/Time: 2016-06-01 10:29:37 PM
HomeSeer Version: HS3 Standard Edition 3.0.0.280
Operating System: Microsoft Windows 10 Home - Work Station
System Uptime: 0 Days 23 Hours 39 Minutes 30 Seconds
IP Address: 172.16.1.71
Number of Devices: 100
Number of Events: 250
Available Threads: 400

Enabled Plug-Ins
1.1.0.3: JowiHue
3.0.0.50: weatherXML
3.0.0.36: X10

The search continues...

[jon00]

The fact that the out of memory errors are related to msscript.ocx and that you are one of a few users running .cs scripts, my hunch is that there could be a memory leak in compiling C# scripts. It may be that memory resources are not being released/reused each time a CS script is run.

[gd1210]

I would hazard a guess that there is as Jon says a memory leak in .cs scripts or maybe you have a script that gets stuck in a continuous loop or something like that.

Graeme


Sent from my iPhone using Tapatalk

[PhilCamH]

Interesting. I'll review my scripts, possibly even get rid of one or two. I think I'll also enable logging for these to see if any are running when this happens.

[jon00]

I would still recommend you install my performance monitor. If it is a memory leak, it will show as an incremental chart on the metrics.

[PhilCamH]

jon00, where do I find your performance monitor? Is it a plug-in? I'd like to give it a try.

I only got about a day or so this time, before HS3 locked up again. After it does, the log fills with errors as in my first example, and sounds no longer function. Unfortunately, I was unable to check the data in the help file, as when I tried, HS crashed completely. I have determined at least that HS Touch Server is not at fault, as I'd disabled it this time.

I am able to tell you that HomeSeerLog.hsd was 9.51 MB in size, Energy.hsd was 23 KB, and the main database file was 1.64 MB in size.

Windows Event Viewer shows nothing in the System log at the time this behaviour began, nor does the Application log. However, when I checked HS this morning -- and it crashed the rest of the way -- it registered the following:

Application: HS3.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.AccessViolationException
at HS3.FrmError.InitializeComponent()
at HS3.FrmError..ctor()
at HS3.Form1.MYThreadHandler(System.Object, System.Threading.ThreadExceptionEventArgs)
at System.Windows.Forms.Application+ThreadContext.OnThreadExcep tion(System.Exception)
at System.Windows.Forms.Control.WndProcException(System.Excepti on)
at System.Windows.Forms.Control+ControlNativeWindow.OnThreadExc eption(System.Exception)
at System.Windows.Forms.NativeWindow.Callback(IntPtr, Int32, IntPtr, IntPtr)

Faulting application name: HS3.exe, version: 3.0.0.280, time stamp: 0x570d2125
Faulting module name: clrjit.dll, version: 4.6.1080.0, time stamp: 0x570c5032
Exception code: 0xc000041d
Fault offset: 0x0000be0b
Faulting process id: 0x1da8
Faulting application start time: 0x01d1bbc110ee25eb
Faulting application path: C:\Program Files (x86)\HomeSeer HS3\HS3.exe
Faulting module path: C:\Windows\Microsoft.NET\Framework\v4.0.30319\clrjit.dll
Report Id: 541ff39e-865b-4d3c-9a09-6944641423e2
Faulting package full name:
Faulting package-relative application ID:

My next step will be to disable various scripts in order to try and narrow this down further.