Announcement

Collapse
No announcement yet.

Increasing Handles and HS Server crashes

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Monk
    replied
    Have not -
    I had one script running every 5 minutes that I disabled on 2/23 - (data scraper). I don't plan any other changes until some more time passes.

    Leave a comment:


  • sparkman
    replied
    Originally posted by Monk View Post
    I'm not sure if this is helpful, sparkman

    The start around 2/12 was HS release version .500
    The start at 3/12 is with Beta .521
    The incline isn't so steep with this version - all else remains the same.
    Thanks Monk, I'll give the new version a try this weekend. Below is the overlap between your plugins and mine. Have you done any experimenting with those to see which, if any of those, make a difference on the handle leak?

    1.0.66.0: BLLED
    0.0.0.45: Pushover 3P
    3.0.6644.26753: UltraLog3
    3.0.6413.20219: UltraNetCam3
    3.0.1.259: Z-Wave

    Leave a comment:


  • Monk
    replied
    I'm not sure if this is helpful, sparkman

    The start around 2/12 was HS release version .500
    The start at 3/12 is with Beta .521
    The incline isn't so steep with this version - all else remains the same.
    Click image for larger version

Name:	20180319-handles.PNG
Views:	43
Size:	18.4 KB
ID:	1293627

    Current Date/Time: 3/19/2019 5:31:08 PM
    HomeSeer Version: HS3 Pro Edition 3.0.0.521
    Operating System: Microsoft Windows 7 Professional - Work Station
    System Uptime: 7 Days 2 Hours 16 Minutes 28 Seconds
    Number of Devices: 671
    Number of Events: 149
    Available Threads: 400
    HSTouch Enabled: True
    Event Threads: 3
    Event Trigger Eval Queue: 0
    Event Trigger Priority Eval Queue: 0
    Device Exec Queue: 0
    HSTouch Event Queue: 0
    Email Send Queue: 0
    Anti Virus Installed: Microsoft Security Essentials
    In Virtual Machine: No MFG: asustek computer inc.
    Enabled Plug-Ins
    2.1.0.34: AmbientWeather
    2.0.59.0: BLBackup
    1.0.66.0: BLLED
    2.0.76.0: BLSpeech
    1.3.7.0: Device History
    3.0.0.33: iTunesDAAP
    3.0.0.14: NetCAM
    0.0.0.45: Pushover 3P
    3.0.6681.34300: UltraCID3
    3.0.6644.26753: UltraLog3
    3.0.6922.16073: UltraM1G3
    3.0.6554.33094: UltraMon3
    3.0.6413.20219: UltraNetCam3
    3.0.6358.20184: UltraRussoundAudio3
    3.0.1.259: Z-Wave

    Leave a comment:


  • sparkman
    replied
    Originally posted by prsmith777 View Post
    Did we ever come up with an answer to this question? I too have a slowly growing handle problem.

    HS.GetURL was questioned. I have several scripts using this to control Hunter Douglas Drapes. I fired off the scripts about 30 times in a few minutes and didn't see any change in handles.
    I have not been able to narrow it down. It seemed like I could get my handles stable for a while with certain plugins disabled, but then a few hours later, they would start increasing again. I gave up on troubleshooting it as there was too much impact on my automation routines by disabling the plugins and then waiting hours to see what the impact would be.

    Leave a comment:


  • prsmith777
    replied
    Did we ever come up with an answer to this question? I too have a slowly growing handle problem.

    HS.GetURL was questioned. I have several scripts using this to control Hunter Douglas Drapes. I fired off the scripts about 30 times in a few minutes and didn't see any change in handles.

    Leave a comment:


  • bsobel
    replied
    Originally posted by sparkman View Post

    Great, thanks Bill. So for every thread that HS3 starts, a handle would also be created and since the actual threads are not increasing, it's strictly the handle to the thread that is not properly disposed of. Since I have a lot of scripts, are there things I should be doing in each script to manage that, or should that all be taken care of in the background by .NET or HS (for hs specific calls). Also, what do the handles to the events represent as those are also increasing?
    If your thread count isn't increasing then yes it sounds like just the handles are remaining. They are typically small so shouldn't have a huge impact (if the threads were staying that would have a much larger impact). (quoted from MS) "Applications can use event objects in a number of situations to notify a waiting thread of the occurrence of an event. For example, overlapped I/O operations on files, named pipes, and communications devices use an event object to signal their completion." Again these are typically small, but generally applications are expected to close the handles they open. This count is mostly likely what Rich was referring to when he mentioned IO causing the issues.

    Leave a comment:


  • sparkman
    replied
    Originally posted by bsobel View Post

    A thread is unit of execution, often the lowest one in an OS (albeit some support fibers, which are a way of sharing a single thread). Typically one thread can operate at a time per CPU core you have, so if you have 4 cores 4 threads will be running (internally the OS will context swap so you may have 400 threads on those 4 cores with each getting 1/100th of the CPU time per second). A thread handle is just an opaque piece of data that you can use to tell the OS which thread you want to perform an action on. One can, for example, suspend or terminate a thread. Typically the API calls to do this take a thread handle, so they know which of the N threads in the system you are referring to. When a thread exits the handle can stay valid (when queried it would show the thread was done), so while the threads may have run to completion, the question is why aren't the handles being closed (free'd) so they are not building up.
    Great, thanks Bill. So for every thread that HS3 starts, a handle would also be created and since the actual threads are not increasing, it's strictly the handle to the thread that is not properly disposed of. Since I have a lot of scripts, are there things I should be doing in each script to manage that, or should that all be taken care of in the background by .NET or HS (for hs specific calls). Also, what do the handles to the events represent as those are also increasing?

    Leave a comment:


  • bsobel
    replied
    Originally posted by sparkman View Post

    I'm not clear the difference between threads and handles and handles to threads. The screen shots I posted are of handles. Most of the handles are related to threads and events and most don't have a name and the ones that do are very generic, so no way to pinpoint.
    A thread is unit of execution, often the lowest one in an OS (albeit some support fibers, which are a way of sharing a single thread). Typically one thread can operate at a time per CPU core you have, so if you have 4 cores 4 threads will be running (internally the OS will context swap so you may have 400 threads on those 4 cores with each getting 1/100th of the CPU time per second). A thread handle is just an opaque piece of data that you can use to tell the OS which thread you want to perform an action on. One can, for example, suspend or terminate a thread. Typically the API calls to do this take a thread handle, so they know which of the N threads in the system you are referring to. When a thread exits the handle can stay valid (when queried it would show the thread was done), so while the threads may have run to completion, the question is why aren't the handles being closed (free'd) so they are not building up.

    Leave a comment:


  • sparkman
    replied
    Originally posted by rjh View Post
    Long ago I made sure all of HS3 threads have names, so you should see a name if HS3 created the thread.
    I'm not clear the difference between threads and handles and handles to threads. The screen shots I posted are of handles. Most of the handles are related to threads and events and most don't have a name and the ones that do are very generic, so no way to pinpoint.

    Leave a comment:


  • rjh
    replied
    Long ago I made sure all of HS3 threads have names, so you should see a name if HS3 created the thread.

    Originally posted by bsobel View Post

    rjh Rich, these are thread handles, so dubious of your IO statement (e.g. these aren't files left open, ports, etc, these are threads that the HSConsole process creates or is created in its process space by a dll you use) Could you add descriptive thread names in the places in your code where you explicitly start threads? That would potentially help narrow this down. If the thread names are know, we know where to look, if not we know its likely a library we need to dive into.

    Leave a comment:


  • bsobel
    replied
    Originally posted by rjh View Post
    I have never created the handles issue here in the office, users have fixed it by making changes to their system. It is an I/O issue though so you must have something that is constantly accessing I/O like the network, serial ports, or some other I/O device. Also, I seem to remember some simply changing to a different PC and the problem went away. Do you have another PC you can try, maybe a laptop? All you need to do is copy over your entire HS3 folder to another system to test.
    rjh Rich, these are thread handles, so dubious of your IO statement (e.g. these aren't files left open, ports, etc, these are threads that the HSConsole process creates or is created in its process space by a dll you use) Could you add descriptive thread names in the places in your code where you explicitly start threads? That would potentially help narrow this down. If the thread names are know, we know where to look, if not we know its likely a library we need to dive into.

    Leave a comment:


  • sparkman
    replied
    Originally posted by rjh View Post
    I have never created the handles issue here in the office, users have fixed it by making changes to their system. It is an I/O issue though so you must have something that is constantly accessing I/O like the network, serial ports, or some other I/O device. Also, I seem to remember some simply changing to a different PC and the problem went away. Do you have another PC you can try, maybe a laptop? All you need to do is copy over your entire HS3 folder to another system to test.
    Thanks, I had the issue on my old PC as well. I had other issues that went away when I moved to this system, but not the handle leak.

    Leave a comment:


  • rjh
    replied
    I have never created the handles issue here in the office, users have fixed it by making changes to their system. It is an I/O issue though so you must have something that is constantly accessing I/O like the network, serial ports, or some other I/O device. Also, I seem to remember some simply changing to a different PC and the problem went away. Do you have another PC you can try, maybe a laptop? All you need to do is copy over your entire HS3 folder to another system to test.

    Originally posted by sparkman View Post

    Thanks Bill. Funny that you mention hs.geturl as I do have a few scripts calling that on a regular basis. I feel like I'm chasing ghosts though, as I've tried all this in the past and was never successful at poinpointing the issue. Short of disabling all my events and plugins and enabling them one at a time and then waiting for a day or two, there does not seem anyway to resolve this. And I don't have the patience for that, nor the time. rjh Can you comment on the points made? I disabled more plugins, events calling certain scripts, etc. and thought the handles were stable, but then they start increasing again.

    Leave a comment:


  • sparkman
    replied
    Originally posted by bsobel View Post

    There is no place where a plugin can directly create threads in HS other than if they do so in their constructor when first loaded into HS to check capabilities (plugins are loaded into the HSConsole to call these settings, then unloaded and ran remotely). Plugin authors have been told to not do this, and if they did the bump would only happen when the plugin first loaded. So while a plugin may be 'triggering' the creation of an HS thread, causing the leak, I do not believe you will find a single plugin responsible. Most likely there is an API in HS which launches the thread (the HS.GetURL is a potential candidate) and some plugins may use such a function more than others... So ideally we need to narrow down the type of functionality the plugins that seem responsible share to help narrow down the root cause.
    Thanks Bill. Funny that you mention hs.geturl as I do have a few scripts calling that on a regular basis. I feel like I'm chasing ghosts though, as I've tried all this in the past and was never successful at poinpointing the issue. Short of disabling all my events and plugins and enabling them one at a time and then waiting for a day or two, there does not seem anyway to resolve this. And I don't have the patience for that, nor the time. rjh Can you comment on the points made? I disabled more plugins, events calling certain scripts, etc. and thought the handles were stable, but then they start increasing again.

    Leave a comment:


  • bsobel
    replied
    Originally posted by sparkman View Post

    Thanks and agreed, but I do need to prove it and it may be potentially an issue with a combination of plugins. I disabled the ones which were the least impact to my HA. The handles did start creeping up again, so I need to disable a few more.
    There is no place where a plugin can directly create threads in HS other than if they do so in their constructor when first loaded into HS to check capabilities (plugins are loaded into the HSConsole to call these settings, then unloaded and ran remotely). Plugin authors have been told to not do this, and if they did the bump would only happen when the plugin first loaded. So while a plugin may be 'triggering' the creation of an HS thread, causing the leak, I do not believe you will find a single plugin responsible. Most likely there is an API in HS which launches the thread (the HS.GetURL is a potential candidate) and some plugins may use such a function more than others... So ideally we need to narrow down the type of functionality the plugins that seem responsible share to help narrow down the root cause.

    Leave a comment:

Working...
X