HS3 Pushover Plugin

[mrhappy]

IMPORTANT RE HS4

I have tested the HS3 plugin with HS4 version 4.0.0.19 and it appears to work OK (you will however have the HS3 configuration pages), not a great user experience but it does seem like it works.

At this present time it is unlikely that I will make a specific HS4 version, I'm not convinced that I will personally upgrade to HS4 anytme soon and as such have little incentive to upgrade the plugin to be a native HS4 one with their new SDK. I have also considered the intent of HS to implement their own push notifications into HS Mobile next year, this plugin may have had limited life going forward in any case. Keep in mind that HST can and likely will kill off parts of HS3 as HS4 develops so whilst it works in HS4 today there is no guarantee it will forever, indeed I would say it was more likely than not that it will suddenly die one day and you may get limited warning.

If you are intent on using the plugin with HS4 then please be advised that I can't support it and any help will be on a best effort basis. It will not include fixing any HS4 specific issues unless it has no impact on the HS3 version.

[bsobel]

HandleAction calls another function but it is not threaded (so I guess it is as good as being called from there), I can try and make it threaded however there will be a little bit of work as I'll have to sort out passing the parameters to the function from the HandleAction.

I'm a bit confused around 26 seconds though, there is a bit in that function but nothing that I would anticipate taking 26 seconds. Mine does and always has taken nothing more than a couple of seconds, even if it was timing out on a web exception it would be well before then. I've just sent three messages immediately one after another and all came through - in your case if you try this are you saying you are prevented because the plugin is hung in the HandleAction?

I've fixed the OSType/GetAppPath calls, I'll hold out on putting it in the updater pending the below.

As an FYI heroes what I see, not sure what's causing the delay sine it sounds much faster when you test locally. Its not a large problem, just ties up a HS IO thread for awhile, hence the suggestion.

Timing: HSPI_PUSHOVER.HSPI_PUSHOVER.HSPI.AccessLevel Total Calls: 1 Invoke: 1ms Deserialization: 3ms Total: 5ms Average: 5ms
Timing: HSPI_PUSHOVER.HSPI_PUSHOVER.HSPI.ActionConfigured Total Calls: 7 Invoke: 45,144ms Deserialization: 2ms Total: 45,147ms Average: 6,449ms
Timing: HSPI_PUSHOVER.HSPI_PUSHOVER.HSPI.HandleAction Total Calls: 7 Invoke: 193,588ms Deserialization: 2ms Total: 193,591ms Average: 27,655ms
Timing: HSPI_PUSHOVER.HSPI_PUSHOVER.HSPI.InitIO Total Calls: 1 Invoke: 421ms Deserialization: 0ms Total: 422ms Average: 422ms
Timing: HSPI_PUSHOVER.HSPI_PUSHOVER.HSPI.InterfaceStatus Total Calls: 1336 Invoke: 32ms Deserialization: 323ms Total: 355ms Average: 0ms
Timing: HSPI_PUSHOVER.HSPI_PUSHOVER.HSPI.SupportsMultipleInstances Total Calls: 1 Invoke: 0ms Deserialization: 0ms Total: 0ms Average: 0ms
Timing: HSPI_PUSHOVER.HSPI_PUSHOVER.HSPI.get_Name Total Calls: 405 Invoke: 10ms Deserialization: 245ms Total: 255ms Average: 0ms

On the parameter issue, what I usually do is this pattern:

// Put needed locals into a tuple
var payload = new Tuple<object, MessageEventArgs, ManualResetEvent>(sender, e, completeEvent);
if (!ThreadPool.QueueUserWorkItem(Client_MessageReceivedThreadP roc, payload))
Client_MessageReceivedThreadProc(payload);
}

private void Client_MessageReceivedThreadProc(object data)
{
var parameters = data as Tuple<object, MessageEventArgs, ManualResetEvent>;
existing code...
}

Finally call log:
[09/11/2019 14:52:40] ******** ActionConfigured Called DataIn IsNothing: False
[09/11/2019 14:52:40] ActionConfigured - 5839 Action Keys Count: 0
[09/11/2019 14:52:40] Call to DeSerialize the Information
[09/11/2019 14:52:41] Return from DeSerialize Key Count: 9
[09/11/2019 14:52:41] sKey ActionConfigured: mKey_5839_2_A0 Actions: All Clients
[09/11/2019 14:52:41] sKey ActionConfigured: mTitle_5839_2_A0 Actions: HomeSeer Startup
[09/11/2019 14:52:41] sKey ActionConfigured: mMessage_5839_2_A0 Actions: HomeSeer has restarted
[09/11/2019 14:52:42] sKey ActionConfigured: mPriority_5839_2_A0 Actions: High
[09/11/2019 14:52:42] sKey ActionConfigured: mSound_5839_2_A0 Actions: System Default
[09/11/2019 14:52:42] sKey ActionConfigured: mAPI_5839_2_A0 Actions: Casa Piedra Alert
[09/11/2019 14:52:42] sKey ActionConfigured: mURL_5839_2_A0 Actions:
[09/11/2019 14:52:43] sKey ActionConfigured: mURLTitle_5839_2_A0 Actions:
[09/11/2019 14:52:43] sKey ActionConfigured: mAttach_5839_2_A0 Actions:
[09/11/2019 14:52:43] itemsConfigured: 6
[09/11/2019 14:52:44] Configured: True
[09/11/2019 14:52:44] ********* HandleAction Called
[09/11/2019 14:52:44] AddToTable : Event Action Row Count: 2
[09/11/2019 14:52:45] APIStr Casa Piedra Alert
[09/11/2019 14:52:45] PushMessage Sub Called - DeviceName All Clients
[09/11/2019 14:52:46] AttachPath Nothing? Yes
[09/11/2019 14:52:46] AddToTable : Send Pushover Message Row Count: 3
[09/11/2019 14:52:46] Priority high
[09/11/2019 14:52:47] Emergency Priority - Second Check
[09/11/2019 14:52:47] > SendStatus Is True - Send The Message
[09/11/2019 14:52:48] > ALL CLIENTS Selected
[09/11/2019 14:52:48] > Device Names:
[09/11/2019 14:52:49] > User Key: xxxxx
[09/11/2019 14:52:49] -> Web Client Busy Status False
[09/11/2019 14:52:50] *** CustomReplacement Function Entered Input String: HomeSeer has restarted
[09/11/2019 14:52:50] *** ReplacementFunction Return: HomeSeer has restarted
[09/11/2019 14:52:50] *** CustomReplacement Function Entered Input String: HomeSeer Startup
[09/11/2019 14:52:51] *** ReplacementFunction Return: HomeSeer Startup
[09/11/2019 14:52:51] *** CustomReplacement Function Entered Input String:
[09/11/2019 14:52:51] *** ReplacementFunction Return:
[09/11/2019 14:52:52] -> URL Encoded Message
[09/11/2019 14:52:52] -> Message Length: 22
[09/11/2019 14:52:52] Sending System Default Sound
[09/11/2019 14:52:53] First Token xxxx
[09/11/2019 14:52:53] -> API Token Found In List - Casa Piedra Alert
[09/11/2019 14:52:53] -> pushToken xxx
[09/11/2019 14:52:54] ---> Key user - Item xxxx
[09/11/2019 14:52:54] ---> Key url - Item
[09/11/2019 14:52:54] ---> Key url_title - Item
[09/11/2019 14:52:54] ---> Key priority - Item 1
[09/11/2019 14:52:55] ---> Key device - Item
[09/11/2019 14:52:55] ---> Key title - Item HomeSeer Startup
[09/11/2019 14:52:55] ---> Key token - Item xxxxx
[09/11/2019 14:52:56] ---> Key message - Item HomeSeer has restarted
[09/11/2019 14:52:56] -> Sending to Pushover...
[09/11/2019 14:52:57] --> WebClient Is Running False
[09/11/2019 14:52:57] --> Web Response Header Server
[09/11/2019 14:52:57] --> Web Response Header Date
[09/11/2019 14:52:58] --> Web Response Header Content-Type
[09/11/2019 14:52:58] --> Web Response Header Transfer-Encoding
[09/11/2019 14:52:58] --> Web Response Header Connection
[09/11/2019 14:52:58] --> Web Response Header X-Frame-Options
[09/11/2019 14:52:59] --> Web Response Header X-XSS-Protection
[09/11/2019 14:52:59] --> Web Response Header X-Content-Type-Options
[09/11/2019 14:52:59] --> Web Response Header Access-Control-Allow-Origin
[09/11/2019 14:53:00] --> Web Response Header Access-Control-Allow-Methods
[09/11/2019 14:53:00] --> Web Response Header Access-Control-Allow-Headers
[09/11/2019 14:53:00] --> Web Response Header Access-Control-Max-Age
[09/11/2019 14:53:00] --> Web Response Header X-Limit-App-Limit
[09/11/2019 14:53:01] --> Web Response Header X-Limit-App-Remaining
[09/11/2019 14:53:01] --> Web Response Header X-Limit-App-Reset
[09/11/2019 14:53:01] --> Web Response Header ETag
[09/11/2019 14:53:02] --> Web Response Header Cache-Control
[09/11/2019 14:53:02] --> Web Response Header X-Request-Id
[09/11/2019 14:53:02] --> Web Response Header X-Runtime
[09/11/2019 14:53:03] --> Web Response Header Strict-Transport-Security
[09/11/2019 14:53:03] ---> Messages Remaining 7416
[09/11/2019 14:53:03] Reply: <?xml version="1.0" encoding="UTF-8"?>
<response>
<status>1</status>
<request>ff119a11-dc9f-4660-7777-006469e7480e</request>
</response>

[09/11/2019 14:53:04] Reply Status 1
[09/11/2019 14:53:04] Pushover Notification Sent - Message Title HomeSeer Startup

[mrhappy]

Hi, one other request. I believe you may be calling out to Pushover inside of your HandleAction call. I see about 26seconds of hang there when a pushover call is made. Can I suggest you put those on a queue and have your background thread execute them? This would allow the thread on the server side in HSConsoel to be released much faster, and free it up for other work. (I just instrumented the COM layer to dump call timings every 5 minutes. All your other methods are super low, that was the only one that stood out so figured I'd let you know.

HandleAction calls another function but it is not threaded (so I guess it is as good as being called from there), I can try and make it threaded however there will be a little bit of work as I'll have to sort out passing the parameters to the function from the HandleAction.

I'm a bit confused around 26 seconds though, there is a bit in that function but nothing that I would anticipate taking 26 seconds. Mine does and always has taken nothing more than a couple of seconds, even if it was timing out on a web exception it would be well before then. I've just sent three messages immediately one after another and all came through - in your case if you try this are you saying you are prevented because the plugin is hung in the HandleAction?

I've fixed the OSType/GetAppPath calls, I'll hold out on putting it in the updater pending the below.

Hi
i moved my homeseer from windows to linux and my pushover for some reason is not working on some places.
i have a weather push that goes through and a news update that i had to trim otherwise it didnt arrive on my cell.
now i am trying to send an attachment file. when sending without the file, the push goes through but when attaching the file i get "Error POST URL Data at the root level is invalid. Line 1, position 1.".

any ideas?

Sorry for the delay on this one, I had to setup a VM with Linux running before I could test. I can send your image fine running on Ubuntu with a basic HS3 installation. Looking at the "Data at the root level is invalid" error would imply that it is having difficulty parsing an XML response which only happens when I get some reply from Pushover. At that point the message has already been sent and it is just the reply that I am trying to read.

Can I ask if you could try this version, I've put some additional calls in the log to see what is being sent back from Pushover.

[bsobel]

Hi, one other request. I believe you may be calling out to Pushover inside of your HandleAction call. I see about 26seconds of hang there when a pushover call is made. Can I suggest you put those on a queue and have your background thread execute them? This would allow the thread on the server side in HSConsoel to be released much faster, and free it up for other work. (I just instrumented the COM layer to dump call timings every 5 minutes. All your other methods are super low, that was the only one that stood out so figured I'd let you know.

[bsobel]

Fair point, they both are in the same function (a logging function, determine the OS type to determine the path structure and then get the HS path to determine where put the log files) so can be changed and in hindsight I think I put them there in a rush as I can see they are going to be called constantly.

There is only one thread in the plugin so that narrows it down somewhat. Most of the code inside that thread is not particularly contentious apart from me running through the enumerator and registering a web page. It registers the web page first then runs, does a bit more like sets up some timers and then through the enumerator to determine whether or not the plugin has devices. I've seen your other posts and from what I can understand I think I will need to check those calls are appropriately safeguarded.

It may be that I can get rid of the enumerator alltogether and like others I am a bit mystified that it takes as long as it takes. I was always a bit surprised that a simple call of something like hs.getplugindevices (where you pass in a plugin name and instance) was not written instead, I remember that in the HS3 developer webinar there was some discussion over the enumerator and I think it was originally not part of the HS3 spec and only added after some protest.

Hi, sorry didn't realize you saw the first post. Ignore the thread issue, it was a HS bug not you. I noticed in your plugin because since you were not doing any work the bug was very pronounced, a worker thread in RPC spins on no traffic. I just tested a fix on my system and sent the update to Rich, you are working just fine (sitting at .3% cpu right now!) Sorry about that, thought I deleted the post fast enough. The other two issues are real.

Commenting on your "a bit mystified that it takes as long as it takes" Before two thing came into play, the number of devices (my system has about 2700) and the timeout, but that threading issue I just mentioned is also involved. Those spinning threads take CPU time from the real system, and defiantly caused part of the issue (two threads were spinning per plugin. With the fix I just did I saw those number drop quite a bit. It still takes 15 seconds, but thats a far cry from 68-120 I was seeing)

Thanks!

[bsobel]

Hi,

I have a few small requests I hope you can consider making in your plugin. I posted a long article in the developer forum on some best practices after working on HS internals bug for quite some time. That has given me a good look into the performance of many plugins now that multithreading is working as intended, as before all of your plugin threads were actually having their API calls performed in sequential order do to the RPC remoting processing call on a single thread on both sides of the RPC connection.

There are two performance issues I would like you to know about and ideally address:

Each HS call entails creating a request message, sending it to the server, processing it, and returning its results. As such, any reduction in call count helps overall system performance. In the current plugin you call GetOSType() quite a few times as well as GetAppPath() almost the same. I think you are building a path to call to get an ini setting. Can you kindly save the results of those two calls and reuse them, their values will not change at runtime.

Best
Bill

[mrhappy]

Each HS call entails creating a request message, sending it to the server, processing it, and returning its results. As such, any reduction in call count helps overall system performance. In the current plugin you call GetOSType() quite a few times as well as GetAppPath() almost the same. I think you are building a path to call to get an ini setting. Can you kindly save the results of those two calls and reuse them, their values will not change at runtime.

Fair point, they both are in the same function (a logging function, determine the OS type to determine the path structure and then get the HS path to determine where put the log files) so can be changed and in hindsight I think I put them there in a rush as I can see they are going to be called constantly.

On my system, with threading fixed, you have a thread I think in a loop somewhere. It is taking up 100% cpu time on one core on my system. This is likely due to the fact that the thread was unintentionally limited prior due to the RPC sequential nature which acted as a natural delay (the system is running MUCH faster now in general). I can confirm that when this is happening you are NOT calling HS api's (so my window into what you are doing is limited). But its probably one of your worker threads spinning in a loop that probably needs a sleep or wait (depending on what would entail that thread having more work to do). Can you take a peek and see if you can find the culprit, I am happy to test a fix here as you may not see the behavior yourself until Rich released an update with the RPC fixes that were made.

There is only one thread in the plugin so that narrows it down somewhat. Most of the code inside that thread is not particularly contentious apart from me running through the enumerator and registering a web page. It registers the web page first then runs, does a bit more like sets up some timers and then through the enumerator to determine whether or not the plugin has devices. I've seen your other posts and from what I can understand I think I will need to check those calls are appropriately safeguarded.

It may be that I can get rid of the enumerator alltogether and like others I am a bit mystified that it takes as long as it takes. I was always a bit surprised that a simple call of something like hs.getplugindevices (where you pass in a plugin name and instance) was not written instead, I remember that in the HS3 developer webinar there was some discussion over the enumerator and I think it was originally not part of the HS3 spec and only added after some protest.

[DJF3]

My fault my IP was incorrect on my Dynamic DNS - should be back up now.

Looks like it's down again
(maybe not but the site is getting blocked by Umbrella: Block Reason: DNS RPZ)

Do you have a new URL?

[mrhappy]

Of Course.
here is the event in full.

and here is the CurrentImage.jpg

I'd have to check whether or not the testing was ever done to send images on Linux. I think I checked it but who knows if there has been changes to Mono in the intervening period, having looked at the code and from memory this was a surprisingly difficult thing to get working correctly as the file had to be read as bytes and encoded into the HTTP form. I don't know how well Linux/Mono behaves whilst doing this and as it is passing through the various checks I put in (file size, file type, file exists etc) then it must be getting to the point it thinkgs that it can deal with the file.

[ez1976]

Of Course.
here is the event in full.

and here is the CurrentImage.jpg

[mrhappy]

Hi
i moved my homeseer from windows to linux and my pushover for some reason is not working on some places.
i have a weather push that goes through and a news update that i had to trim otherwise it didnt arrive on my cell.
now i am trying to send an attachment file. when sending without the file, the push goes through but when attaching the file i get "Error POST URL Data at the root level is invalid. Line 1, position 1.".

any ideas?

The event structures may be different between the OS's which is why things may not work. Can you please attach a screenshot of your event with the attachment? Also if the file is not sensitive would you be able to send it me so I can verify that I can send it to myself?

[ez1976]

Hi
i moved my homeseer from windows to linux and my pushover for some reason is not working on some places.
i have a weather push that goes through and a news update that i had to trim otherwise it didnt arrive on my cell.
now i am trying to send an attachment file. when sending without the file, the push goes through but when attaching the file i get "Error POST URL Data at the root level is invalid. Line 1, position 1.".

any ideas?

[mrhappy]

Is there a way to automatically reset the Pushover message count at the beginning of the month?



Edit:
OK, this is weird, it did reset. I guess I never saw it on the first of the month last month.

I set this device with data from the HTTP header replies from Pushover when you send a message, some more detail is on their API page;

Once message limits have been reached, requests will be rejected with a 429 HTTP status code. Message limits are reset at00:00:00 Central Time on the 1st of each month. Per-day and per-month usage statistics can be viewed on each application's page, as well as through HTTP headers returned in each API call to the messages endpoint showing your app's monthly message limit (plus any additional purchased capacity), the number of messages sent this month, and the Unix timestamp of when the count will reset.
X-Limit-App-Limit: 7500
X-Limit-App-Remaining: 7496
X-Limit-App-Reset: 1393653600

[Dan-O]

Is there a way to automatically reset the Pushover message count at the beginning of the month?



Edit:
OK, this is weird, it did reset. I guess I never saw it on the first of the month last month.

[mrhappy]

Thanks MattL0

Sorry it was my auto correct that corrected ping to pick, your results are a little slower than I would expect (I get a ping of 88ms, I'm on a fairly average ADSL line) but it should not introduce a delay of that long. I will drop you a PM if that is OK to run through a few things.