www.homeseer.com    
 

Go Back   HomeSeer Message Board > 3rd Party Plug-Ins/Scripts > Plug-ins by Author > Click Here for List of Author Forums > MCS Plug-Ins and Scripts > mcsMQTT (3P)

mcsMQTT (3P) Discussion of mcsMQTT plug-in

Reply
 
Thread Tools Display Modes
  #1  
Old December 13th, 2017, 11:40 PM
Michael McSharry's Avatar
Michael McSharry Michael McSharry is offline
OverSeer
 
Join Date: Jul 2001
Location: North Bend, WA, USA
Posts: 13,157
mcsMQTT Plugin

I have become involved with low cost IoT devices with MQTT being the primary means for non-cloud (or cloud) communication and integration. To make it easy to fit into my existing Homeseer environment I have developed mcsMQTT.

This plugin retains HS as the central node in the automation environment and allows MQTT-literate devices to be utilized. While HS Events can be triggered based upon MQTT Topics and Event Actions can publish Topics, the primary integration is with the HS DeviceValue and DeviceString. Changes in HS devices or changes in MQTT Topic payloads are seamlessless reflected with the mcsMQTT bridge.

Change Log
PR 1 12/20/2017 3.0.1.0 Add optional payload specification in MQTT event trigger
PR 2 12/20/2017 3.0.1.0 Define payload OPEN/CLOSE, ON/OFF Device-Status-Graphics pairs for DeviceValue 1/0, map these payloads to 1/0 DeviceValue updates
PR 3 12/20/2017 3.0.1.0 Honor "SET_DOES_NOT_CHANGE_LAST_CHANGE" device property checkbox when updating DeviceLastChange
PR 4 12/21/2017 3.0.1.1 Device Value and String changes not being reflected in new MQTT messages
PR 5 12/21/2017 3.0.1.2 Add debug focused around HSEvent processing
PR 6 12/22/2017 3.0.2.0 Provide bidirectional status and control in single HS3 device
PR 7 12/29/2017 3.0.3.0 JSON parsing of MQTT payload results in exception and log message
PR 8 12/29/2017 3.0.3.0 "A"ccept set true on initialization based upon presence of HS device with the associated Device Ref property
PR 9 12/29/2017 3.0.3.0 Backup of database created on each plugin start, backup retained for 6 months
PR10 1/3/2018 3.0.4.0 Add info to HS3 Device PED and synchronize with database
PR11 1/3/2018 3.0.4.0 Improve initial creation of database so restart is not needed
PR12 1/3/2018 3.0.4.1 Change HS3 connection from 192.168.0.194 to localhost
PR13 1/3/2018 3.0.4.2 Protect Topic from single and double quote in database operations
PR14 1/7/2018 3.0.4.3 Parsing error with training JSON } not removed
PR15 1/8/2018 3.0.4.4 Parsing error reversion, broke earlier correct parsing
PR16 1/9/2018 3.0.5.0 Add ability to specify arbitrary subscription topics
PR17 1/9/2018 3.0.6.0 Parsing exception ala PR7
PR18 1/9/2018 3.0.6.0 Provide Regular Expression replacement on Payload to Device
PR19 1/9/2018 3.0.6.1 Subscription does not include both $SYS and # Topics
PR20 1/10/2018 3.0.7.0 Provide Extract vs. Replace option on regular expressions

Documentation/Manual http://mcsSprinklers.com/mcsMQTT.pdf

Last edited by Michael McSharry; January 10th, 2018 at 07:52 PM.
Reply With Quote
  #2  
Old December 14th, 2017, 03:07 AM
Summerguy Summerguy is offline
Seer Plus
 
Join Date: Aug 2017
Location: Canada
Posts: 122
Hi,

how can we install this plugin?

thank you
Reply With Quote
  #3  
Old December 14th, 2017, 08:11 AM
vasrc's Avatar
vasrc vasrc is offline
Seer Master
 
Join Date: May 2003
Location: Locust Dale, VA
Posts: 1,169
Quote:
Originally Posted by Michael McSharry View Post
I have become involved with low cost IoT devices with MQTT being the primary means for non-cloud (or cloud) communication and integration. To make it easy to fit into my existing Homeseer environment I have developed mcsMQTT.

This plugin retains HS as the central node in the automation environment and allows MQTT-literate devices to be utilized. While HS Events can be triggered based upon MQTT Topics and Event Actions can publish Topics, the primary integration is with the HS DeviceValue and DeviceString. Changes in HS devices or changes in MQTT Topic payloads are seamlessless reflected with the mcsMQTT bridge.
Nice work, you can tell what you do for a living

I wasn't clear on whether the Broker connection will support port 8883/TLS? Will it support a CA cert?

Also, did you use the Event/script interfaces for processing pub/sub rather than using the plugin to map them directly to the HS3 device (or did I miss that?). Clearly the Event/script method gives you more flexibility (ie translating HS3 device/string values to/from field device payloads)

When do we get to play with it

Thanks,
Z
Reply With Quote
  #4  
Old December 14th, 2017, 09:48 AM
kideon kideon is offline
Seer Master
 
Join Date: May 2014
Location: virginia
Posts: 762
So what sort of applications would mqtt be useful for? I’ve been wanting to mess around with something new.
Reply With Quote
  #5  
Old December 14th, 2017, 10:05 AM
Fischi Fischi is offline
Seer Master
 
Join Date: Oct 2012
Location: Germany
Posts: 515
Hi,
Isn't it the same as this http://dzjee.xs4all.nl/hs3/mqtt/mqtt.html ?

Fischi
Reply With Quote
  #6  
Old December 14th, 2017, 03:40 PM
Michael McSharry's Avatar
Michael McSharry Michael McSharry is offline
OverSeer
 
Join Date: Jul 2001
Location: North Bend, WA, USA
Posts: 13,157
Quote:
how can we install this plugin?
I have submitted the request to HST to include it in the updater. The install file that the updater will use is at http://mcsSprinklers.com/mcsMQTT_3_0_0_0.zip. You can manually installed now per guidance in the manual or wait for it to show up in the updater.

Quote:
I wasn't clear on whether the Broker connection will support port 8883/TLS? Will it support a CA cert?
I have not been concerned with security in my environment, but the underlying software does support it. If you are concerned then I can enable it. It is something we can pursue together if you are interested.

Quote:
Also, did you use the Event/script interfaces for processing pub/sub rather than using the plugin to map them directly to the HS3 device (or did I miss that?). Clearly the Event/script method gives you more flexibility (ie translating HS3 device/string values to/from field device payloads)
I have provided all options that best suits the user's orientation and environment. In my case I am Device and not Event oriented so I have mapped Topics to Devices and used Device Value/String to manage the Payload. Others that are Event oriented can trigger an Event on a Topic received or use the Event Action to publish a Topic/Payload.

Quote:
So what sort of applications would mqtt be useful for? I’ve been wanting to mess around with something new.
I have been using an xAP environment for distributed computing for many years. MQTT is a minor variation from xAP with respect to integrating many remote entities. This mcsMQTT plugin was a very easy port of mcsXap plugin.
What moved me to include MQTT is the ESP8266/Sonoff units that sell for under $5 and these include the power supply, Wifi, Relay, multiple discretes and a well supported software environment for hacking. Emulation for Wemo and Hue is included so Echo/Alexa voice control is built in. My first two applications are to provide intelligent Wifi Garage Door interface and a Wifi water meter counter to replace one that has sometimes given me problems on 1-Wire.

Quote:
Isn't it the same as this http://dzjee.xs4all.nl/hs3/mqtt/mqtt.html ?
Both understand MQTT protocol. It is the user interface, orientation and the degree of integration with HS that are different. If one has a distributed mind-set and HS3 is just one more node then a simple publish/subscribe works well. If one has a master/central mind-set then mcsMQTT should be easier to integrate MQTT with HS3.

Last edited by Michael McSharry; December 14th, 2017 at 06:19 PM.
Reply With Quote
  #7  
Old December 14th, 2017, 04:42 PM
snowboarder snowboarder is offline
Seer
 
Join Date: Oct 2002
Posts: 43
when i click the download link i get a 404 error ?

Quote:
Originally Posted by Michael McSharry View Post
I have submitted the request to HST to include it in the updater. The install file that the updater will use is at http://mcsSprinklers.com/mcsMQTT_3.0.0.0.zip. You can manually installed now per guidance in the manual or wait for it to show up in the updater.


I have not been concerned with security in my environment, but the underlying software does support it. If you are concerned then I can enable it. It is something we can pursue together if you are interested.


I have provided all options that best suits the user's orientation and environment. In my case I am Device and not Event oriented so I have mapped Topics to Devices and used Device Value/String to manage the Payload. Others that are Event oriented can trigger an Event on a Topic received or use the Event Action to publish a Topic/Payload.


I have been using an xAP environment for distributed computing for many years. MQTT is a minor variation from xAP with respect to integrating many remote entities. This mcsMQTT plugin was a very easy port of mcsXap plugin.
What moved me to include MQTT is the ESP8266/Sonoff units that sell for under $5 and these include the power supply, Wifi, Relay, multiple discretes and a well supported software environment for hacking. Emulation for Wemo and Hue is included so Echo/Alexa voice control is built in. My first two applications are to provide intelligent Wifi Garage Door interface and a Wifi water meter counter to replace one that has sometimes given me problems on 1-Wire.


Both understand MQTT protocol. It is the user interface, orientation and the degree of integration with HS that are different. If one has a distributed mind-set and HS3 is just one more node then a simple publish/subscribe works well. If one has a master/central mind-set then mcsMQTT should be easier to integrate MQTT with HS3.
Reply With Quote
  #8  
Old December 14th, 2017, 05:51 PM
vasrc's Avatar
vasrc vasrc is offline
Seer Master
 
Join Date: May 2003
Location: Locust Dale, VA
Posts: 1,169
Quote:
Originally Posted by snowboarder View Post
when i click the download link i get a 404 error ?
Try:
http://mcssprinklers.com/mcsMQTT_3_0_0_0.zip

Z
Reply With Quote
  #9  
Old December 14th, 2017, 08:20 PM
petez69's Avatar
petez69 petez69 is offline
Seer Master
 
Join Date: Oct 2005
Location: Alice Springs, Australia
Posts: 842
Michael

As you say the Sonoff devices are a no-brainer. I grabbed a few on the 11.11 Aliexpress sale and having fun with them. I've just bought teh 16A version with current sense and one of the tri-colour LEDS .....all very neat stuff :-)

Pete
Reply With Quote
  #10  
Old December 16th, 2017, 09:37 AM
snowboarder snowboarder is offline
Seer
 
Join Date: Oct 2002
Posts: 43
When i enable the plugin in homeseer i get the this error ?

WARNING: Failed getting InterfaceStatus from mcsMQTT - the interface was not found in the list of active interfaces, the list may need to be refreshed
Reply With Quote
  #11  
Old December 16th, 2017, 09:49 AM
snowboarder snowboarder is offline
Seer
 
Join Date: Oct 2002
Posts: 43
Quote:
Originally Posted by snowboarder View Post
When i enable the plugin in homeseer i get the this error ?

WARNING: Failed getting InterfaceStatus from mcsMQTT - the interface was not found in the list of active interfaces, the list may need to be refreshed
When i start in Homeseer in Developer Mode i see that the plugin want to connect to server ip 192.168.0.194 . this is not the ip of my homeseer server.
Reply With Quote
  #12  
Old December 16th, 2017, 10:36 AM
vasrc's Avatar
vasrc vasrc is offline
Seer Master
 
Join Date: May 2003
Location: Locust Dale, VA
Posts: 1,169
Quote:
Originally Posted by snowboarder View Post
When i start in Homeseer in Developer Mode i see that the plugin want to connect to server ip 192.168.0.194 . this is not the ip of my homeseer server.
Download the zip file again. He's fixed that in the latest one.

Z
Reply With Quote
  #13  
Old December 16th, 2017, 01:40 PM
Michael McSharry's Avatar
Michael McSharry Michael McSharry is offline
OverSeer
 
Join Date: Jul 2001
Location: North Bend, WA, USA
Posts: 13,157
I compiled/uploaded a third time to connect to localhost rather than .194.
Reply With Quote
  #14  
Old December 17th, 2017, 09:11 AM
petez69's Avatar
petez69 petez69 is offline
Seer Master
 
Join Date: Oct 2005
Location: Alice Springs, Australia
Posts: 842
Michael

Thanks mate, this is a ripper of a plugin. Took me a little time to understand your config page but I've got it happily driving a Tasmota/Sonoff basic relay. Setup a virtual device with a back end script to turn the device on and off, thats now also working with Alexa....

Great work mate, very happy

Pete
Reply With Quote
  #15  
Old December 20th, 2017, 08:14 AM
petez69's Avatar
petez69 petez69 is offline
Seer Master
 
Join Date: Oct 2005
Location: Alice Springs, Australia
Posts: 842
Michael

I'm a little stumped as to how to trigger an event from a received MQTT topic. Actually its probably not a topic, its a device I've created as a result of /stat/wemos-sonoff/POWER

The device on the web page turns on and off fine as I ground the wemos PIN. What I cant seem to work out is how to trigger an event from this.

What device value is on/off or do I need to define this myself ?

I've gone into status graphics and set 0 to off and 1 to on....I cant seem to trigger an event from on or off/0 or 1 device status and value....

Clearly missing something simple....

Appreciate your guidance

Pete
Reply With Quote
  #16  
Old December 20th, 2017, 08:29 AM
petez69's Avatar
petez69 petez69 is offline
Seer Master
 
Join Date: Oct 2005
Location: Alice Springs, Australia
Posts: 842
Michael

I've created a device from stat/mqtt/alive

Seem to have an issue with the device adhering to the checkbox being turned off:

Do not update device last change time
if device value does not change:

I've turned this off and sent the following:

mosquitto_pub -d -t stat/mqtt/alive -m ping

If I send it in a minutes time, the timestamp on the web page doesnt change. If I change the payload then it updates the data on the web page fine. I was going to run a crontab on the raspberrypi every 5 mins so I can keep tab of the device being alive. I know you have another way to do it which is fine but curious as to why this is happening ?

Any ideas ?

Thanks Pete
Reply With Quote
  #17  
Old December 20th, 2017, 03:20 PM
Michael McSharry's Avatar
Michael McSharry Michael McSharry is offline
OverSeer
 
Join Date: Jul 2001
Location: North Bend, WA, USA
Posts: 13,157
Quote:
I'm a little stumped as to how to trigger an event from a received MQTT topic. Actually its probably not a topic, its a device I've created as a result of /stat/wemos-sonoff/POWER

The device on the web page turns on and off fine as I ground the wemos PIN. What I cant seem to work out is how to trigger an event from this.

What device value is on/off or do I need to define this myself ?

I've gone into status graphics and set 0 to off and 1 to on....I cant seem to trigger an event from on or off/0 or 1 device status and value....

Clearly missing something simple....

Appreciate your guidance
What is the payload on the Topic /stat/wemos-sonoff/POWER?

If it is 0/1 then the DeviceValue will change between these two values. If you have status-graphics pairs setup for 0 and 1 values then the Status should show whatever graphics you setup for these two states.

If it is OFF/ON then the DeviceValue should never change from 0 and the DeviceString will contain the payload.

HS Events will support a change in DeviceValue. There are 3rd party plugins that will do DeviceString as the trigger.

There are two things I have been considering. One is the case where non-numeric payloads are received. In this case then look up to see if a status pair has been defined by the user and then store the numeric value if it matches.

The other is to expand the MQTT trigger to include the payload. This would allow trigger on a message reception based upon a payload for a topic rather than just a topic. This would allow two events to be setup where one is for an ON payload and the other for an OFF payload, as an example.

Quote:
I've created a device from stat/mqtt/alive

Seem to have an issue with the device adhering to the checkbox being turned off:

Do not update device last change time
if device value does not change:

I've turned this off and sent the following:

mosquitto_pub -d -t stat/mqtt/alive -m ping

If I send it in a minutes time, the timestamp on the web page doesnt change. If I change the payload then it updates the data on the web page fine. I was going to run a crontab on the raspberrypi every 5 mins so I can keep tab of the device being alive. I know you have another way to do it which is fine but curious as to why this is happening ?

Any ideas ?
I was not aware of HS3 providing the setting to control the device last change being used for device last refreshed. I will research the API and honor it in the next plugin update.
Reply With Quote
  #18  
Old December 20th, 2017, 07:46 PM
petez69's Avatar
petez69 petez69 is offline
Seer Master
 
Join Date: Oct 2005
Location: Alice Springs, Australia
Posts: 842
Quote:
Originally Posted by Michael McSharry View Post
What is the payload on the Topic /stat/wemos-sonoff/POWER?

If it is OFF/ON then the DeviceValue should never change from 0 and the DeviceString will contain the payload.


There are two things I have been considering. One is the case where non-numeric payloads are received. In this case then look up to see if a status pair has been defined by the user and then store the numeric value if it matches.

The other is to expand the MQTT trigger to include the payload. This would allow trigger on a message reception based upon a payload for a topic rather than just a topic. This would allow two events to be setup where one is for an ON payload and the other for an OFF payload, as an example.
Michael

OK, payload is ON or OFF, that's what the tasmota firmware sends, hence this is why devicevalue never changes. Understand I could use another plugin but I'd prefer not to as it would then require additional configuration for a single device to enable triggers to occur.

So to expand the trigger on payload, that is something you would need to code ? I did read your documentation and I see you can trigger on topic but not payload. Is this something you could consider as it would make the triggering of events very simple if we could define the payload and not just the topic. This will make using your plugin with the sonoff tasmota firmware a total breeze to use.

And thankyou for considering the change in code to honour the API. Again this means I can run a cronjob on the broker and track the "keepalive". I did note you have a trigger option if it doesnt get a message in a self determined timeframe.

Cheers & regards..Pete
Reply With Quote
  #19  
Old December 21st, 2017, 12:05 AM
Michael McSharry's Avatar
Michael McSharry Michael McSharry is offline
OverSeer
 
Join Date: Jul 2001
Location: North Bend, WA, USA
Posts: 13,157
I made the updates discussed. I do not see the plugin in the updater yet. I have not updated the manual yet either. You can get it at http://mcsSprinklers.com/mcsMQTT_3_0_1_0.zip

If you have a message received that has case-insensitive OPEN, CLOSED, ON, or OFF as the Payload when the Topic is accepted then the created device will create Device-Value-Graphic pairs and handle OPEN/ON as DeviceValue=1 and CLOSED/OFF as DeviceValue=0. In this case the DeviceString will not be updated and the Status shown will be from the Device-Value-Graphic that is setup. If you want different text or icons then they can be edited. If it not a number and not one of these four then the DeviceValue will not be updated. In the future I think I will expand this to map every text string received on a topic to a unique DeviceValue so they will be treated as an enumeration and DeviceValue changes can be used in events to achieve desired trigger results.

For other non-numeric payloads the DeviceString will be updated and the Status will show up as the DeviceString. For all others the DeviceString will be cleared. Device Management of Devices can be used to provide suffix, prefix and some other formatting information for Status. For example, a number is treated by default as a Dim level. I change mine to add a suffix (e.g. Minutes) to provide a meaning to the number and remove the Dim prefix.

DeviceLastChange is now explicitly updated based upon the device's checkbox. It will either update the date on a value change or upon new topic reception based upon the checkbox.

MQTT Receive trigger how has provisions for the Payload. It likely means that you will need to redefine existing MQTT Receive Trigger Events.

Last edited by Michael McSharry; December 21st, 2017 at 12:17 AM.
Reply With Quote
  #20  
Old December 21st, 2017, 02:21 AM
drodegeb drodegeb is offline
Seer Deluxe
 
Join Date: Oct 2002
Posts: 174
This plugin is great!

How do you do the suffix part? I've been searching through the device config pages, and the plugin and can't find it. I have temperature and humidify values that are coming up as "DIM xx%", for the temperature one I would like to remove DIM and % and add F, for humidity I would like to get rid of the DIM.

Thanks

Quote:
Originally Posted by Michael McSharry View Post
For other non-numeric payloads the DeviceString will be updated and the Status will show up as the DeviceString. For all others the DeviceString will be cleared. Device Management of Devices can be used to provide suffix, prefix and some other formatting information for Status. For example, a number is treated by default as a Dim level. I change mine to add a suffix (e.g. Minutes) to provide a meaning to the number and remove the Dim prefix.
__________________
Dave
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
ERRO Calling SetIOMulti in plugin Arduino Plugin:Object reference not set to an in fvhemert Arduino Plugin (3P) 5 April 28th, 2017 10:15 AM
Nest Plugin Errors - cannot change temperature via plugin TimSRQ Nest Devices (3P) 2 August 15th, 2016 08:48 AM
Upgrade procedure for ACRF plugin from W800 free plugin htsource ACRF Processor (3P) 14 January 26th, 2006 11:21 AM
Switch-to messages, plugin restart, and plugin startup Michael McSharry Plug-In Development 4 January 27th, 2003 11:20 PM


All times are GMT -4. The time now is 02:09 AM.


Copyright HomeSeer Technologies, LLC