Wildcard Template for Auto-Association of MQTT control topics to existing HS devices

[Loafdude]

Subscriptions seem to work now! (Thanks!)
Changing the subscription wildcard field does appear enumerate the devices

But wildcard publishing is now broken?

I can recreate by this;
1. Delete contents of Data/mcsMQTT/
2. Set publish wildcard in general tab
3. Test devices and check for mqtt messages

At this point no mqtt messages are sent

4. Open device on Edit/Add tab with REF number
5. Change nothing (The publish field is already populated correctly)
5. Test devices and check for mqtt messages

Messages are now sent

[Michael McSharry]

Why are you deleting the database? Auto-Associations are only made when a message is received so if you do not see the relationship on the Association tab, then there is no association yet. Could it be that it is just a timing thing between when you deleted and restarted vs when subscription topics have been received?

[Loafdude]

I've been deleting the database because the last version was sending me for a loop and it was easier to test consistently with a fresh db. haha

My fault. I should read the field names more closely.
I was expecting the Default Topic Template to behave like the subscription wildcard where it would publish everything automatically.

Apologies Sir!

Thanks for your hard work on this update and humoring my requests.

Is %20 fix on your radar or should I workaround it on my end?

[Michael McSharry]

I do not know what you mean by a fix for %20. If I publish a topic that contains %20 from another node then that topic shows up with %20 in the topic in mcsMQTT. What are you doing that is not giving you the expected result?

It is possible to auto-create a status response for every HS device, but for those who have many devices that would create a lot of undesired traffic and use of system resources. It then gets to be problematic as edits are made to individual associations and then as the default publish template is changed. Would all the existing devices have their status/publish topic changed or would the edit to the default only change the new devices created thereafter. Overall it likely would be confusing.

[Loafdude]

In hs/mcsMQTT the $$NAME:, $$type:, etc variables have their spaces replaced by %20s when published
When sending spaces in other clients they are still spaces on the other end (including when sending to mcsMQTT)

Yes I got myself confused. The way you've written it makes sense.

[Michael McSharry]

On the publish side this is intentional. /, # and + are HTML-encoded because they are part of the MQTT syntax. Space is also based upon the following recommendation. What I can do is provide an option for the Space to be either encoded or unencoded on the next update. Never use spaces in a topic

A space is the natural enemy of every programmer. When things are not going the way they should, spaces make it much harder to read and debug topics. As with leading forward slashes, just because something is allowed, doesn’t mean it should be used. UTF-8 has many different white space types, such uncommon characters should be avoided.

[Loafdude]

Good point.
I'll just handle it on the other end. Don't bother implementing for me.

[Michael McSharry]

It was an easy addition with option on the General Tab / Outgoing messages. I am actually surprised the subject never came up before since a space is a common character in locations and names. http://mcsSprinklers.com/mcsMQTT_3_4_15_0.zip

[cytec]

This Node-Red flow works for on / off control of a HomeSeer device via REF

[{"id":"61ef33df.d4748c","type":"mqtt out","z":"a92a988d.6dafa8","name":"","topic":"/homeseer/Control/105/Command","qos":"","retain":"","broker":"5e06ebd0.e4e9d4","x" :780,"y":500,"wires":[]},{"id":"ad3232e6.9bf6b","type":"inject","z":"a92a988d.6dafa 8","name":"","topic":"","payload":"On","payloadType":"str"," repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":570 ,"y":500,"wires":[["61ef33df.d4748c"]]},{"id":"5e06ebd0.e4e9d4","type":"mqtt-broker","z":"","name":"MQTT","broker":"192.168.0.5","port":" 1883","clientid":"","usetls":false,"compatmode":true,"keepal ive":"60","cleansession":true,"birthTopic":"","birthQos":"0" ,"birthPayload":"","closeTopic":"","closeQos":"0","closePayl oad":"","willTopic":"","willQos":"0","willPayload":""}]

I entered this into mcsMQTT plugin > General tab "Wildcard Non-Plugin Control Template" which is under the "Inbound (Subscription) Management" section:

/homeseer/Control/$$REF:/Command

[Michael McSharry]

I have never started to use node-red so the flow does not mean much to me. What I get from it is that the topic is "/homeseer/Control/105/Command" and the payload in "On". I think you are indicating that it is working for you.

As a long term consideration you may want to use human names for identification (e.g. Room & Name) rather than computer names (Device Ref). If for some reason the device at 105 needs to be recreated then you will never again get it to be 105, but will be the next available number assigned by HS3. You can control the Room, Floor, and Name. $$ROOM:, $$FLOOR:, and $$NAME: are the substitution patterns for these in mcsMQTT. Table 1 of the manual will give you the set of available substitution patterns.

[cytec]

@Michael McSharry

I actually use Domoticz as the core of my home automation system, everything feeds back into it either via Node-Red or the JSON interface.

I agree with your suggestion, RE using names over the ref ID.

My full project is to create a seemless interface between HomeSeer and Domoticz (via Node-Red).

I've already succeeded in automatically:

* If a Domoticz virtual device is switched on or off, see if there is an identical named device in HomeSeer and set it to the same (using Node-Red and mcsMQTT).



Goal is to have devices (switches, motion sensors, door sensors etc...) all automatically match up between Domoticz and HomeSeer if they are named the same.

Also I can't see any noticeable delay in controlling a Z-Wave device via Domoticz > MQTT > Node-Red > HomeSeer which is fantastic!

This way I can use HomeSeer for it's fantastic certified Z-Wave support but have all the flexibility of Domoticz.

I guess ultimately it's personal preference, but after 4 years of using OpenZWave, suffice to say I am not a fan of it.

I'll post a Github link once I've finished the interface.

[cytec]

@Michael McSharry

Found a bug in the implementation, see attached screenshot.

If attempting to control "/homeseer/Control/Office/Command" and the top device is named "Office Root", mcsMQTT attempts to control "Office Root" instead of "Office".

There is somewhere a problem with the $$NAME: code, $$REF: works fine (obviously my application depends on ultimately using the $$NAME.

[cytec]

Found another problem, if the tick box in this screenshot is ticked, I get status updates via MQTT, if it's not then no status updates occur.

Isn't it supposed to be automatic for all devices?



MQTT output received via Node-Red (when the checkbox above is ticked).



For this application, I wonder if it would be better if the plugin was massively simplified, eg remove tabs:

Associations, Edit/Add, Pub List, Statistics, History, Chart

There really isn't even need for defining the wildcards, it could just be hard coded into the plugin.

User only needs to enter their MQTT server settings?

In Domoticz MQTT implementation, this is all the user needs to fill out, everything else is fully automatic.

We can control any Domoticz device via MQTT topic domoticz/in

Anytime a device status changes, it is automatically published to MQTT topic domoticz/out

[Michael McSharry]

For office, what is the actual Hs device name property for the root device? While HS provides a display of office root it does not mean the name is “office root”, but likely is “office” and Hs appends “root” in the display to distinguish it from the child. Take a look and let me know. Note also the association tab will show the associations that have been automatically added.

Domoticz looks to be designed as if it is king. I see much special logic in the Tasmota source so it can be configured to comply with Domoticz conventions. McsMQTT makes no assumptions about conventions used by other MQTT nodes with the intent of being able to bring any MQTT node into the HS world.

Most HS users will have a small percentage of their devices as being MQTT cognizant. Having everything publish will consume system resources and add no value for most users. I could add option for auto accept of non-plugin devices, but I can foresee confusion and difficulty when one desired to change the templates after associations already exist.

[cytec]

@Michael McSharry
I've built 90% of the solution now using Domoticz MQTT and HomeSeer JSON interface (not using mcsMQTT).

Interestingly I can pull the entire HomeSeer device list via JSON using curl in just "0m0.059s", that's just 59 milliseconds to get the status of my 30 or so Z-Wave devices.

Also it's using a very small amount of system CPU < 3% calling it every 100ms, I've decided to continue building the interface upon the concept of requesting the entire device list non-stop from HomeSeer via JSON.

It sounds like waste of resources, but I can turn a light on in Domoticz and it is so fast I cannot see any distinguishable delay between previously using OpenZ-Wave directly with Domoticz or this new concept of off-loading Z-Wave to HomeSeer.

Devices reported via HomeSeer JSON are up to date RE their status, even the motion sensors are working wonderfully again with no distinguishable delay.

I'm surprised but the HomeSeer JSON interface is looking like the best way to go forward, in particular it provides very rich information I can use to further enhance the solution. Eg search for child devices containing a battery value and automatically attach it to the Domoticz device.

Cheers