Hosed my mcsMQTT Install.

[ksum]

Added OpenMQTT Devices and hosed something. HomeSeer web interface is now slow to unresponsive and mcsMQTT is hovering around 33% CPU usage. If I disable mcsMQTT things work better. Tried uninstalling and reisntalling the plugin. Didn't do any reboots in between, though. Maybe I should have...

I did unplug my 2 OpenMQTTGateways to see if it was a traffic issue and I am not seeing a lot of MQTT traffic on the network. Both of these were on most of the day without issue. Occurred while trying to get a virtual device to control one of the OpenMWTTGateways and troubleshooting an issue on that. Any thoughts on repairing this?

Attached are todays mcsMQTT debug logs. The zipped docuent for file 7 is too large for attaching. It and a zip of hte db file, as well as what is attached here, is available for download on my Google Drive at https://drive.google.com/drive/folde...Mz?usp=sharing For those reading this in the future, please note: These files will be removed at some point.

[Michael McSharry]

The debugs do not show much other than the most current indicating 5 minutes and still going through initialization. Based upon prior experience my guess is OpenMQTTGateway flooded the network and resulted in a request for mcsMQTT to manage well beyond the number of endpoints than it expected.

\Data\mcsMQTT\mcsMQTT.db is likely where your problem is right now. If you view it with DB Browser for SQLite or other tool you will likely find a boatload of topics for which you have no interest. mcsMQTT needs to evaluate each during initialization.

To repair, restore an earlier mcsMQTT.db from backup or run a query that removes the OpenMQTTGateway topics on the current database. The idea of more is better does not work in this case. Just because a message can be delivered does not mean it should. The ideal situation is to tame OpenMQTTGateway so you can select what you have interest before it sends it to the MQTT Broker and then gets fanned out to all the MQTT clients. IT shoudl also be sending to the Broker and then at a rate that is consistent with your needs.

You likely do not need a slew of messages every second. There have been multiple benchmarks done for mcsMQTT performance. I have not done it recently, but what is documented in mcsMQTT.pdf is 32 milliseconds per message for simple messages and it increases for JSON that have multiple endpoints. If HS has minimal involvement, then this decreases to 6 ms. I think this benchmark was done on a RPi 3, but is indicative of any host platform.

If you post or email mcsMQTT.db (zipped) then I can confirm and also run the query to get it back to something reasonable.

[ksum]

Sounds as if it is the ble root topic items from the Bluetooth gateway. I know I need to throttle that IF I use the Bluetooth gateway. I will look into how to only send specific topics from both gateways. I'm guessing the big problem comes from adding a new device. Is there a setting in mcsMQTT which could help avoid this when searching for a new device with an OpenMQTTGateway gateway?

The zip file is big. To large to attach. Here are links to it on Google drive. I can try SQLlite once I get back to a keyboard. Might be able to review in with my phone later tonight as well.

File (may try to open it)
https://drive.google.com/file/d/1-CN...w?usp=drivesdk

Folder containing zip file:

https://drive.google.com/drive/folde...Mz?usp=sharing

[Michael McSharry]

On General tab, there is a setting to restore to specific topics after some period of time. This way you can enable discovery, but limit its duration. Even with this OpenMQTTGateway could be too agressive.

Your database has 141,380 records of which are 1200 are misc topics, about 3000 are ble/OpenMQTTGateway and the remainder are home/OpenMQTTGateway/# messages. After removing OpenMQTTGateway there are over 4000 records. This condensed database is at http://mcsSprinklers.com/mcsMQTT.db.zip

[ksum]

Wow! This is a shock! I thought the ble set would be larger than the rest. I was very certain there were many more ble topics as opposed to the RF topics. I also created an IR gateway but it isn't receiving anything yet. It will, however, send IR codes so I don't have to build something to do that. (Refer to my Broadlink post). Is the database only holding records where Discovery is enabled or devices are created? OpenMQTTGateway has the ability to turn sending discovery on or off. I think it is off for the ble gateway but was on for the RF gateway. This would explain the difference. Or if the json data is busted into multiple records. I'm remoted into the honest computer via my phone

When I started experiencing issues I was creating HomeSeer Devices to turn this Off/On as well as make other settings. If I want to remove all the devices and keep the topics specific to the gateway itself, the statements below will delete all records where these topics start with the text strings in the LIKE, correct? Am I looking at the correct table and field?

DELETE from MQTT_MESSAGE WHERE (lower(Source) LIKE 'home/openmqttgateway_1/rtl_433%');


DELETE from MQTT_MESSAGE WHERE (lower(Source) LIKE 'ble/openmqttgateway/bttomqtt%');

[Michael McSharry]

A database record is created for each endpoint. JSON payload typically contains multiple endpoints.

I believe the SQL syntax is correct. When I tried to use it on your database I received a read-only protection error so I just sorted the table by topic and selected the home and ble topics visually and deleted them using DB Browser for SQLite.

I did not try to use the lower() function and just had the case as was in the Source column

[ksum]

Thank you. Things are much better now!

I need to look into the openMQTTGateway more and see what the available options are to limit traffic. It seems there is a white/blacklist for BLE but may not be one for RF gateways. I will keep digging. I may have to go with a less powerful antenna if I use this at all.