Plugin struggles under heavy load?

[mk1 black limited]

Just installed a Happy Bubbles unit in my network. It discovers and publishes all Bluetooth devices within reach to my (local) MQTT-server. As it happens, it discovers _a lot_ of devices, and floods the MQTT-server with several payloads per second. The server seems to handle this fine (at least I think it does, mosquitto_sub shows the published data just fine from the server).

mcsMQTT seems to struggle with this, it misses quite a few topics it is subscribed to and memory usage creeps up from 30MB to more than 300MB within a few hours. A restart of the plugin doesn't help.

Turning off the Happy Bubbles unit 'fixes' the problem (together with a restart of the plugin), but I'd really like to keep it running - is there something in the plugin I can adjust or turn on/off? Or shpuld it actually handle this as is?

[Michael McSharry]

Mark the high rate topics with E (Express) mode checkbox on Association tab. There is a performance benchmark section in the mcsMQTT manual that should give you a reasonable idea of what to expect. Also look at the statistics tab toward the bottom to see how deep the receive queue gets. I should not be growing if it can handle the reception rate.

Another approach that is available is to subscribe to only specific topics so that mcsMQTT would not receive those topics from Happy Bubbles that are not of interest. If you do this then automatic discovery of new topics would be disabled. The specifics of how to set this up should be in the manual.

[mk1 black limited]

It's now been 4 hours since I unplugged the Happy Bubbles unit, and the que is still at about 550000 and going down, Max Que Depth is at almost 700000. I guess it can't handle the traffic Happy Bubbles generate. And to think I have 3 more Happy Bubbles units I was going to add...

I'll have a look at the manual. Thanks!

[Michael McSharry]

Does it still get backlogged with Express mode on the Happy Bubble topics?

All you should need to do to stop discovery, and subscribed to only accepted messages, is to select the Topic Discovery radio on the General Tab, Inbound section. Restart the plugin just to clear up a backlogged queue and assure the broker is no longer sending unwanted topics.

[Michael McSharry]

Are you aware that mcsMQTT has a built in support for Bluetooth presence detection and location trilateration? It is based upon the ESP32 microcontroller that can be obtained for under $5 and uses the free firmware described in the mcsMQTT manual Section 15.15.

[mk1 black limited]

The Happy Bubbles were essentially just a test, I've stopped using it (for now, anyhow). It's kinda stupid to have units flooding the MQTT-server anyway (should be able to turn off reporting units that are new/unrecognized). Haven't tried the Express mode, too much clicking

Yes, I did see you've added presence detection, it was just that I had these Happy Bubbles lying around...

[Michael McSharry]

Express mode "clicking" is at the Topic level and not the individual JSON item level so there should not be that many topics from a single source. But I do agree that it is a poor system design that floods messages.