What are the limits of MQTT broker and mscMQTT

[Thedude]

I have adopted a service lately that produces heavy MQTT traffic like 500 messages per minute. My Mosquito MQTT broker is running on RPi with obviously limited resources. Question is where do these messages go? Are they stored by Mosquito ? Is there a chance for Mosquito to run out of memory or out of processing power and start skipping messages? What will happen if Mosquito runs out of memory? How does mcsMQTT retrieve MQTT messages ? Will mscMQTT skip messages if traffic is heavy ? Will mscMQTT slow down my HS4 server if there is heavy MQTT traffic ?

Thanks,

[Michael McSharry]

Section 14 of mcsMQTT.pdf (user manual) describes performance considerations. Statistics are collected and available as described in Section 14.3. The 40 milliseconds per item is a reasonable ballpark for planning. If you are transmitting JSON payload with 10 JSON keys per message at a rate of 500 messages/minute then you will exceed the ability of HS update to be maintained. If you are transmitting simple payloads then 500 should be doable.

I suggest continuing to use Mosquitto for the broker as it has a broader user base and a likely greater focus on efficiency, but in either case the broker should not have a problem with this message rate. I do not know the design provisions for either broker with respect to exceeding bandwidth. I suspect they will start discarding messages. You will want to user QOS=0 to reduce the burden on the broker.

mcsMQTT message parameters are a user setup option. You would want to keep the receive queue size relatively small to minimize latency. A larger queue is useful for spare peak loads, but if steady state traffic is beyond the processing capability then the queue depth will not make any difference as messages will eventually be discarded and those remaining in the queue will be delayed as the queue is worked through.

mcsMQTT is multithreaded so processing messages is a different process than other HS activities. The issue comes in when HS device needs to be updated then the CPU in the HS thread will be doing the update and not doing something else.

There are other users that have a relatively high MQTT traffic rate, but only have a very limited set of this traffic needs to be reflected in HS devices. In this case the recommended setup is to subscribe on only those topics that are mapped to HS Devices. This means rather than 500 messages/minute to HS there may be only 10% of that. The broker does need to route all traffic, but no need to publish to a client if the client is not subscribed to a topic.

[Thedude]

Michael McSharry

Thanks for the prompt and detailed response.
Still unclear to me how does Mosquito handle messages.
I think if a topic has subscribers than Mosquito hands over the message to the subscriber and doesn't store it.
If a topic doesn't have subscribers than Mosquito would just receive and drop the message.
This is my thinking but I could be wrong as I have been many times before :-) Can you confirm or correct ?

[Wade]

Still unclear to me how does Mosquito handle messages.
I think if a topic has subscribers than Mosquito hands over the message to the subscriber and doesn't store it.
If a topic doesn't have subscribers than Mosquito would just receive and drop the message.

I've found this to be a very helpful resource in learning how mqtt works. Part 4 regarding retained flag and Part 8 Retained Messages may give some insight into your question.

[Thedude]

Wade

Thanks a lot. Very helpful indeed.