How to sync publish and subscribe?

[Cor]

Hi Michael,

I am doing measurements with sensors mounted on ESP8266 nodes. This is working fine. One of the sensors is measuring atmospheric pressure (baro) and this value is published with topic "Sensor/Buiten/baro". All sensors publish every minute there values.

Now in another ESP node I want to use baro to do some calculations. I subscribe to "Sensor/Buiten/baro" in order to get the value. Now with the time interval of 1 minute it is almost impossible to get the value. If I go to time intervals of 10 seconds then I can get the payload of "Sensor/Buiten/baro".

Is there another way to get this working without all the traffic every 10 seconds?

Regards, Cor

[Michael McSharry]

My guess is there is something else happening that is causing the loss of data that just happens to be masked with higher data rates.

To isolate I would start with a third client that the MQTT broker serves. mcsMQTT could be this client. I'm guess you already are doing this based upon your first paragraph where one-minute updates are being received. If I am correct then the issue becomes your second ESP node. Its internal receive logic appears to be misplacing the data. A serial connection or other debug method to place selective print statements.

Wireshark or tcpdump are good tools for network analysis. This will allow you to confirm that the broker is sending data reliably to the second ESP.

Another technique to improve the communication certainty is to increase the Quality Of Service of the published topic. The default is 0 (fire and forget junk mail). 1 is next level like mail with tracking. 2 is like a certified letter where the recipient needs to acknowledge. Higher QOS places greater network and client burden.

[vasrc]

Also, MQTT-Spy is your friend...
https://www.eclipse.org/paho/components/mqtt-spy/

[Guest]

Or MQTTsnooper if you are using Android. Simple login/sub interface and you can easily filter messages to monitor/diagnose issues.

[ksum]

Could there be something in the Arduino code that is either clogging memory, putting the WiFi module to sleep, or something else so you just do not receive the command because of that? I am assuming this is custom code on the Arduino and not one of the HomeSeer plugins communicating with the unit. What if you set the interval to 20 seconds? If that works, try 30. If it does not work, try 25. See how many seconds it takes before it stops working and see if that helps pinpoint anything.

[Cor]

Thank you all for the suggestions.

Let me explain my situation: I have a separate broker (Mosquitto, RaspberryPi3) and about 10 clients, ESP8266 and ESP32. I use the mcsMQTT plugin with my HS4 HomeSeer system (RaspberryPi4). The clients are programmed via the Arduino interface.

All clients except one perform a variety of measurements, most of them are also HS devices, these end up in 39 associations. All clients publish there measurements every 1 minute. For this reason there is a delay time in the loop subroutine of the Arduino code.

In two of the ten cliens I need the barometric pressure (baro) for some calculations, for this I subscribe in the code to the topic 'Sensor/Buiten/baro'. In these clients also measurements and calculated data are published with a 1 minute interval.

In these two clients with the 1 minute interval I never see a payload coming. If I decrease the interval time (delay) to 10 seconds, then I do see the payload. The topic 'Sensor/Buiten/baro' is still published every 1 minute.
@ksun: if I increase to 20 seconds subsciption its over again, no more payloads.

How I see it: the chance to 'see' the published payload is bigger with the 10 seonds interval. But is this how it works? I do not fully understand how things work. If I subscribe to a topic, can I only see the payload if this is published at the same time? Or is the information available (stored?) at the broker?

As I understood from Michael the QoS does influence this but experimenting with this in the 'Outbound (Publish) Management' paragraph did not make any difference. I probably did not make the right settings because my knowledge of this matter is still to little. Forgive me.

I can live with this situation but I wonder what the impact is on the traffic, how much can the broker handle? Is one of you able to guess if in my situation the traffic is to busy or am I far from a sort of overload?

Hope you can help.
-- Cor --

[vasrc]

As Michael said, in your case, increasing the QoS should be done to ensure delivery.

I'll assume you have a delay in your Loop subroutine??? If so:

Typically delays in the Loop subroutine are not a good thing to do, for exactly the reason you stated, possible timing issues.

If you're trying to only Publish every minute, you should build a timer (using millis()) and publish when it's time to, not waiting for a minute in the loop. This way you'll execute your loop as fast as possible and won't miss a Subscription.

Here's an example. I have one of these for every time period I need (second, minute, 5 min, 15 min, 30 min, hour, day, etc). Probably a more elegant way to do it, but it's easy to read and debug.

/* 1 second */
LoopNow = millis();

/* 1 second */
if (LoopNow - lastSecMsg > 1000) {
lastSecMsg = LoopNow;

execute code
}

/* Send every minute */
if (LoopNow - lastMinMsg > 60000) {
lastMinMsg = LoopNow;

execute code here

}


Z

[vasrc]

Humm...
What MQTT library are you using (Pubsublclient??)

I believe the ReceiveCallback subroutine should still be able to capture the subscription when it's sent even if the Loop is busy/delayed..
Do you have USB access to the ESP8266? If so, can you stick a Serial.print in the callback routine to see if it's being received using the Serial monitor window in the Arduino IDE.

Z

[Cor]

Hi Howard,
thanks a lot for your suggestion. I will remone all the delays I do have now in the loop routines.
Your sample code works perfect! I see the baro payload every minute now!


yes, I am using Pubsublclient and I can see a payload coming in.

-- Cor --

[Cor]

Now about the QoS. All is working OK but hoe can I switch QoS on? Somewhere here?



I did try several things like retain checked, default QoS to At least but it did not make any difference. How can I see if it is an improvement?

-- Cor --

[vasrc]

Here's a good overview of QoS. You have to consider both the Publish AND Subscribe portions:

https://www.hivemq.com/blog/mqtt-ess...ervice-levels/

[ksum]

LOL. Glad I refreshed this before sending that same link!

Note that since you are sending messages every minute you probably want QoS 0, which is the At Most option you have selected. If you changed your code to only send when there was a change outside of a tolerance, then you may want to add QoS and that article can best guide you on the level. Read them from left to right at 0, 1, 2 but they do match the common terms used in the article.

[Michael McSharry]

The QOS is set in the client that does the publishing so the mcsMQTT settings for QOS would not apply in this situation. For periodic data a QOS of 0/At Most/Fire-and-Forget is likely the most appropriate. Higher levels may be appropriate for state or event-based information.

[Cor]

To conclude, I have learned a lot in this topic. Thank you all for your contributions. The timing as suggested by Howard in post #7 is a huge improvement.

Just because my local WiFi is very fast and stable and the remarks of Michael in post #13 I am using a QoS of 0.

-- Cor --