Announcement

**Michael McSharry** · August 27th, 2018, 08:27 PM

I suspect the error log is an effect and not a cause with the cause being the loss of connection to the broker. The broker uses ICMP to assess connectivity to the client (mcsMQTT). The broker will drop connection if ICMP response is not received from the HS computer. mcsMQTT polls at 10 second intervals for the connection to the broker. If the connection as been dropped it will attempt to reconnect. There is information in the mcsMQTT debug log related to connection attempts. I do not know where your 30 second heartbeat comes into play. What recovery actions are you taking when the heartbeat is not present?

It seems you are essentially trying to diagnose a network issue. Wireshark is a good tool to provide visibility of network activity.

**onepointbrewing** · August 27th, 2018, 08:54 PM

Hey Michael,

Thanks for replying so promptly.

The heartbeat Im referring to is just something I send out to all clients from HS to get a visual representation of whether mcsMQTT/HS is still connected to the broker - or some other connectivity issue. I actually send it and my phone picks it up through tasker while im at work. I use the timestamp in the heartbeat as the "last connected". If the heartbeat is more than 30s old, I know its disconnected.

"There is information in the mcsMQTT debug log related to connection attempts." can you give me a heads up on what I should be looking for in the log. I can see the connection, no worries, ive just got to wait for the disconnect to figure it out. Also, is it possible to make the log bigger?

**Michael McSharry** · August 27th, 2018, 09:15 PM

The thread that maintains broker connection is below. The socket level transactions are in m2mqtt.dll which was updated by another user on this board for SSL operation. The source for this is on Github.com
https://github.com/eclipse/paho.mqtt...ive/master.zip
What is the heartbeat protocol you are using?

Code:

    Sub StartMQTTThread()
        Try
            Dim sResponse As String() = {"Accepted", "Refused, unacceptable protocol version", "Refused, identifier rejected", "Refused, server unavailable", "Refused, bad username/password", "Refused not authorized"}
            Dim iBrokerResponse As Byte = 0
            Dim iMessageCount As Integer = 0
            Dim bConnected As Boolean = False
            Dim bFirstPass As Boolean = True
            Dim bSubscribed As Boolean = False
            Dim dStartTime As Date = Now

            If gDebugLog Then
                Dim s As String
                If oMQTTClient IsNot Nothing Then
                    s = oMQTTClient.IsConnected.ToString
                Else
                    s = "False"
                End If
                hsWritelog(PLUGIN_DEBUG, "MQTT Thread Started with broker " & gMQTTBroker & ", Shutdown=" & gShutdown.ToString & ", Disconnect=" & gDisconnect.ToString & ", Client=" & (oMQTTClient IsNot Nothing).ToString & ", Connected=" & s)
            End If

            Do
                If gShutdown Then
                    If gCreateStatusDevices Then
                        hs.SetDeviceValueByRef(gStatusRef(Statistics.BrokerStatus), 0.0, True)
                    End If
                    Exit Sub
                End If

                If gValueCallback Then
                    RegisterEvents(1)
                End If
                If gStringCallback Then
                    RegisterEvents(2)
                End If

                If Not gDisconnect AndAlso gMQTTBroker <> "" Then

                    If oMQTTClient Is Nothing OrElse Not oMQTTClient.IsConnected Then
                        If bFirstPass Then
                            hsWritelog(PLUGIN_DEBUG, "MQTT Thread Not Connected Yet")
                        End If
                        If gCreateStatusDevices Then
                            hs.SetDeviceValueByRef(gStatusRef(Statistics.BrokerStatus), 0.0, True)
                        End If

                        Dim bSecure As Boolean = False
                        Dim caCert As X509Certificate2 = Nothing
                        Dim clientCert As X509Certificate2 = Nothing
                        Dim sslProtocol As MqttSslProtocols = MqttSslProtocols.None

                        If gMQTTBrokerSSL <> MqttSslProtocols.None Then 'AndAlso gMQTTBrokerCaCert <> "" Then 'AndAlso gMQTTBrokerClientCert <> "" Then
                            Dim sCert As String = "CaCert"
                            Try
                                hsWriteLog(PLUGIN_DEBUG, "Installing CA Cert - " & gMQTTBrokerCaCert)
                                If gMQTTBrokerCaCert <> "" Then
                                    caCert = New X509Certificate2(gMQTTBrokerCaCert)
                                    ' Get the value.
                                    Dim resultsTrue As String = caCert.ToString(True)
                                    ' Display the value to the console.
                                    hsWriteLog(PLUGIN_DEBUG, "CACert Results = " & resultsTrue)
                                End If
                                sCert = "Client Cert"
                                If gMQTTBrokerClientCert <> "" Then
                                    hsWriteLog(PLUGIN_DEBUG, "Client Cert - " & gMQTTBrokerClientCert)
                                    'clientCert = X509Certificate.CreateFromCertFile(gMQTTBrokerClientCert)
                                    clientCert = New X509Certificate2(gMQTTBrokerClientCert, gMQTTBrokerClientCertPW)
                                    Dim resultsTrue As String = clientCert.ToString(True)
                                    ' Display the value to the console.
                                    hsWriteLog(PLUGIN_DEBUG, "Client Cert Results = " & resultsTrue)
                                End If
                                sslProtocol = gMQTTBrokerSSL
                                bSecure = True
                            Catch ex As Exception
                                hsWritelog(PLUGIN_DEBUG, sCert & " issue " & ex.Message)
                            End Try
                        End If
                        hsWriteLog(PLUGIN_DEBUG, "Calling MQTTclient")
                        Try
                            oMQTTClient = New MqttClient(gMQTTBroker,
                            gMQTTBrokerPort,
                            bSecure,
                            caCert,
                            clientCert,
                            sslProtocol,
                            New Security.RemoteCertificateValidationCallback(AddressOf ValidateBrokerCertificate))
                            'New Security.LocalCertificateSelectionCallback(AddressOf SelectClientCertificate))
                        Catch ex As Exception
                            hsWriteLog(PLUGIN_NAME, "StartMQTT Client attempt Failed to Broker" & gMQTTBroker & ex.Message & ":" & ex.InnerException.Message)
                        End Try

                        bSubscribed = False
                        If bFirstPass Then
                            hsWriteLog(PLUGIN_DEBUG, "MQTT Thread Client Created")
                        End If

                        AddHandler oMQTTClient.MqttMsgPublishReceived, AddressOf client_MqttMsgPublishReceived
                        Dim clientId As String = Guid.NewGuid().ToString()
                        If bFirstPass Then
                            hsWriteLog(PLUGIN_DEBUG, "MQTT Thread Client ID=" & clientId)
                        End If
                        hsWriteLog(PLUGIN_DEBUG, "Calling MQTT Connect")
                        Try
                            iBrokerResponse = oMQTTClient.Connect(
                                clientId,
                                gMQTTBrokerUsername,
                                gMQTTBrokerPassword,
                                gDefaultRetain,
                                gDefaultQOS,
                                True,
                                gThisComputer & "/" & PLUGIN_NAME & "/LWT",
                                "Offline",
                                True,
                                60)  'False,, 'Messages.MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE,
                            If bFirstPass Then
                                hsWriteLog(PLUGIN_DEBUG, "MQTT Thread Broker " & gMQTTBroker & " Connect Response=" & iBrokerResponse.ToString)
                            End If

                        Catch ex As Exception
                            hsWritelog(PLUGIN_DEBUG, "StartMQTT Connection attempt1 to Broker. Exception: " & ex.ToString() & " / InnerEx: " & ex.InnerException.ToString())
                            If gDebugLog Then
                                hsWritelogEx(PLUGIN_DEBUG, "StartMQTT Connection attempt1 to Broker " & gMQTTBroker, ex.Message & ":" & ex.InnerException.Message)
                            End If
                            oMQTTClient = Nothing
                        End Try

                        If iBrokerResponse < 0 OrElse iBrokerResponse > 5 Then
                            hsWritelog(PLUGIN_DEBUG, "StartMQTT Connection attempt2 Failed to Broker. BrokerResponse:" & iBrokerResponse.ToString)
                        End If

                        If oMQTTClient IsNot Nothing Then
                            If iBrokerResponse < 0 OrElse iBrokerResponse > 5 Then
                            Else
                                If iMessageCount < 5 Then
                                    iMessageCount = iMessageCount + 1
                                    hsWritelog(PLUGIN_DEBUG, "MQTT Broker Connection " & sResponse(iBrokerResponse) & ", Connected=" & oMQTTClient.IsConnected.ToString)
                                End If
                            End If

                            If oMQTTClient.IsConnected Then
                                hsWritelog(PLUGIN_NAME, "MQTTClient is Connected")
                                If iBrokerResponse <> 0 Then
                                    oMQTTClient = Nothing
                                Else
                                    gOnlineTime = Now
                                    If gCreateStatusDevices Then
                                        hs.SetDeviceValueByRef(gStatusRef(Statistics.BrokerStatus), 1.0, True)
                                    End If
                                    Dim sTopic As String = gThisComputer & "/" & PLUGIN_NAME & "/LWT"
                                    Dim sPayload As String = ONLINE
                                    oMQTTClient.Publish(sTopic, Encoding.UTF8.GetBytes(sPayload), gDefaultQOS, gDefaultRetain)
                                    StatPublish(sTopic, sPayload, False)
                                    If gDebugLog Then
                                        hsWritelog(PLUGIN_DEBUG, "MQTT Thread Subscribing ")
                                    End If
                                    bSubscribed = MQTTSubscribe()
                                    RaiseConnectionEvent(True)
                                    bConnected = True
                                End If
                            End If
                        Else
                            oMQTTClient = Nothing
                        End If 'just connected
                    End If 'previously connected
                Else ' Not gDisconnect AndAlso gMQTTBroker <> "" 
                End If 'disconnect
                bFirstPass = False
                'use this thread as timer to manage periodic activities

                If gLastDay <> Now.Day Then
                    gReceiveMessageCountToday = 0
                    gPublishCountToday = 0
                    gReceiveAcceptedCountToday = 0
                    If gCreateStatusDevices Then
                        hs.SetDeviceValueByRef(gStatusRef(Statistics.PublishCountToday), 0.0, True)
                        hs.SetDeviceValueByRef(gStatusRef(Statistics.ReceiveAcceptedCountToday), 0.0, True)
                        hs.SetDeviceValueByRef(gStatusRef(Statistics.ReceiveNotAcceptedCountToday), 0.0, True)
                    End If
                    PruneHistory(DateAdd(DateInterval.Day, -gDaysOfHistory, Now))
                    PruneRef(DateAdd(DateInterval.Day, -gDaysOfHistory, Now))
                    For Each sSource As String In AccumResetList
                        If MQTTReceiveDictionary.ContainsKey(sSource) Then
                            Dim oMQTT As MqttReport = MQTTReceiveDictionary(sSource)
                            Dim oTransform As Transform = oMQTT.Transform
                            Dim iRef As Integer = oTransform.AccumDevice
                            If oTransform.AccumDevice = AccumTypes.ResetDelta Then
                                Dim dv As Scheduler.Classes.DeviceClass = hs.GetDeviceByRef(iRef)
                                If dv IsNot Nothing Then
                                    Dim iValue As Double = dv.devValue(hs)
                                    oTransform.AccumMidnight = iValue
                                    SaveTransform(sSource, oTransform)
                                    oMQTT.Transform = oTransform
                                End If
                            End If
                            hs.SetDeviceValueByRef(iRef, 0.0, True)
                        End If
                    Next
                End If
                gLastDay = Now.Day
                hs.SaveEventsDevices()

                Threading.Thread.Sleep(10000)

                If oMQTTClient IsNot Nothing AndAlso oMQTTClient.IsConnected AndAlso Not bSubscribed Then
                    bSubscribed = MQTTSubscribe()
                End If

                If gTotalMessagesReceived > 0 Then
                    Dim iElapsedTime As Long = DateDiff(DateInterval.Second, dStartTime, Now)
                    gAverageReceiveInterval = (iElapsedTime * 1000) / gTotalMessagesReceived
                Else
                    gAverageReceiveInterval = 0
                End If
                If bConnected AndAlso (oMQTTClient Is Nothing OrElse Not oMQTTClient.IsConnected) Then
                    If gCreateStatusDevices Then
                        hs.SetDeviceValueByRef(gStatusRef(Statistics.BrokerStatus), 0.0, True)
                        hs.SetDeviceValueByRef(gStatusRef(Statistics.AverageReceiveTime), CType(gAverageReceiveMilliseconds, Double), True)
                        hs.SetDeviceValueByRef(gStatusRef(Statistics.AverageReceiveInterval), CType(gAverageReceiveInterval, Double), True)
                    End If
                    RaiseConnectionEvent(False)
                    bConnected = False
                End If

                Try
                    Dim oList As New List(Of String)
                    For Each kvp As KeyValuePair(Of String, MQTTASP) In WebPageDictionary
                        If kvp.Value IsNot Nothing Then
                            If DateDiff(DateInterval.Minute, kvp.Value.PostbackDate, Now) > 5 Then
                                oList.Add(kvp.Key)
                            End If
                        End If
                    Next
                    For Each sKey As String In oList
                        If WebPageDictionary.ContainsKey(sKey) Then
                            WebPageDictionary.Remove(sKey)
                            If gDebugLog Then
                                hsWritelog(PLUGIN_DEBUG, "Removed expired web page")
                            End If
                        End If
                    Next

                Catch ex As Exception
                    hsWritelogEx(PLUGIN_DEBUG, "Trying to remove expired web page " & WebPageDictionary.Count.ToString, ex.Message)
                End Try
            Loop

        Catch ex As Exception
            hsWritelogEx(PLUGIN_NAME, "StartMQTTThread", ex.Message)
        End Try
    End Sub

**onepointbrewing** · August 28th, 2018, 06:34 AM

Hey Michael,

Ive been using a number of cloud services just recently because my ISP uses a double NAT scenario which means I couldnt use anything outside the first layer of NAT - so I had to use cloudmqtt (using SSL).

BUT - in the last 4 hours - Ive just managed to get a staticIP for my from my ISP and sparked up a RPI and will be testing it using that. Lets see how we go. So far so good.

The heartbeat - hehehehe - something super simple - 30s timer in HS - publish a string to a private topic - tasker then compares thatt message timestamp to the previous timestamp - if it exceeds 45 seconds - houston we have a problem.

By doing it that way i can check the whole chain - including if HS hangs. Stupid Simple, didnt have to think about any code etc - simple and it works.

Announcement

Continual disconnects

Continual disconnects

Comment

Comment

Comment

Comment