Announcement

Collapse
No announcement yet.

Continual disconnects

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Continual disconnects

    For about a week Ive had this working however my system will disconnect every few hours.

    It takes a complete reboot of HS to get it to reconnect to my broker. My system uses a heartbeat every 30s to determine connectivity.

    The error log shows this:
    Aug-28 09:04:37 mcsMQTT HandleAction Line 0 Object reference not set to an instance of an object.
    Aug-28 09:04:06 mcsMQTT HandleAction Line 0 Object reference not set to an instance of an object.
    Aug-28 09:03:35 mcsMQTT HandleAction Line 0 Object reference not set to an instance of an object.
    Aug-28 09:03:04 mcsMQTT HandleAction Line 0 Object reference not set to an instance of an object.
    Aug-28 09:02:33 mcsMQTT HandleAction Line 0 Object reference not set to an instance of an object.
    The stats page looks like this:

    MQTT Broker Online Stats

    Last Online @ 2018-08-27 23:03:39
    Offline @ 2018-08-28 08:43:44








    Can anyone help, else im going to have to go back the other MQTT plugin (which is insecure).

    Thanks

  • #2
    I suspect the error log is an effect and not a cause with the cause being the loss of connection to the broker. The broker uses ICMP to assess connectivity to the client (mcsMQTT). The broker will drop connection if ICMP response is not received from the HS computer. mcsMQTT polls at 10 second intervals for the connection to the broker. If the connection as been dropped it will attempt to reconnect. There is information in the mcsMQTT debug log related to connection attempts. I do not know where your 30 second heartbeat comes into play. What recovery actions are you taking when the heartbeat is not present?

    It seems you are essentially trying to diagnose a network issue. Wireshark is a good tool to provide visibility of network activity.

    Comment


    • #3
      Hey Michael,

      Thanks for replying so promptly.

      The heartbeat Im referring to is just something I send out to all clients from HS to get a visual representation of whether mcsMQTT/HS is still connected to the broker - or some other connectivity issue. I actually send it and my phone picks it up through tasker while im at work. I use the timestamp in the heartbeat as the "last connected". If the heartbeat is more than 30s old, I know its disconnected.

      "There is information in the mcsMQTT debug log related to connection attempts." can you give me a heads up on what I should be looking for in the log. I can see the connection, no worries, ive just got to wait for the disconnect to figure it out. Also, is it possible to make the log bigger?

      Comment


      • #4
        The thread that maintains broker connection is below. The socket level transactions are in m2mqtt.dll which was updated by another user on this board for SSL operation. The source for this is on Github.com
        https://github.com/eclipse/paho.mqtt...ive/master.zip
        What is the heartbeat protocol you are using?

        Code:
            Sub StartMQTTThread()
                Try
                    Dim sResponse As String() = {"Accepted", "Refused, unacceptable protocol version", "Refused, identifier rejected", "Refused, server unavailable", "Refused, bad username/password", "Refused not authorized"}
                    Dim iBrokerResponse As Byte = 0
                    Dim iMessageCount As Integer = 0
                    Dim bConnected As Boolean = False
                    Dim bFirstPass As Boolean = True
                    Dim bSubscribed As Boolean = False
                    Dim dStartTime As Date = Now
        
                    If gDebugLog Then
                        Dim s As String
                        If oMQTTClient IsNot Nothing Then
                            s = oMQTTClient.IsConnected.ToString
                        Else
                            s = "False"
                        End If
                        hsWritelog(PLUGIN_DEBUG, "MQTT Thread Started with broker " & gMQTTBroker & ", Shutdown=" & gShutdown.ToString & ", Disconnect=" & gDisconnect.ToString & ", Client=" & (oMQTTClient IsNot Nothing).ToString & ", Connected=" & s)
                    End If
        
                    Do
                        If gShutdown Then
                            If gCreateStatusDevices Then
                                hs.SetDeviceValueByRef(gStatusRef(Statistics.BrokerStatus), 0.0, True)
                            End If
                            Exit Sub
                        End If
        
                        If gValueCallback Then
                            RegisterEvents(1)
                        End If
                        If gStringCallback Then
                            RegisterEvents(2)
                        End If
        
                        If Not gDisconnect AndAlso gMQTTBroker <> "" Then
        
                            If oMQTTClient Is Nothing OrElse Not oMQTTClient.IsConnected Then
                                If bFirstPass Then
                                    hsWritelog(PLUGIN_DEBUG, "MQTT Thread Not Connected Yet")
                                End If
                                If gCreateStatusDevices Then
                                    hs.SetDeviceValueByRef(gStatusRef(Statistics.BrokerStatus), 0.0, True)
                                End If
        
                                Dim bSecure As Boolean = False
                                Dim caCert As X509Certificate2 = Nothing
                                Dim clientCert As X509Certificate2 = Nothing
                                Dim sslProtocol As MqttSslProtocols = MqttSslProtocols.None
        
                                If gMQTTBrokerSSL <> MqttSslProtocols.None Then 'AndAlso gMQTTBrokerCaCert <> "" Then 'AndAlso gMQTTBrokerClientCert <> "" Then
                                    Dim sCert As String = "CaCert"
                                    Try
                                        hsWriteLog(PLUGIN_DEBUG, "Installing CA Cert - " & gMQTTBrokerCaCert)
                                        If gMQTTBrokerCaCert <> "" Then
                                            caCert = New X509Certificate2(gMQTTBrokerCaCert)
                                            ' Get the value.
                                            Dim resultsTrue As String = caCert.ToString(True)
                                            ' Display the value to the console.
                                            hsWriteLog(PLUGIN_DEBUG, "CACert Results = " & resultsTrue)
                                        End If
                                        sCert = "Client Cert"
                                        If gMQTTBrokerClientCert <> "" Then
                                            hsWriteLog(PLUGIN_DEBUG, "Client Cert - " & gMQTTBrokerClientCert)
                                            'clientCert = X509Certificate.CreateFromCertFile(gMQTTBrokerClientCert)
                                            clientCert = New X509Certificate2(gMQTTBrokerClientCert, gMQTTBrokerClientCertPW)
                                            Dim resultsTrue As String = clientCert.ToString(True)
                                            ' Display the value to the console.
                                            hsWriteLog(PLUGIN_DEBUG, "Client Cert Results = " & resultsTrue)
                                        End If
                                        sslProtocol = gMQTTBrokerSSL
                                        bSecure = True
                                    Catch ex As Exception
                                        hsWritelog(PLUGIN_DEBUG, sCert & " issue " & ex.Message)
                                    End Try
                                End If
                                hsWriteLog(PLUGIN_DEBUG, "Calling MQTTclient")
                                Try
                                    oMQTTClient = New MqttClient(gMQTTBroker,
                                    gMQTTBrokerPort,
                                    bSecure,
                                    caCert,
                                    clientCert,
                                    sslProtocol,
                                    New Security.RemoteCertificateValidationCallback(AddressOf ValidateBrokerCertificate))
                                    'New Security.LocalCertificateSelectionCallback(AddressOf SelectClientCertificate))
                                Catch ex As Exception
                                    hsWriteLog(PLUGIN_NAME, "StartMQTT Client attempt Failed to Broker" & gMQTTBroker & ex.Message & ":" & ex.InnerException.Message)
                                End Try
        
                                bSubscribed = False
                                If bFirstPass Then
                                    hsWriteLog(PLUGIN_DEBUG, "MQTT Thread Client Created")
                                End If
        
                                AddHandler oMQTTClient.MqttMsgPublishReceived, AddressOf client_MqttMsgPublishReceived
                                Dim clientId As String = Guid.NewGuid().ToString()
                                If bFirstPass Then
                                    hsWriteLog(PLUGIN_DEBUG, "MQTT Thread Client ID=" & clientId)
                                End If
                                hsWriteLog(PLUGIN_DEBUG, "Calling MQTT Connect")
                                Try
                                    iBrokerResponse = oMQTTClient.Connect(
                                        clientId,
                                        gMQTTBrokerUsername,
                                        gMQTTBrokerPassword,
                                        gDefaultRetain,
                                        gDefaultQOS,
                                        True,
                                        gThisComputer & "/" & PLUGIN_NAME & "/LWT",
                                        "Offline",
                                        True,
                                        60)  'False,, 'Messages.MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE,
                                    If bFirstPass Then
                                        hsWriteLog(PLUGIN_DEBUG, "MQTT Thread Broker " & gMQTTBroker & " Connect Response=" & iBrokerResponse.ToString)
                                    End If
        
                                Catch ex As Exception
                                    hsWritelog(PLUGIN_DEBUG, "StartMQTT Connection attempt1 to Broker. Exception: " & ex.ToString() & " / InnerEx: " & ex.InnerException.ToString())
                                    If gDebugLog Then
                                        hsWritelogEx(PLUGIN_DEBUG, "StartMQTT Connection attempt1 to Broker " & gMQTTBroker, ex.Message & ":" & ex.InnerException.Message)
                                    End If
                                    oMQTTClient = Nothing
                                End Try
        
                                If iBrokerResponse < 0 OrElse iBrokerResponse > 5 Then
                                    hsWritelog(PLUGIN_DEBUG, "StartMQTT Connection attempt2 Failed to Broker. BrokerResponse:" & iBrokerResponse.ToString)
                                End If
        
                                If oMQTTClient IsNot Nothing Then
                                    If iBrokerResponse < 0 OrElse iBrokerResponse > 5 Then
                                    Else
                                        If iMessageCount < 5 Then
                                            iMessageCount = iMessageCount + 1
                                            hsWritelog(PLUGIN_DEBUG, "MQTT Broker Connection " & sResponse(iBrokerResponse) & ", Connected=" & oMQTTClient.IsConnected.ToString)
                                        End If
                                    End If
        
                                    If oMQTTClient.IsConnected Then
                                        hsWritelog(PLUGIN_NAME, "MQTTClient is Connected")
                                        If iBrokerResponse <> 0 Then
                                            oMQTTClient = Nothing
                                        Else
                                            gOnlineTime = Now
                                            If gCreateStatusDevices Then
                                                hs.SetDeviceValueByRef(gStatusRef(Statistics.BrokerStatus), 1.0, True)
                                            End If
                                            Dim sTopic As String = gThisComputer & "/" & PLUGIN_NAME & "/LWT"
                                            Dim sPayload As String = ONLINE
                                            oMQTTClient.Publish(sTopic, Encoding.UTF8.GetBytes(sPayload), gDefaultQOS, gDefaultRetain)
                                            StatPublish(sTopic, sPayload, False)
                                            If gDebugLog Then
                                                hsWritelog(PLUGIN_DEBUG, "MQTT Thread Subscribing ")
                                            End If
                                            bSubscribed = MQTTSubscribe()
                                            RaiseConnectionEvent(True)
                                            bConnected = True
                                        End If
                                    End If
                                Else
                                    oMQTTClient = Nothing
                                End If 'just connected
                            End If 'previously connected
                        Else ' Not gDisconnect AndAlso gMQTTBroker <> "" 
                        End If 'disconnect
                        bFirstPass = False
                        'use this thread as timer to manage periodic activities
        
                        If gLastDay <> Now.Day Then
                            gReceiveMessageCountToday = 0
                            gPublishCountToday = 0
                            gReceiveAcceptedCountToday = 0
                            If gCreateStatusDevices Then
                                hs.SetDeviceValueByRef(gStatusRef(Statistics.PublishCountToday), 0.0, True)
                                hs.SetDeviceValueByRef(gStatusRef(Statistics.ReceiveAcceptedCountToday), 0.0, True)
                                hs.SetDeviceValueByRef(gStatusRef(Statistics.ReceiveNotAcceptedCountToday), 0.0, True)
                            End If
                            PruneHistory(DateAdd(DateInterval.Day, -gDaysOfHistory, Now))
                            PruneRef(DateAdd(DateInterval.Day, -gDaysOfHistory, Now))
                            For Each sSource As String In AccumResetList
                                If MQTTReceiveDictionary.ContainsKey(sSource) Then
                                    Dim oMQTT As MqttReport = MQTTReceiveDictionary(sSource)
                                    Dim oTransform As Transform = oMQTT.Transform
                                    Dim iRef As Integer = oTransform.AccumDevice
                                    If oTransform.AccumDevice = AccumTypes.ResetDelta Then
                                        Dim dv As Scheduler.Classes.DeviceClass = hs.GetDeviceByRef(iRef)
                                        If dv IsNot Nothing Then
                                            Dim iValue As Double = dv.devValue(hs)
                                            oTransform.AccumMidnight = iValue
                                            SaveTransform(sSource, oTransform)
                                            oMQTT.Transform = oTransform
                                        End If
                                    End If
                                    hs.SetDeviceValueByRef(iRef, 0.0, True)
                                End If
                            Next
                        End If
                        gLastDay = Now.Day
                        hs.SaveEventsDevices()
        
                        Threading.Thread.Sleep(10000)
        
                        If oMQTTClient IsNot Nothing AndAlso oMQTTClient.IsConnected AndAlso Not bSubscribed Then
                            bSubscribed = MQTTSubscribe()
                        End If
        
                        If gTotalMessagesReceived > 0 Then
                            Dim iElapsedTime As Long = DateDiff(DateInterval.Second, dStartTime, Now)
                            gAverageReceiveInterval = (iElapsedTime * 1000) / gTotalMessagesReceived
                        Else
                            gAverageReceiveInterval = 0
                        End If
                        If bConnected AndAlso (oMQTTClient Is Nothing OrElse Not oMQTTClient.IsConnected) Then
                            If gCreateStatusDevices Then
                                hs.SetDeviceValueByRef(gStatusRef(Statistics.BrokerStatus), 0.0, True)
                                hs.SetDeviceValueByRef(gStatusRef(Statistics.AverageReceiveTime), CType(gAverageReceiveMilliseconds, Double), True)
                                hs.SetDeviceValueByRef(gStatusRef(Statistics.AverageReceiveInterval), CType(gAverageReceiveInterval, Double), True)
                            End If
                            RaiseConnectionEvent(False)
                            bConnected = False
                        End If
        
                        Try
                            Dim oList As New List(Of String)
                            For Each kvp As KeyValuePair(Of String, MQTTASP) In WebPageDictionary
                                If kvp.Value IsNot Nothing Then
                                    If DateDiff(DateInterval.Minute, kvp.Value.PostbackDate, Now) > 5 Then
                                        oList.Add(kvp.Key)
                                    End If
                                End If
                            Next
                            For Each sKey As String In oList
                                If WebPageDictionary.ContainsKey(sKey) Then
                                    WebPageDictionary.Remove(sKey)
                                    If gDebugLog Then
                                        hsWritelog(PLUGIN_DEBUG, "Removed expired web page")
                                    End If
                                End If
                            Next
        
                        Catch ex As Exception
                            hsWritelogEx(PLUGIN_DEBUG, "Trying to remove expired web page " & WebPageDictionary.Count.ToString, ex.Message)
                        End Try
                    Loop
        
                Catch ex As Exception
                    hsWritelogEx(PLUGIN_NAME, "StartMQTTThread", ex.Message)
                End Try
            End Sub

        Comment


        • #5
          Hey Michael,

          Ive been using a number of cloud services just recently because my ISP uses a double NAT scenario which means I couldnt use anything outside the first layer of NAT - so I had to use cloudmqtt (using SSL).

          BUT - in the last 4 hours - Ive just managed to get a staticIP for my from my ISP and sparked up a RPI and will be testing it using that. Lets see how we go. So far so good.

          The heartbeat - hehehehe - something super simple - 30s timer in HS - publish a string to a private topic - tasker then compares thatt message timestamp to the previous timestamp - if it exceeds 45 seconds - houston we have a problem.

          By doing it that way i can check the whole chain - including if HS hangs. Stupid Simple, didnt have to think about any code etc - simple and it works.

          Comment

          Working...
          X