Announcement

Collapse
No announcement yet.

SSL Support for mcsMQTT

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    #31
    ClientCert set to Nothing
    Last edited by Michael McSharry; April 19, 2018, 03:30 PM.

    Comment


      #32
      Originally posted by Pete View Post
      Trying from scratch here.
      Is the plugin on a Windows platform or Linux? Windows "should" reject self-signed CA certs unless you stick them in the Trusted Root or somesuch.

      Also, what CN are you using and how are you resolving it so it matches the cert?

      Thanks,
      Z

      Comment


        #33
        Originally posted by Michael McSharry View Post
        The two edits were done in the attached. Still need some SSL selected on mcsMQTT setup but the 1.2 will be used.
        Code:
                                If gMQTTBrokerSSL <> uPLibrary.Networking.M2Mqtt.MqttSslProtocols.None AndAlso gMQTTBrokerCaCert <> "" Then 'AndAlso gMQTTBrokerClientCert <> "" Then
                                    Dim sCert As String = "CaCert"
                                    Try
                                        caCert = System.Security.Cryptography.X509Certificates.X509Certificate.CreateFromCertFile(gMQTTBrokerCaCert)
                                        sCert = "Client Cert"
                                        clientCert = System.Security.Cryptography.X509Certificates.X509Certificate.CreateFromCertFile(gMQTTBrokerClientCert)
                                        sslProtocol = uPLibrary.Networking.M2Mqtt.MqttSslProtocols.TLSv1_2 'gMQTTBrokerSSL
                                        bSecure = True
                                    Catch ex As Exception
                                        hsWritelog(PLUGIN_DEBUG, sCert & " issue " & ex.Message)
                                    End Try
                                End If
        I see these edits, but was this ever fixed?

        I have my broker using TLS1.2 with client certs successfully being utilized on other devices, but mcsMQTT v3.3.6.0 running on a Raspberry pi 3 (looking to upgrade this soon) using the same method, set to TLS 1.2 fails over and over with: (mosquitto log)

        Code:
        May 15 17:12:47 raspberrypi mosquitto[431]: New connection from [HS3-IP] on port 1883.
        May 15 17:12:47 raspberrypi mosquitto[431]: 1526425967: OpenSSL Error: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
        May 15 17:12:47 raspberrypi mosquitto[431]: OpenSSL Error: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
        May 15 17:12:47 raspberrypi mosquitto[431]: Socket error on client <unknown>, disconnecting.
        Other clients connect fine:
        Code:
        New client connected from 192.168.1. as mqtt_test (c0, k3600, u'MQTT_TEST').
        It worked without any security, however once I put the certs in place, HS3/mcsMQTT refuses to connect.

        Changing the TLS setting for "MQTT Broker Security" doesn't change the error at all.

        Thanks

        Comment


          #34
          It is my understanding that Pete and vasrc have SSL functional with mcsMQTT. vasrc indicated that he was unable to get a self-signed certificate to work, however. I have no background in this area so am leaning on the users to help.

          Comment


            #35
            Originally posted by Jeeves View Post
            I see these edits, but was this ever fixed?

            I have my broker using TLS1.2 with client certs successfully being utilized on other devices, but mcsMQTT v3.3.6.0 running on a Raspberry pi 3 (looking to upgrade this soon) using the same method, set to TLS 1.2 fails over and over with: (mosquitto log)

            Code:
            May 15 17:12:47 raspberrypi mosquitto[431]: New connection from [HS3-IP] on port 1883.
            May 15 17:12:47 raspberrypi mosquitto[431]: 1526425967: OpenSSL Error: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
            May 15 17:12:47 raspberrypi mosquitto[431]: OpenSSL Error: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
            May 15 17:12:47 raspberrypi mosquitto[431]: Socket error on client <unknown>, disconnecting.
            Other clients connect fine:
            Code:
            New client connected from 192.168.1. as mqtt_test (c0, k3600, u'MQTT_TEST').
            It worked without any security, however once I put the certs in place, HS3/mcsMQTT refuses to connect.

            Changing the TLS setting for "MQTT Broker Security" doesn't change the error at all.

            Thanks
            Looks like you're using port 1883? For TLS1_X you need to use port 8883
            I'm guessing you setup your broker for 1883?
            You can dual port the mosquitto broker so it responds to both, but you'll need a CA and server certificate.

            To see if the broker responds correctly using port 8883:
            openssl s_client -connect IP of broker:8883

            Z

            Comment


              #36
              Originally posted by vasrc View Post
              Looks like you're using port 1883? For TLS1_X you need to use port 8883
              I'm guessing you setup your broker for 1883?
              You can dual port the mosquitto broker so it responds to both, but you'll need a CA and server certificate.

              To see if the broker responds correctly using port 8883:
              openssl s_client -connect IP of broker:8883

              Z
              I'm unfamiliar with the need to dual-port mosquitto.

              It was working and responding to two other clients fine on port 1883 (which I had configured it to use).

              Mosquitto is setup with both the ca and server certificate using TLS1.2 and the clients are using client certificates.
              Only mcsMQTT is failing with the error related to SSL3. In my previous post I provided logs and tried to indicate that the HS3 IP was the only one having issues while the other clients connected without problem.

              Comment


                #37
                Here did it all via Node Red which is very intuitive.

                Node Red test broker was configured at port 8883 where as my other Red Node test broker was configured at port 1883.

                Thinking Node Red defaulted to port 8883?

                I have not paid attention lately working on my Node Red counter thing.

                [ATTACH]68864[/ATTACH]

                [ATTACH]68865[/ATTACH]

                [ATTACH]68866[/ATTACH]
                Last edited by Pete; May 17, 2018, 05:14 AM.
                - Pete

                Auto mator
                Homeseer 3 Pro - 3.0.0.548 (Linux) - Ubuntu 18.04/W7e 64 bit Intel Haswell CPU 16Gb
                Homeseer Zee2 (Lite) - 3.0.0.548 (Linux) - Ubuntu 18.04/W7e - CherryTrail x5-Z8350 BeeLink 4Gb BT3 Pro
                HS4 Lite - Ubuntu 22.04 / Lenovo Tiny M900 / 32Gb Ram

                HS4 Pro - V4.1.18.1 - Ubuntu 22.04 / Lenova Tiny M900 / 32Gb Ram
                HSTouch on Intel tabletop tablets (Jogglers) - Asus AIO - Windows 11

                X10, UPB, Zigbee, ZWave and Wifi MQTT automation-Tasmota-Espurna. OmniPro 2, Russound zoned audio, Alexa, Cheaper RFID, W800 and Home Assistant

                Comment


                  #38
                  Originally posted by Jeeves View Post
                  I'm unfamiliar with the need to dual-port mosquitto.

                  It was working and responding to two other clients fine on port 1883 (which I had configured it to use).

                  Mosquitto is setup with both the ca and server certificate using TLS1.2 and the clients are using client certificates.
                  Only mcsMQTT is failing with the error related to SSL3. In my previous post I provided logs and tried to indicate that the HS3 IP was the only one having issues while the other clients connected without problem.
                  It's quite simple in mosquitto. In the config file look for the section labeled Extra Listeners. This is where you define another port to listen on. Each listener has it's own SSL settings so make sure you're setting the cafile, certfile and certkey on the correct listener. The certs are for the port 8883 listener. Restart the broker and you should now be able to accept traffic from both non-secure (1883) and secure (8883) devices.

                  I haven't been able to get the plugin on windows to accept a self-signed cert as it needs some internal programming to catch those exceptions. It works well on both my ESP8266 and ESP32 devices though. Haven't tried it on a PI yet, have to try that still.

                  Additionally, since this is SSL the CN of the cert (you enter this when you make it) will need to match the DNS name of your broker so that assumes you either have a DNS in your system to resolve IP's to names or your host file defines it.

                  Z

                  Comment


                    #39
                    Yeah here using Node Red on RPi2's (two of them).

                    Wondering now if I am the only tester here using Node Red on the RPi's?
                    - Pete

                    Auto mator
                    Homeseer 3 Pro - 3.0.0.548 (Linux) - Ubuntu 18.04/W7e 64 bit Intel Haswell CPU 16Gb
                    Homeseer Zee2 (Lite) - 3.0.0.548 (Linux) - Ubuntu 18.04/W7e - CherryTrail x5-Z8350 BeeLink 4Gb BT3 Pro
                    HS4 Lite - Ubuntu 22.04 / Lenovo Tiny M900 / 32Gb Ram

                    HS4 Pro - V4.1.18.1 - Ubuntu 22.04 / Lenova Tiny M900 / 32Gb Ram
                    HSTouch on Intel tabletop tablets (Jogglers) - Asus AIO - Windows 11

                    X10, UPB, Zigbee, ZWave and Wifi MQTT automation-Tasmota-Espurna. OmniPro 2, Russound zoned audio, Alexa, Cheaper RFID, W800 and Home Assistant

                    Comment


                      #40
                      Unrelated to SSL here while I can enable two listeners with two ports on mcsMQTT.

                      Testing it the other day changed the port number of one of them and kept the same IP and mcsMQTT did go off line. When I changed the port to the same one as the source messages then mcsMQTT went back on line.
                      - Pete

                      Auto mator
                      Homeseer 3 Pro - 3.0.0.548 (Linux) - Ubuntu 18.04/W7e 64 bit Intel Haswell CPU 16Gb
                      Homeseer Zee2 (Lite) - 3.0.0.548 (Linux) - Ubuntu 18.04/W7e - CherryTrail x5-Z8350 BeeLink 4Gb BT3 Pro
                      HS4 Lite - Ubuntu 22.04 / Lenovo Tiny M900 / 32Gb Ram

                      HS4 Pro - V4.1.18.1 - Ubuntu 22.04 / Lenova Tiny M900 / 32Gb Ram
                      HSTouch on Intel tabletop tablets (Jogglers) - Asus AIO - Windows 11

                      X10, UPB, Zigbee, ZWave and Wifi MQTT automation-Tasmota-Espurna. OmniPro 2, Russound zoned audio, Alexa, Cheaper RFID, W800 and Home Assistant

                      Comment


                        #41
                        Originally posted by vasrc View Post
                        It's quite simple in mosquitto. In the config file look for the section labeled Extra Listeners. This is where you define another port to listen on. Each listener has it's own SSL settings so make sure you're setting the cafile, certfile and certkey on the correct listener. The certs are for the port 8883 listener. Restart the broker and you should now be able to accept traffic from both non-secure (1883) and secure (8883) devices.

                        I haven't been able to get the plugin on windows to accept a self-signed cert as it needs some internal programming to catch those exceptions. It works well on both my ESP8266 and ESP32 devices though. Haven't tried it on a PI yet, have to try that still.

                        Additionally, since this is SSL the CN of the cert (you enter this when you make it) will need to match the DNS name of your broker so that assumes you either have a DNS in your system to resolve IP's to names or your host file defines it.

                        Z
                        Having now done my share of reading, and testing. I can safely say some of that is incorrect (I am not on windows, so some does not apply).

                        MQTT by protocol has port 1833 (both TCP & UDP) and 8883 (secure-mqtt, both TCP & UDP) reserved with IANA (Internet Assigned Numbers Authority). So, by having my secure-MQTT operating on 1833, I am 'against' protocol. However, it works just fine. Like many other services on the web, I can utilize whichever port I so choose.

                        With my mosquitto.conf configured, I have a single listener, and I can change it to "listener 8683". Then I restart mosquitto and change my client devices and I see:

                        Code:
                        1526610607: mosquitto version 1.4.15 (build date Sat, 07 Apr 2018 11:13:41 +0100) starting
                        1526610607: Config loaded from /etc/mosquitto/mosquitto.conf.
                        1526610607: Opening ipv4 listen socket on port 8683.
                        1526610607: Opening ipv6 listen socket on port 8683.
                        1526610660: New connection from [OwntracksDevice] on port 8683.
                        My phone, by way of Owntracks is using both the CA cert and a P12 client key communicates to the broker and won't communicate if I remove them. The broker is using TLS1.2 over port 8683 (Previously 1883). We can safely ignore the port issue.

                        The only thing I'm wondering about is DNS, since everything comes back to DNS and since HS3 is running on the MQTT Broker itself.

                        Otherwise, this comes back to mcsMQTT.

                        Thanks for the pointers however, I'll dig into DNS, it's always fun.

                        Comment


                          #42
                          Originally posted by Jeeves View Post
                          Having now done my share of reading, and testing. I can safely say some of that is incorrect (I am not on windows, so some does not apply).

                          MQTT by protocol has port 1833 (both TCP & UDP) and 8883 (secure-mqtt, both TCP & UDP) reserved with IANA (Internet Assigned Numbers Authority). So, by having my secure-MQTT operating on 1833, I am 'against' protocol. However, it works just fine. Like many other services on the web, I can utilize whichever port I so choose.

                          With my mosquitto.conf configured, I have a single listener, and I can change it to "listener 8683". Then I restart mosquitto and change my client devices and I see:

                          Code:
                          1526610607: mosquitto version 1.4.15 (build date Sat, 07 Apr 2018 11:13:41 +0100) starting
                          1526610607: Config loaded from /etc/mosquitto/mosquitto.conf.
                          1526610607: Opening ipv4 listen socket on port 8683.
                          1526610607: Opening ipv6 listen socket on port 8683.
                          1526610660: New connection from [OwntracksDevice] on port 8683.
                          My phone, by way of Owntracks is using both the CA cert and a P12 client key communicates to the broker and won't communicate if I remove them. The broker is using TLS1.2 over port 8683 (Previously 1883). We can safely ignore the port issue.

                          The only thing I'm wondering about is DNS, since everything comes back to DNS and since HS3 is running on the MQTT Broker itself.

                          Otherwise, this comes back to mcsMQTT.

                          Thanks for the pointers however, I'll dig into DNS, it's always fun.
                          You are absolutely correct, the port is SSL agnostic, you can select any port number you want to run on. I run two ports so I can support both secure and non-secure devices.

                          I'd guess that since the PI is running Mono, it's still pretty much a windows device and may fail for the same reason a plugin running on Windows would. When I add a ValidationCallback to the SSL verify in the test plugin I use, I can get it to accept a self-signed cert, but I'm not 100% convinced it's not a DNS/CN issue still?

                          LMK if you get it working.

                          Z

                          Comment


                            #43
                            Originally posted by vasrc View Post
                            You are absolutely correct, the port is SSL agnostic, you can select any port number you want to run on. I run two ports so I can support both secure and non-secure devices.

                            I'd guess that since the PI is running Mono, it's still pretty much a windows device and may fail for the same reason a plugin running on Windows would. When I add a ValidationCallback to the SSL verify in the test plugin I use, I can get it to accept a self-signed cert, but I'm not 100% convinced it's not a DNS/CN issue still?

                            LMK if you get it working.

                            Z
                            Mono adds it's own layers of frustration at time, but in essence does make most of the .NET application work much like windows.

                            I was already mid-migration, so I decided to finish things up. I moved off the pi, and onto an Odroid with Ubuntu and migrated the HS3 install across.

                            With that. I'm on a new host, I've configured, tested and generated new certs. My phone connects to the broker just like it used to, again using p12 certs.

                            I can't get mcsMQTT to accept a self-signed cert to save my life. Now the documentation is sparse, so there's no explanation of what formats anything should be in. But still, I've tried p12, the certs themselves as they come out of openssl, etc. Worse yet, with debug loggin there's nothing even written in the mcsMQTT Debug log. I'm pretty much at an impasse it seems. Unless I ignore TLS (ugh) and go simple auth, it's unusable.


                            Worse yet, while mcsMQTT is doing *NOTHING*, it seems to be burning the CPU as hard as it can. I had already uninstalled and reinstalled it, hence the mis-matched start time. If I leave it running, it crashes and takes hs3 with it. This might be a bug report on it's own?

                            From ps aux:
                            Code:
                             hs3      1328  6.9  5.8 233528 118860 ?       Ssl  18:47   2:05 /usr/bin/mono /usr/local/homeseer/HSConsole.ex
                            hs3      1368  0.4  1.7  84680 36140 ?        Sl   18:47   0:08 /usr/bin/mono /usr/local/homeseer/HSPI_BLOccup
                            hs3      1369  0.4  1.7  86800 35444 ?        Sl   18:47   0:07 /usr/bin/mono /usr/local/homeseer/HSPI_BLEmail
                            hs3      1371  0.6  2.9 115848 60840 ?        Sl   18:47   0:12 /usr/bin/mono /usr/local/homeseer/HSPI_ImperiH
                            hs3      1373  0.3  1.9  90184 39364 ?        Sl   18:47   0:05 /usr/bin/mono /usr/local/homeseer/HSPI_SCREPOS
                            hs3      1375  0.5  3.3 126000 68840 ?        Sl   18:47   0:10 /usr/bin/mono /usr/local/homeseer/HSPI_BULLET.
                            hs3      1377  0.6  2.9 121900 59924 ?        Sl   18:47   0:11 /usr/bin/mono /usr/local/homeseer/HSPI_ULTRANE
                            hs3      2294  [B][COLOR="Red"]101[/COLOR][/B]  3.0 117924 62904 ?        Sl   19:09   8:32 /usr/bin/mono /usr/local/homeseer/HSPI_MCSMQTT

                            Comment


                              #44
                              Relating to using SSL with mcsMQTT it is not soup yet and ready to utilize.

                              Going to try to set up a Mosquitto broker here on one of my test micro routers running OpenWRT using Python and try the encryption stuff again. (currently using Node Red on two RPi's)

                              Michael has asked for our help getting the plugin to work with SSL.


                              Curious what activity you see in your debug logs when this (HS3 Debend) is happening?

                              A few days back on an earlier revision of the plugin some sql chit chat took down Homeseer Pro here.

                              Homeseer Pro is running in Ubuntu 16.04 64 bit with an iSeries CPU / 16Gb of RAM.

                              I posted about this issue here:

                              Version 3.3.x.x

                              Also testing plugin on HS3 lite on a Pine64 2Gb / Ubuntu 16.04 64 bit server and it does well.
                              - Pete

                              Auto mator
                              Homeseer 3 Pro - 3.0.0.548 (Linux) - Ubuntu 18.04/W7e 64 bit Intel Haswell CPU 16Gb
                              Homeseer Zee2 (Lite) - 3.0.0.548 (Linux) - Ubuntu 18.04/W7e - CherryTrail x5-Z8350 BeeLink 4Gb BT3 Pro
                              HS4 Lite - Ubuntu 22.04 / Lenovo Tiny M900 / 32Gb Ram

                              HS4 Pro - V4.1.18.1 - Ubuntu 22.04 / Lenova Tiny M900 / 32Gb Ram
                              HSTouch on Intel tabletop tablets (Jogglers) - Asus AIO - Windows 11

                              X10, UPB, Zigbee, ZWave and Wifi MQTT automation-Tasmota-Espurna. OmniPro 2, Russound zoned audio, Alexa, Cheaper RFID, W800 and Home Assistant

                              Comment


                                #45
                                Originally posted by Pete View Post
                                Relating to using SSL with mcsMQTT it is not soup yet and ready to utilize.
                                That is unfortunate. Hopefully some progress will be made, I'd love to get TLS up and running. Is there a more direct way to provide feedback? I had hoped/expected he would read/chime into threads in his sub-forum.

                                Originally posted by Pete View Post
                                Curious what activity you see in your debug logs when this (HS3 Debend) is happening?
                                Do you mean the high cpu usage?

                                Since mcsMQTT is not in communication with the broker, it should not be doing anything. It has nothing to communicate to.

                                When I enable the plugin, this is what it does:

                                Startup:
                                Code:
                                5/18/2018 8:14:16 PM    3       | mcsMQTT Version 3.3.8.0 running at /usr/local/homeseer, HS is at /usr/local/$
                                5/18/2018 8:14:16 PM    20      | mcsMQTT InitHW ComputerName= odie, IOEnabled=False
                                5/18/2018 8:14:17 PM    309     | mcsMQTT Debug InitHW Database Ready
                                5/18/2018 8:14:17 PM    315     | mcsMQTT Debug Receive Ready
                                5/18/2018 8:14:17 PM    319     | mcsMQTT Debug Trigger Ready
                                5/18/2018 8:14:17 PM    455     | Spawning MQTT Threads
                                5/18/2018 8:14:17 PM    455     | mcsMQTT Debug MQTT Ready
                                5/18/2018 8:14:17 PM    455     | HW Init Complete
                                5/18/2018 8:14:17 PM    457     | Background Init Started
                                5/18/2018 8:14:17 PM    466     | PopulateReceiveDict , PluginDevice=False
                                5/18/2018 8:14:17 PM    466     | PopulateReceiveDict , PluginDevice=False
                                5/18/2018 8:14:17 PM    467     | PopulateReceiveDict , PluginDevice=False
                                5/18/2018 8:14:17 PM    467     | PopulateReceiveDict , PluginDevice=False
                                5/18/2018 8:14:17 PM    467     | PopulateReceiveDict , PluginDevice=False
                                5/18/2018 8:14:17 PM    467     | PopulateReceiveDict , PluginDevice=False
                                5/18/2018 8:14:17 PM    468     | PopulateReceiveDict , PluginDevice=False
                                5/18/2018 8:14:17 PM    468     | PopulateReceiveDict , PluginDevice=False
                                5/18/2018 8:14:17 PM    468     | MQTT Thread Started with broker 192.168.1.35, Shutdown=False, Disconnect=Fal$
                                5/18/2018 8:14:17 PM    468     | PopulateReceiveDict , PluginDevice=False
                                5/18/2018 8:14:17 PM    468     | MQTT Thread Not Connected Yet
                                5/18/2018 8:14:17 PM    468     | PopulateReceiveDict , PluginDevice=False
                                5/18/2018 8:14:17 PM    468     | PopulateReceiveDict , PluginDevice=False
                                5/18/2018 8:14:17 PM    469     | PopulateReceiveDict , PluginDevice=False
                                5/18/2018 8:14:17 PM    469     | PopulateReceiveDict , PluginDevice=False
                                ... there's plenty more here, I'm trimming for space and this doesn't really belong in this thread anyway.

                                End of Log
                                Code:
                                5/18/2018 8:14:22 PM    5998    | AddToMQTTSend1 94, inDict = False
                                5/18/2018 8:14:22 PM    6000    | PopulateSendDict 94, PluginDevice=False
                                5/18/2018 8:14:22 PM    6015    | AddToMQTTSend1 95, inDict = False
                                5/18/2018 8:14:22 PM    6017    | PopulateSendDict 95, PluginDevice=False
                                5/18/2018 8:14:23 PM    6035    | AddToMQTTSend1 96, inDict = False
                                5/18/2018 8:14:23 PM    6037    | PopulateSendDict 96, PluginDevice=False
                                5/18/2018 8:14:23 PM    6054    | AddToMQTTSend1 97, inDict = False
                                5/18/2018 8:14:23 PM    6056    | PopulateSendDict 97, PluginDevice=False
                                5/18/2018 8:14:23 PM    6073    | AddToMQTTSend1 98, inDict = False
                                5/18/2018 8:14:23 PM    6074    | PopulateSendDict 98, PluginDevice=False
                                5/18/2018 8:14:23 PM    6090    | AddToMQTTSend1 99, inDict = False
                                5/18/2018 8:14:23 PM    6091    | PopulateSendDict 99, PluginDevice=False
                                5/18/2018 8:14:23 PM    6091    | Background Send
                                5/18/2018 8:14:23 PM    6107    | Background Init Filters - Background Complete
                                5/18/2018 8:15:23 PM    66978   | Page Timing Start
                                5/18/2018 8:15:25 PM    68765   | Page Timing End
                                Past this point, mcsMQTT still is running and the CPU load is extreme. Nothing further gets written to the log.

                                It doesn't matter if "Disconnect from MQTT Broker" is checked. CPU load is the same and the same messages are written to the log.
                                Last edited by Jeeves; May 18, 2018, 09:55 PM.

                                Comment

                                Working...
                                X