Announcement

Collapse
No announcement yet.

HomeSeer MQTT plugin.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Nice little bit of digging which shows there is some work done to address the enterprise-like reliability concerns. Note that HAProxy is still a single point, but if properly deployed it should be a useful configuration. The general concept of load balancing with the potential that multiple backend servers could be offline is an elegant solution. emqtt appears to use router to provide the redundancy switching. Routers are typcially disk/ssd for storage so have normal longevity.

    What I am trying to point out is not so much that there is no solution, but that the user community that uses Pi and similar SBC are not aware of the SD limitations and the mosquitto documentation I have seen has not discussed the implications, if any, of the host and memory usage. When the user asks that messages persist then it mosquitto continually pounding on the SD or is it using RAM and is not concerned with persistence between restarts. Was it designed for a SBC or was it desired for a Server? Does QOS selected by the client affect SD utilization? Do we expect months, one year, five years, etc before the SD life has expired. It is not only mosquitto, but other applications that use the SD as a general storage device with high duty cycles.

    When I started with my mcsSprinklers application on Linux I used the SD and found that it had a short lifetime with a periodic database write every 60 seconds. I also found that the SD data was easily trashed if I made no provisions to deal with power outages. SD are likely better today, but same considerations apply.

    Comment


      Originally posted by Michael McSharry View Post
      Nice little bit of digging which shows there is some work done to address the enterprise-like reliability concerns. Note that HAProxy is still a single point, but if properly deployed it should be a useful configuration. The general concept of load balancing with the potential that multiple backend servers could be offline is an elegant solution. emqtt appears to use router to provide the redundancy switching. Routers are typcially disk/ssd for storage so have normal longevity.

      What I am trying to point out is not so much that there is no solution, but that the user community that uses Pi and similar SBC are not aware of the SD limitations and the mosquitto documentation I have seen has not discussed the implications, if any, of the host and memory usage. When the user asks that messages persist then it mosquitto continually pounding on the SD or is it using RAM and is not concerned with persistence between restarts. Was it designed for a SBC or was it desired for a Server? Does QOS selected by the client affect SD utilization? Do we expect months, one year, five years, etc before the SD life has expired. It is not only mosquitto, but other applications that use the SD as a general storage device with high duty cycles.

      When I started with my mcsSprinklers application on Linux I used the SD and found that it had a short lifetime with a periodic database write every 60 seconds. I also found that the SD data was easily trashed if I made no provisions to deal with power outages. SD are likely better today, but same considerations apply.

      Sweet!
      You couldn't have explained it better than you have done here. When I had the fear of R/W of the SD cards, on my part it was how small I thought the SD cards were but not how technically you have explained.

      I did get me a BITSCOPE blade a while back for trying go green and have all my SBC servers on same board but not entirely to save data or to play it safe for that matter.
      Project still on the shelf (lazy!). But now you have said this, you have given me even more work to do.

      Here is another an topic which I found and think is quite good for everyone : https://www.linux-tips-and-tricks.de/en/backup

      Thanks for highlighting that issue. I found it so informative.


      Eman.
      TinkerLand : Life's Choices,"No One Size Fits All"

      Comment


        Is it possible to use this plugin with cloudMQTT? I tried, but there is no input field for port and cloudMQTT is using different ports. Anyone got this working? I have to push some stuff over the internet, and I would like to use cloudMQTT for it.

        Comment


          mcsMQTT does allow the port to be specified. I am also adding SSL support, but need a user that actually implements SSL on the broker to help with testing.

          Comment


            Originally posted by Michael McSharry View Post
            I am also adding SSL support, but need a user that actually implements SSL on the broker to help with testing.
            Pick me, pick me!

            Comment


              Originally posted by Michael McSharry View Post
              Many are using Pi or similar with SD the means of storage. If mosquito persists power cycles then it is constantly writing to SD to hold retained messages. Has there been any discussion on the lifetime write considerations for MQTT using brokers hosted on SD devices? With the broker being a single point failure it should be robust and not kill itself by destroying its SD.
              I am a long time user of MQTT on a RPI, even before I started using HS. My first mosquito broker was an RPI model B with the full size SD Card. It lasted about 2.5 years before I had a SD card failure. I cannot be certain that it was MQTT that caused the failure as there was also a Postgres database with high levels of read/write on the same box.

              The replacement and current RPI2 B is at 18 months with no issues.

              As I have said on other threads, I have found RPI2s to be a little more tolerant of power supply quality, whereas RPI3s need a good 5V supply.

              HTH

              A

              Comment


                Pick me, pick me!
                I opened up a discussion thread at

                https://forums.homeseer.com/showthre...85#post1360885

                Comment


                  Need help running multiple instances

                  I'm trying to run multiple instances of this plugin to connect to several different mqtt brokers.
                  Is this possible and could someone tell me how to do this?

                  Comment


                    Originally posted by joshk View Post
                    I'm trying to run multiple instances of this plugin to connect to several different mqtt brokers.
                    Is this possible and could someone tell me how to do this?

                    If you can't do it native with the plugin, FWIW you can stand up your own MQTT server internally on a Pi, and sub to multiple MQTT sources. I do this with TheThingsnetwork where I connect to their MQTT end point for my LoRa devices, and that feeds into my Mosquitto server.

                    EDIT:
                    Here's an example of what it looks like in my mosquitto.conf for adding another broker, obviously username/password removed:
                    Code:
                    connection bridge-to-ttn
                    address eu.thethings.network:8883
                    bridge_cafile /etc/mosquitto/certs/ttn-mqtt-ca.pem
                    bridge_insecure false
                    start_type automatic
                    notifications false
                    try_private false
                    remote_username <TTNUSERNAME>
                    remote_password <TTNPASSWORD>
                    topic +/devices/+/up/# in 0 /ttn/
                    Last edited by mloebl; April 26, 2018, 11:01 AM.

                    Comment


                      Originally posted by joshk View Post
                      I'm trying to run multiple instances of this plugin to connect to several different mqtt brokers.
                      Is this possible and could someone tell me how to do this?
                      You might check out the bridging option on MQTT brokers (broker dependent). That might be a more efficient way than creating multiple plugins.

                      Z

                      Comment


                        Originally posted by mloebl View Post
                        If you can't do it native with the plugin, FWIW you can stand up your own MQTT server internally on a Pi, and sub to multiple MQTT sources. I do this with TheThingsnetwork where I connect to their MQTT end point for my LoRa devices, and that feeds into my Mosquitto server.

                        EDIT:
                        Here's an example of what it looks like in my mosquitto.conf for adding another broker, obviously username/password removed:
                        Code:
                        connection bridge-to-ttn
                        address eu.thethings.network:8883
                        bridge_cafile /etc/mosquitto/certs/ttn-mqtt-ca.pem
                        bridge_insecure false
                        start_type automatic
                        notifications false
                        try_private false
                        remote_username <TTNUSERNAME>
                        remote_password <TTNPASSWORD>
                        topic +/devices/+/up/# in 0 /ttn/

                        Thanks! I wasn't aware that you could bridge mosquitto brokers together. this solves my problem!

                        Comment


                          I see that sometimes the plugin updates wrong information from MQTT, I use a "smart meater" connected to an rpi and sending the information over cloudmqtt via your plugin. There is a timestamp from the sample coming from the meater and sometimes the timestamp from the meater and from the plugin is way off. I know that sampling is ok as I can check it in node-red running on rpi, and restarting plugin fixes the problem. So pretty sure it is on plugin side.

                          Comment


                            Originally posted by salvesen View Post
                            I see that sometimes the plugin updates wrong information from MQTT, I use a "smart meater" connected to an rpi and sending the information over cloudmqtt via your plugin. There is a timestamp from the sample coming from the meater and sometimes the timestamp from the meater and from the plugin is way off. I know that sampling is ok as I can check it in node-red running on rpi, and restarting plugin fixes the problem. So pretty sure it is on plugin side.

                            Hi,

                            Forward any faults with the plugin here :
                            https://www.domoticaforum.eu/viewtopic.php?f=74&t=10525


                            The author can be contacted there. I wrote about it.

                            There is also mcsMQTT in the updater



                            Eman.
                            Last edited by Eman; May 4, 2018, 04:30 PM.
                            TinkerLand : Life's Choices,"No One Size Fits All"

                            Comment


                              Thanks for sharing the plugin! I’m new to this topic, and I didn’t know HAProxy is SPOF…
                              If you are also trying to do your research on the topic, then you might appreciate this article that thoroughly explains how to establish an MQTT client and broker connection. It explains the matter in simple words, and even a nonprofessional would be able to understand that. It also contains a step-by-step guide on how to enable the MQTT client and broker connection, what are the CONNECT and CONNACK data packets and what roles a client and a broker play in this process.
                              And if you have any additional questions, a friendly community in Cedalo Forum would be happy to help.

                              Comment

                              Working...
                              X