Announcement

Collapse
No announcement yet.

Grouped devices without using JSON?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Grouped devices without using JSON?

    I know it's a long shot... The reason I don't use JSON is I feed all my MQTT to influxdb via Telegraf, then I can easily graph thru Grafana sensor data. Telegraf at least from what I have seen is really flexible for parsing incoming data, but can't parse JSON MQTT messages (which is funny as I believe it has no problem doing them independently.) I was using Sonoff-Tasmota, but since I couldn't shut off the JSON strings, they looked pretty in HS thru mcsMQTT, but couldn't easily get data into my influxdb.

    I've recently been experimenting with espurna to avoid this problem as I can disable JSON and dump my sensor payload as individual devices. However as a result, the devices are no longer nicely grouped. Is it possible to group non-JSON devices, or they too independent for the plugin to do that?

    My Plan B is to custom compile espurna. It has the option to natively support influxdb as a destination, so I could have MQTT as JSON and sensor data injection direct, but was hoping to avoid this as easier to download a released firmware bin to upload to sensors when there's an update vs a custom compile and uploads.
    Thanks!

    ALSO WHY did I wait so long for trying Sonoff-Tasmoto and epurna... I rolled my own custom firmware for ESP devices, this looks so much easier...

  • #2
    The grouping is done by parent/child associations properties for the device. This means any set of devices can be grouped. What I cannot visualize is a UI that allows a user to specify a parent and which devices should be associated to that parent. Perhaps adding a Parent Ref text box on the Edit tab so the user can specify a parent.

    Another concept is to provide an influxdb driver in mcsMQTT. I have not used influxdb, but doesn't it just accept SQL expressions? What does Telegraf do?

    Comment


    • #3
      Wow, thanks for the quick reply! Telegraf is kind of a data collector, supports a bunch of different formats:
      https://www.influxdata.com/time-seri...form/telegraf/
      You can see a sampling of the official plugin list for it, it has quite a few input/outputs for data:
      https://github.com/influxdata/telegr...master/plugins


      Here's some info on InfluxDB, the idea being a database "designed with IoT in mind". It's really light weight, and runs well on an Odroid HC1 with a SATA had drive added:
      https://www.influxdata.com/time-seri...form/influxdb/

      I believe it does support direct SQL like connections:
      https://docs.influxdata.com/influxdb...uery_language/

      Those charts in the links are a bit overkill and over complex. In a couple hours I had a MQTT->Telegraf->InfluxDB->Grafana box running, even with SSL certs following some online tutorials. Really the longest part was getting the certs all in place and in formats it expected (not required, but why not go secure?) The Odroid HC1 is like an inexpensive version of their 8core XU4 series designed for storage and such. It works great for this applications, and sits at <30% CPU, usually just 1-5% most of the time, and this is with devices sending status quite frequently. Really the only bottleneck I see that it can take a minute to chart results going back over a period of several months. I attached a chart with power measures that has been going for months at 1s resolution, and my 100gb drive is barely 1-2% usage (which also includes months of other sensor data.)



      Originally I was trying to find a way for the plugin to send out the data as raw MQTT after it received it, but didn't see a way to do that as obviously for most uses cases makes really zero sense.

      Comment


      • #4
        Originally posted by Michael McSharry View Post
        The grouping is done by parent/child associations properties for the device. This means any set of devices can be grouped. What I cannot visualize is a UI that allows a user to specify a parent and which devices should be associated to that parent. Perhaps adding a Parent Ref text box on the Edit tab so the user can specify a parent.
        Michael, that would be good to allow us to set the parent/child relationship to do the groupings.....when experimenting it currently gets messy when you extract additional data out of a JSON string for a device (tasmota for example) and that device is no where near the original devices....

        Pete

        HS 2.2.0.11

        Comment


        • #5
          I should be able to insert data directly into influxdb. Before I start I want to make certain the UI makes sense. For influxdb it should be General tab to specify influx IP, port and database name. Selection of items to store would be the Association tab history and device checkboxes. The data being stored would be same as in the HS DeviceValue. Is there need for database login credentials and is there other forms of security used?

          mcsMQTT now has provision to create both parent and child devices and this will keep JSON items together. What are you looking to do different? Is it that you do not want the grouping in every case?

          Comment


          • #6
            Originally posted by Michael McSharry View Post
            mcsMQTT now has provision to create both parent and child devices and this will keep JSON items together.
            It already does this ? I must go add a few more devices....

            Thankyou..Pete
            HS 2.2.0.11

            Comment


            • #7
              As I was thinking about it last night, adding InfluxDB may or may not be ideal either as unless I can filter what goes into the database, going to end up duplicated (one from standard non-JSON MQTT messages I'm already receiving that InfluxDB is getting, as well as the same devices into mcsMQTT.) i.e. \raw\foobar\1 will end up being added by telefraf->influxdb and mcsMQTT if I am accepting it's messages into HS. If the ability to do " adding a Parent Ref text box on the Edit tab so the user can specify a parent." may be ideal and hopefully less overhead? Also I can see people potentially wanting to do parent/child device relationships more likely than I *may* be the only person using influxDB?

              Comment


              • #8
                For existing parent/child groupings look at General Tab, Inbound, JSON decoding.

                For database filtering the Association tab now has checkboxes on a device by device basis to allow user select what goes into the History database.

                Comment


                • #9
                  3.4.10.0 has added an Edit Tab row for user-specified parent group device. Devices with the same parent will be grouped on the HS Device Management page. To create a new parent use a negative number entry (as indicated in the tool tip). A blank entry will remove the device from a group.

                  This provides a second means to group devices. The other is based upon JSON content of the payload. While normally groupings are associated with physical relationships, they can also be done with virtual relationships. For example, all Uptime reports can be grouped together or all Temperature sensors could be grouped.

                  It has been submitted to updater and available at http://mcsSprinklers.com/mcsMQTT_3_4_10_0.zip

                  Comment


                  • #10
                    Thank you very much for doing this! Looking forward to trying this out.

                    Comment


                    • #11
                      This is working great so far, thank you very much for adding this!!!

                      Comment

                      Working...
                      X