Announcement

Collapse
No announcement yet.

Scripting assistance - Parse/interpret MQTT data from temperature and humidity sensor

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Michael McSharry
    replied
    This tangent to the original thread has been continued at https://forums.homeseer.com/forum/li...-of-the-device

    Leave a comment:


  • 123qweasd
    replied
    Makes sense, let me know once you have a working beta, I'll be glad to do the testing

    Thanks for your precious assistance,

    Cheers,

    Yann

    Leave a comment:


  • Michael McSharry
    replied
    What I am thinking about doing on the Edit tab is a checkbox to identify the item (e.g home/OpenMQTTGateway_SRFB/PilighttoMQTT:message:id) to be elevated as a topic component for all other members of the json group. Continuing your example further it would become

    home/OpenMQTTGateway_SRFB/PilighttoMQTT:message:id (no change from is currently visible and remains as only a placeholder and cannot be associated with HS device)
    home/OpenMQTTGateway_SRFB/PilighttoMQTT:message:63:temperature
    home/OpenMQTTGateway_SRFB/PilighttoMQTT:message:63:humidity
    :

    home/OpenMQTTGateway_SRFB/PilighttoMQTT:message:75:temperature
    etc

    There may be implications that I don't remember right now, but seems like a reasonable approach. It is not something, however, that can be done in a couple hours.

    Leave a comment:


  • 123qweasd
    replied
    Would it be feasible to add a form of secondary topic "dependency" in the Edit page?

    IE: Create the device as usual (ex: based on topic home/OpenMQTTGateway_SRFB/PilighttoMQTT:message:temperature)

    and an optional "dependency" field in the edit page which would require to identify a secondary topic
    Ex: home/OpenMQTTGateway_SRFB/PilighttoMQTT:message:id + associated "trigger" value (ex: 63)

    Which would mean :

    IF message:id = 63 THEN register temperature value to the associated HS device.
    ELSE, ignore.

    not sure if it's worth having more than one level of logic but I guess it should cover most cases.


    This would be targeted at single messages containing multiple payloads. Like the Temp/humidity example
    10 S 2020-01-21 12:51:46 home/OpenMQTTGateway_SRFB/PilighttoMQTT {"message":{"id":63,"temperature":24.30,"humidity":10.00, "ba ttery":1,"channel":1},"protocol":"tfa","length":"63","repeat s":2,"status":2}
    However, you would need the ability to create multiple HS devices over the same topic.

    There's surely other options, let me know what you think,

    Thx

    Leave a comment:


  • 123qweasd
    replied
    Depending on the devices, the unit code is either


    message:unit (almost all devices - first three above + everything else I have tested so far)

    or

    message:id (Temp/humidity sensor - probably when it's a "raw" decoded signal)

    Thx

    Leave a comment:


  • Michael McSharry
    replied
    It is not an uncommon situation where the publisher has designed messages where part of the topic is put into the payload. In you examples where does the unit code show up? I’m guessing it is the id key. Shelly also uses id in its announce topic to identify which unit is making the announcement.

    The problem dealing with this is not making the topic include the id so it becomes unique, but to know when id key is used for this purpose and then when other keys need to used in a similar way. Global settings are easy, topic-specific settings can only be established after a topic has been seen. Let me see what I can do to handle the general case.

    Leave a comment:


  • 123qweasd
    replied
    Here are some examples for more clarity:

    The following 3 devices (Door sensor, motion sensor, water leak) share the same protocol but have a different "unit" code, however their "state" is published under the same topic.

    Sub: home/OpenMQTTGateway_SRFB/PilighttoMQTT:message:state

    Check time stamps for Topic changes.


    So I would need to combine two topics to make it unique


    Otherwise if I create a HS device for Sub:

    home/OpenMQTTGateway_SRFB/PilighttoMQTT:message:state,

    I will get status from all devices....


    DOOR SENSOR "CLOSED"
    Click image for larger version  Name:	door sensor closed.JPG Views:	0 Size:	68.5 KB ID:	1355797


    DOOR SENSOR "OPENED"

    Click image for larger version  Name:	door sensor opened.JPG Views:	0 Size:	76.9 KB ID:	1355798

    MOTION SENSOR "ON"
    Click image for larger version  Name:	Motion sensor on.JPG Views:	0 Size:	75.8 KB ID:	1355799

    WATER LEAK SENSOR "ON"
    Click image for larger version  Name:	Water leak detector.JPG Views:	0 Size:	74.2 KB ID:	1355800


    For Temperature/Humidity, the "id" differentiates the correctly decoded signal (id 63, protocol tfa) vs the incorrect "noise" signal which is of no use (id 147.0, protocol teknihall)

    Again here if I created a single device with Sub: home/OpenMQTTGateway_SRFB/PilighttoMQTT:message:temperature
    Temperature value will cycle between "id 63" and "id 147"

    Click image for larger version  Name:	Temperature-Humidity incorrect protocol.JPG Views:	0 Size:	73.9 KB ID:	1355801Click image for larger version  Name:	Temperature-Humidity correct protocol.JPG Views:	0 Size:	73.4 KB ID:	1355802

    I need to have a way to "group" things so that it points to unique devices.

    Let me know what you think,

    Thx.,


    Click image for larger version

Name:	image_85486.jpg
Views:	301
Size:	66.8 KB
ID:	1355803

    Leave a comment:


  • Michael McSharry
    replied
    [qoute]Can you provide your comments on the approach to to create separate HS devices for each device/sub ?[/quote]
    I am not understanding your question. Each line that you highlighted, as well the others, will go into separate HS devices.

    Leave a comment:


  • 123qweasd
    replied
    Thanks for looking into it Michael, much appreciated;
    The algorithm was based on the DG-R8H, I assumed it would be similar but doesn's seem to be the case.
    Could still be useful if someone doesn't want to hack the Sonoff Bridge.

    The above (my previous post - after hacking device and getting decoded MQTT data) should be simpler to tackle I guess.
    Can you provide your comments on the approach to to create separate HS devices for each device/sub ?

    I did some further testing and highlighted below the data that is either unique to each device (message:unit, message:ID) or can help distinguish one from another (protocol, message:channel)

    Thanks!

    Click image for larger version  Name:	Capture.JPG Views:	0 Size:	76.9 KB ID:	1355765

    Leave a comment:


  • Michael McSharry
    replied
    I did a decode of your packet and the values do not look reasonable based upon the algorithm I saw at https://github.com/Portisch/RF-Bridg...BB1/issues/119 -

    In any case the attached is the script with instructions for use at the top. It goes in the \scripts folder. I tested on Windows. The event I used to register the callback is
    Click image for larger version

Name:	Capture.PNG
Views:	303
Size:	22.9 KB
ID:	1355752

    Leave a comment:


  • 123qweasd
    replied
    Done!
    Hacking completed and OpenMQTTgateway uploaded successfully.
    Not an easy hack but works a charm.
    Since only one SonoffRF unit seems enough to cover a regular house it's worth the effort.
    Everything centralized & processed via the same unit.

    Receiving all MQTT codes correctly including the decoded/parsed Temperature and Humidity (and battery status) data from the Digoo DG-R8S.

    The only thing is that the unit sends two distinct MQTT messages simultaneously (ID 63 & ID 147.0) but only one is valid: ID63 "TFA protocol"
    How would you suggest I manage the data in order to create 2 separate HS devices for :

    ID63 "TFA protocol" : Temperature
    ID63 "TFA protocol" : Humidity

    since they share the same SUB?
    let me know if I should move this to your Forum/Thread.


    10 S 2020-01-21 12:51:46 home/OpenMQTTGateway_SRFB/PilighttoMQTT {"message":{"id":63,"temperature":24.30,"humidity":10.00,"ba ttery":1,"channel":1},"protocol":"tfa","length":"63","repeat s":2,"status":2}
    11 S 2020-01-21 12:51:46 home/OpenMQTTGateway_SRFB/PilighttoMQTT {"message":{"id":147.0,"temperature":1.5,"humidity":24.0,"ba ttery":1.0},"protocol":"teknihall","length":"147","repeats": 2,"status":2}
    12 S 2020-01-21 12:52:36 home/OpenMQTTGateway_SRFB/PilighttoMQTT {"message":{"id":63,"temperature":24.30,"humidity":10.00,"ba ttery":1,"channel":1},"protocol":"tfa","length":"63","repeat s":2,"status":2}
    13 S 2020-01-21 12:52:36 home/OpenMQTTGateway_SRFB/PilighttoMQTT {"message":{"id":147.0,"temperature":1.5,"humidity":24.0,"ba ttery":1.0},"protocol":"teknihall","length":"147","repeats": 2,"status":2}
    14 S 2020-01-21 12:53:26 home/OpenMQTTGateway_SRFB/PilighttoMQTT {"message":{"id":63,"temperature":24.30,"humidity":10.00,"ba ttery":1,"channel":1},"protocol":"tfa","length":"63","repeat s":2,"status":2}
    15 S 2020-01-21 12:53:26 home/OpenMQTTGateway_SRFB/PilighttoMQTT {"message":{"id":147.0,"temperature":1.5,"humidity":24.0,"ba ttery":1.0},"protocol":"teknihall","length":"147","repeats": 2,"status":2}
    16 S 2020-01-21 12:54:16 home/OpenMQTTGateway_SRFB/PilighttoMQTT {"message":{"id":63,"temperature":24.30,"humidity":10.00,"ba ttery":1,"channel":1},"protocol":"tfa","length":"63","repeat s":2,"status":2}
    17 S 2020-01-21 12:54:16 home/OpenMQTTGateway_SRFB/PilighttoMQTT {"message":{"id":147.0,"temperature":1.5,"humidity":24.0,"ba ttery":1.0},"protocol":"teknihall","length":"147","repeats": 2,"status":2}
    18 S 2020-01-21 12:55:06 home/OpenMQTTGateway_SRFB/PilighttoMQTT {"message":{"id":63,"temperature":24.30,"humidity":10.00,"ba ttery":1,"channel":1},"protocol":"tfa","length":"63","repeat s":2,"status":2}
    19 S 2020-01-21 12:55:06 home/OpenMQTTGateway_SRFB/PilighttoMQTT {"message":{"id":147.0,"temperature":1.5,"humidity":24.0,"ba ttery":1.0},"protocol":"teknihall","length":"147","repeats": 2,"status":2}

    Click image for larger version

Name:	Capture.PNG
Views:	314
Size:	30.3 KB
ID:	1355750

    Leave a comment:


  • Michael McSharry
    replied
    I looked at one of your references and now understand the raw format of the 0, 1, 2 stream. The 0 is the separator. The 1 is binary 0. The 2 is binary 1. For humidity use the last 8 binary bits. For temperature throw away the rightmost 12 and then the new rightmost 12 is the temp scaled by 10. This specific decode is not hard in script and is just a variant of the one you posted.

    Leave a comment:


  • 123qweasd
    replied
    Yes. All other devices I want to use are already tested and working pefectly with the SonoffRFbridge/Tasmota/Portisch combo:

    https://github.com/arendst/Tasmota/w...-RF-Bridge-433

    Easy hack. Works perfectly/tested for : Motion sensors, Door sensors, Water leak sensors, Smoke detectors. (All from Aliexpress, 5-10$ each)
    All of these devices typically send just a few simple codes (on, off, etc) so the MQTT/HS integration is quite easy.
    But for the Temp/Humidity sensor, the range of data is much larger and requires conversion/parsing of the raw data to extract Temp/Humidity values.

    I also deployed DS18x20 and Si7021 wired sensors with Sonoff Basic and TH10/16. great solution.
    But I like the idea of having a few wireless sensors I can move around as needed.

    I am currently flashing the stock OMG software to my Sonoff Bridge, will see how it behaves... https://docs.openmqttgateway.com/

    Worst case, I'll do the physical hack and use this solution instead which is documented as working:

    https://1technophile.blogspot.com/20...or-how-to.html
    https://github.com/xoseperez/espurna...---Direct-Hack

    hopefully it decodes both the simple and more advanced codes so I don't have to deploy 2 bridges (which wouldn't be a major problem either)

    Will report back later.

    Cheers,

    Leave a comment:


  • Michael McSharry
    replied
    So your only need is for decoding this one specific device?

    Leave a comment:


  • 123qweasd
    replied
    Thanks Michael, actually the decoding "recipe" has already been identified for this specific device:

    https://github.com/Portisch/RF-Bridg...BB1/issues/119 - see last post

    The format is constant and temp/humidity values can be extracted manually from the raw data. The objective is to "automate" the extraction.

    --------------------

    As a plan B, I'll purchase a second Sonoff bridge and hack it to use a different platform that seems to decode this natively. (VS Tasmota+Portisch)

    https://1technophile.blogspot.com/20...or-how-to.html
    https://github.com/xoseperez/espurna...---Direct-Hack

    Will report back later. Thx

    Leave a comment:

Working...
X