Ring to MQTT installation and use

[Pete]

Refer to: ring-mqtt

Been running this plugin since it's inception. Note I am running Homeseer here (HS3 and HS4) on Ubuntu 18.04 and Ubuntu 20.04. Never tried a Windows installation of Docker here.

Note here have never been much of a wireless / automation person. Current WAP of choice here is Ruckus these days.
Wall switch testing and use now is Tasmota upgraded switches, TTS is Alexa devices (have DOTs, Alexa standard, Alexa Show devices).
All of the automation is MQTT based.

Note that the Ring Alarm system and yearly fees are inexpensive. The Ring hub has a SIM card failover which works well. Here mounted Ring Hub near ceiling on wall with a POE connection to managed POE switch. I also have the Ring keyboards connected to 5VDC such that I never have to charge them.

Well now HS manages two alarm panels here in two homes.

1 - Leviton OmniPro 2
2 - Ring Alarm panel

I have it integrated to to Homeseer TTS and Alexa TTS (via Alexa Media player).

This script leverages the excellent ring-client-api to provide a bridge between MQTT and suppoted Ring devices such as alarm control panel, lights and cameras (full list of supported devices and features). It also provides support for Home Assistant style MQTT auto-discovery which allows for easy Home Assistant integration with minimal configuration (requires Home Assistant MQTT integration to be enabled). This also includes an optional Home Assistant Addon for users of HassOS/Home Assistant Installer. It can also be used with any other tool capable of working with MQTT as it provides consistent topic naming based on location/device ID.

Docker install (original installation for me)

It is important that you read entire document:

Environment Variables

• RINGTOKEN - The refresh token received after authenticating with 2FA, see Authentication section for details
• MQTTHOST - Hostname for MQTT broker
• MQTTPORT - Port number for MQTT broker
• MQTTUSER - MQTTPASSWORD
• MQTTPASSWORD - Password for MQTT broker
• ENABLECAMERAS - Enable camera support, otherwise only alarm devices will be discovered
• SNAPSHOTMODE - Enable still snapshot image updates from camera, see Snapshot Options for details
• ENABLEMODES - Enable support for Location Modes for sites without a Ring Alarm Panel
• ENABLEPANIC - Enable panic buttons on Alarm Control Panel device
• ENABLEVOLUME - Enable volume control on Keypads and Base Station, see Volume Control for important information about this feature
• RINGLOCATIONIDS - Array of location Ids in format: "loc-id","loc-id2", see Limiting Locations for details
• BRANCH - During startup pull latest master/dev branch from Github instead of running local copy, see Branch Feature for details.

Standard Installation:

Originally installed it on Ubuntu 18.04 ==> /opt/ring-mqtt

Stanard installation is fully supported, please make sure Node.js is installed (tested with 12.18.x but should work on 10.x and higher) on your system and then clone this repo:

git clone https://github.com/tsightler/ring-mqtt.git

Change to the ring-mqtt directory and run:

chmod +x ring-mqtt.js npm install

This will install all required dependencies. Edit config.js to configure your Ring refresh token and MQTT broker connection information and any other settings (see Config Options. Note that the user the script runs as will need permission to write the config.json as, for the standalone version of the script, updated refresh tokens are written directly to the config.json file.

Docker Installation:

For Docker installtion details, please read this section entirely. While it is possible to build the image locally from the included Dockerfile, it is recommended to install and update by pulling the official image directly from Docker Hub. You can pull the image with the following command:

docker pull tsightler/ring-mqtt

docker run --rm -e "MQTTHOST=host_name" -e "MQTTPORT=host_port" -e "MQTTRINGTOPIC=ring_topic" -e "MQTTHASSTOPIC=hass_topic" -e "MQTTUSER=mqtt_user" -e "MQTTPASSWORD=mqtt_pw" -e "RINGTOKEN=ring_refreshToken" -e "ENABLECAMERAS=true-or-false" -e "RINGLOCATIONIDS=comma-separated_location_IDs" tsightler/ring-mqtt

Starting ring-mqtt during boot

For standalone installs the repo includes a sample unit file which can be used to automaticlly start the script during system boot as long as your system uses systemd (most modern Linux distros). The unit file assumes that the script is installed in /opt/ring-mqtt and it runs the script as the root user (to make sure it has permissions to write config.json), but you can easily modify this to any path and user you'd like. Just edit the file as required and drop it in /lib/systemd/system then run the following:


systemctl daemon-reload
systemctl enable ring-mqtt
systemctl start ring-mqtt

Using mcsMQTT autogen you will see a Ring Alarm keypad with all functions (except entering codes)

I have been having some issues with TTS reporting alarm status at 3 AM which is low on the WAF. Most likely it is relating to having 3 instances of this plugin running.

[awhitton]

Pete, I saw your post about ring-mqtt elsewhere which was exactly what I needed, thank you. I managed to get ring-mqtt running on my Synology NAS and all my ring devices are appearing in mcsMQTT. However, have you been able to use mcsMQTT to get the snapshot images that ring-mqtt says it provides?

[Michael McSharry]

If I need to do something with the plugin to achieve what you desire then let me know. I do not have any Ring products.

[Pete]

have you been able to use mcsMQTT to get the snapshot images that ring-mqtt says it provides

No, I removed my Ring Doorbell after about a month after purchase and installation and purchased a Hikvision Doorbell.

I will probably install a Ring Doorbell in house #2 sometime in the near future....Amazon Prime days...now that I can control it and monitor it with MQTT versus using my smartphone...

IE: House number 2 is only going to be using MQTT. IE: keeping adding in wall WiFI switches talking MQTT.

[awhitton]

Michael McSharry, the snapshot is captured by mcsMQTT, but it is presented as a huge long text string. I have not tried using images before in Homeseer but I am assuming to make use of this I'd have to have a way of saving this mqtt payload as a file on the server such that clients like HS Touch, HS Buddy etc can then pick them up to display. I have not immediately seen a way to do this with your plugin. Any thoughts?

[Michael McSharry]

The MQTT spec has a relatively small payload max length. This is likely driven by its intended use in small-resource devices. When I did the jpg download via MQTT to the LED sign I did a multipart publication and the sign rebuilt it. This protocol is documented in mcsMQTT.pdf.

While it may be out of bounds of the spec, the long string you are describing could be stored in a file and to support this another Control/Status UI item of "file" could be added. The filename would be derived from the Topic. I suspect it would be stored in a folder that is accessible from the HS server so in something like \html\mcsMQTT\file\TopicName.file. Rather than "file" it may be better to use the image type that is being supported here such as "jpg" or "png". This way the file would be directly usable by an App that recognizes file types.

[awhitton]

Michael,

Yes the author said they only did the image snapshot as opposed to video because of this limitation in MQTT, which makes sense.

Thanks for the tips on how to proceed. I'll give it a go.

[awhitton]

Michael McSharry, had a quick look at moving this project along and see that you meant you would need to add the Control/Status UI option of making a file to your plugin to allow a file to be created; as opposed to it already being there. Apologies I misunderstood. The key for the filename is that it remains the same in my particular use case; so your suggestion makes sense. This is an example of the Ring Snapshot topic "ring/afa06d00-91a1-49ed-8698-1653ad64c51f/camera/10082c56ad93/snapshot/image" which doesn't seem to contain any invalid characters and would guarantee uniqueness. Another option that might lead to a more friendly naming but would put uniqueness in the hands of the user would be /file/floor/room/name.jpg I agree that being able to put the right filetype ending would be extremely helpful and avoid the need for another process to be running. Can't obviously see where that might fit in the dialogue without an extra field. Anyway would be great if you could add this but no worries if not.

[Michael McSharry]

It seems that the topic is a use setup parameters MQTTRINGTOPIC. Is there anything other than the image that is sent on this topic? It would be easier to key off of the topic than off of the Control/StatusUI. No additional setup needed by user and then I also do not need to do conversions between binary as text and then back to binary again as the file is being saved.

If this is not possible the the following is the proposed approach for using Control/Status UI.

What I will do is add the Control/Status UI of "jpg".
If associated with HS device then the payload will be stored in /html/mcsMQTT/jpg\{floor}/{room}/{name}.jpg. This will allow it to be served by HS server.
The HS Device Value will be incremented for each time the topic is received/file stored for ease of triggering if desired.
On error case where the file is locked or other issue the file will not be stored and feedback will be in the debug log.

What I do not know is the ability of the MQTT library I am using to handle large payloads. We will see. I assume you already have a broker that handles them.

Any suggestions to modify the above approach?

[awhitton]

Hi Michael,

In my Docker image I don't use the parameter MQTTRINGTOPIC and the topic in my previous post is what it came up with by default i.e. a root of "ring" I presume.

As far as I can tell there are three entries in your Associations page that are relevant. A single line says "ring/afa06d00-91a1-49ed-8698-1653ad64c51f/camera/10082c56ad93/snapshot/attributes".
This is followed by "Sub: ring/afa06d00-91a1-49ed-8698-1653ad64c51f/camera/10082c56ad93/snapshot/attributes:timestamp" which has a 10 digit number as the payload.
Followed by "Dev: ring|afa06d00-91a1-49ed-8698-1653ad64c51f|image
Sub: ring/afa06d00-91a1-49ed-8698-1653ad64c51f/camera/10082c56ad93/snapshot/image"

The content of image topic is a very long text representation of presumably a binary string - certainly I cant copy it and save it as a JPG to view it.

I think what you have suggested for the file naming is fine. Also, incrementing the device value is a terrific idea to support triggers as you say.

The one issue for automatic file use is the file type which could be one of many. Making the ControlStatusUI be jpg makes it clear to the user you are supporting jpg - but in reality it is any binary payload and any filetype could be attached - for future enhancement if needed.

I have looked at https://github.com/tsightler/ring-mqtt and the referenced https://github.com/dgreif/ring/ but I can't find any reference to the actual file type that these images are coming in. All I can see is that it is using ffmpeg which supports almost every image format. I have downloaded a snapshot manually from the RING app and it provided me with a JPG file so I think that this is a safe bet for this use case.

Having looked at a few instances of the payload (as represented in text); if there is any additional information in them then it is binary encoded as no obvious structure is present. They do all seem to have the same/similar starts which I presume is part of the file format to set out some meta data for the image., Here are the first few characters "

Sub: ring/afa06d00-91a1-49ed-8698-1653ad64c51f/camera/10082c56ad93/snapshot/image Pub: the following Topic on Device command

����Lavc58.35.100��C

".

Having google the text string Lavc58.35.100. It seems to be linked to metadata to describe the encoder so I think this is the start of the file and it is just the image in the payload

I currently using the broker you provide with mcsMQTT - I am assuming the whole payload is coming in but we shall see. Now I have figured out Docker I think I can install a different broker if need be quite easily.

Thanks

Andy

[Michael McSharry]

The update is made at http://mcsSprinklers.com/MCSMQTTHS4_51620.zip for HS4 and http://mcsSprinklers.com/MCSMQTT_51620.zip for HS3. I tested the HS4 and then updated the source for HS3. I used Mosquitto as the broker during the test. I evaluated jpg files up to 5.8 MB. I assumed no encoding was done, but a binary transfer of the file in the payload. Let me know how it goes on your end. Only user change needed is to select "jpg File" as the Control/Status UI type.

[Michael McSharry]

Another thing that could be done is to populate the HS DeviceString with a hyperlink to the saved image, an image tag that show the picture in the HS Device Feature, or a thumbnail created that could be the target of the image tag. I just don't know what would be useful.

[Michael McSharry]

I made the updates to create a thumbnail, include it in the VSP definition so it shows as an icon and put it in the DeviceString as a hyperlinked image. If clicked the full size image will be shown in the browser. Below are two test images. One is a 150x200 screenshot and the other is a 6 MB camera picture. When the plugin starts it clears the device string so HS will show the DeviceValue count of images recevied (28 in this example). When an image is received the DeviceString is populated as shown for device 8841 with the thumbnail image and hyperlink to full size image.

An oddity I saw with the picture is that it was taken as a portrait and the file properties both before and after MQTT transfer showed this to be the case. When I generated the thumbnail the image appeared rotated -90 degrees in the .NET code so was converted to a landscape. It did not happen with my other test files.

Updates at http://mcsSprinklers.com/MCSMQTT_51621.zip for HS3 and http://mcsSprinklers.com/MCSMQTTHS4_51621.zip for HS4. I tested both HS3 and HS4 versions and confirmed the image transfered using the mcsMQTT internal broker.

[awhitton]

Michael, I upgrade to the HS4 plugin from the HS3 and had installed the previous upgraded version and it worked in that it created the files with the correct names that I could open - Thank you so much for doing this; and so quickly. I will now try this newer version. As an aside, is it my browser or has the HS4 UI made your UI seem much less elegant - everything is so large and the filter boxes have no edges! Might revert back to the HS3 version as much preferred that look.

[Michael McSharry]

The presentation was designed using HS3 API and converted to the HS4 API. This means it was designed for a desktop and converted to the smartphone presentation. HS4 presentation will never be as compact at HS4 because of the dynamic sizing that is done to be able to work on any size smartphone. If I was to start from scratch and designed around HS4 API it could be better, but never will be as concise as what is available with HS3.