Help with implementing BLE on RPi running HS4

[Michael McSharry]

I am not familiar with blueman-applet, but with a quick google search did not find any use of MQTT protrocol by it. Why do you believe it uses this protocol.

Another observation is that the topic being used starts with /. This means there is a null for the first level in the MQTT topic. While valid, it will be removed by mcsMQTT as the messages are received.

What are you trying to do that BLEMQTT is not doing? BLEMQTT's purpose is to provide BLE beacon reporting from RPi for use by HS/mcsMQTT.

[Richel]

I am not familiar with blueman-applet, but with a quick google search did not find any use of MQTT protrocol by it. Why do you believe it uses this protocol.

Another observation is that the topic being used starts with /. This means there is a null for the first level in the MQTT topic. While valid, it will be removed by mcsMQTT as the messages are received.

What are you trying to do that BLEMQTT is not doing? BLEMQTT's purpose is to provide BLE beacon reporting from RPi for use by HS/mcsMQTT.

Michael:
Thanks. I was just using blueman-applet to make certain that my Bluetooth adapter was working and not for connecting to anything. I will now go back and try to implement BLEMQTT, because what I am doing presently is too complex for me. I figured out one thing, though: I needed to change a setting in the config.json file: "http_proxy" : false, to "http_proxy" : true. Now it connects to the Bluetooth adapter, after I start Espruino (./start.sh) works briefly, then shuts down:
[Discover] Scanning started.
[Discover] BLE broken? No advertising packets in 20 seconds - restarting!
[CONNECT] Connections [] IDLE

Elliott

[Richel]

With version 5.25.1.0 I added the ability to publish the BLE Beacon topics recognized in the Windows BLE stack. It is done by replicating the Sub Topic into the Pub Topic text box on the Edit tab or Association tab.

For your Linux problem I suspect it is due to BLEMQTTHS4 vs. BLEMQTT. It was originally created with BLEMQTT and then compatibility/cross-platform use cause issues so BLEMQTTTHS4 was created to be used with the HS4 plugin. I believe the current Updater files have the correct files present. Confirm the file exists on the path you posted. Another problem could be HomeSeer being in the patch vs. HomeSeerHS4 (or any other variant). A third potential problem is that BLEMQTTHS4 is not flagged as executable. chmod or WinSCP can be used to change the properties.

Back to running HS4 on my RPi. I am not detecting BLE beacons with mcsMQTT and I think it is because I cannot get BLEMQTTHS4 to start.
Again, to recap:
1) BLEMQTTHs4 has been flagged as executable.
2) BLEMQTTHS4 is in /usr/local/HomeSeer/bin/mcsMQTT
3) /usr/local/HomeSeer/bin/mcsMQTT$ ./BLEMQTTHS4 gives "No such file or directory.
4) Plug in version, which is the latest, is 5.25.1.0 on the General page, while the HS4 plugin in page shows it to be 5.24.10.0.
5) HS4 version is HS4 Standard Edition 4.2.13.0 (Linux).

Elliott

[Michael McSharry]

The zip file contained in the Updater download is attached. It contains a readMe.txt file. It also contains CommandLine.ini that is an alternate way to specify the broker IP and login credentials. It could be that it is looking for CommandLine.ini in the same folder as BLEMQTTHS4 since it appears your test does not include the broker IP on the command line used to start BLEMQTT.

The original syntax for starting from the command line is shown below when using broker at IP 192.168.0.100 and broker needing no login credentials

sudo /usr/local/HomeSeer/bin/mcsMQTT/BLEMQTTHS4 192.168.0.100

or

cd /usr/local/HomeSeer/bin/mcsMQTT
sudo ./BLEMQTTHS4 192.168.0.100

Another option with the latest version is to put 192.168.0.100 in /usr/local/HomeSeer/bin/mcsMQTT/CommandLine.ini and then 192.168.0.100 would not be used in the above two examples such as

sudo /usr/local/HomeSeer/bin/mcsMQTT/BLEMQTTHS4

If HS4 is also running on the computer where BLEMQTTHS4 is being run the mcsMQTT will manage starting BLEMQTTHS4 based upon the settings on the Bluetooth tab. I see that is assumes there are no broker login credentials when it does start it and it uses the first broker setup in mcsMQTT.

[Richel]

The zip file contained in the Updater download is attached. It contains a readMe.txt file. It also contains CommandLine.ini that is an alternate way to specify the broker IP and login credentials. It could be that it is looking for CommandLine.ini in the same folder as BLEMQTTHS4 since it appears your test does not include the broker IP on the command line used to start BLEMQTT.

The original syntax for starting from the command line is shown below when using broker at IP 192.168.0.100 and broker needing no login credentials

sudo /usr/local/HomeSeer/bin/mcsMQTT/BLEMQTTHS4 192.168.0.100

or

cd /usr/local/HomeSeer/bin/mcsMQTT
sudo ./BLEMQTTHS4 192.168.0.100

Another option with the latest version is to put 192.168.0.100 in /usr/local/HomeSeer/bin/mcsMQTT/CommandLine.ini and then 192.168.0.100 would not be used in the above two examples such as

sudo /usr/local/HomeSeer/bin/mcsMQTT/BLEMQTTHS4

If HS4 is also running on the computer where BLEMQTTHS4 is being run the mcsMQTT will manage starting BLEMQTTHS4 based upon the settings on the Bluetooth tab. I see that is assumes there are no broker login credentials when it does start it and it uses the first broker setup in mcsMQTT.

Michael:
Thank you.
1) I downloaded the Zip file, extracted the files, and copied them to /usr/local/HomeSeer/bin/mcsMQTT
2) I did do cd /usr/local/HomeSeer/bin/mcsMQTT$ ./BLEMQTTHS4 [broker ip address]. I tried both the IP address of the RPi running HS4 (192.168.4.109) and the IP address on the mcsMQTT General page (127.0.0.1). This always resulted in "No such file or directory." I am not using any login credentials.
3) I edited the CommandLine file to include the IP address of the broker, with the same result.
4) Nonetheless, since HS4 is also running on the computer where BLEMQTTHS4 is being run, mcsMQTT is not managing starting it for me, despite having the radio button selected on the Local>Bluetooth tab to discover BLE beacons.
What can I try next?
Elliott

[Michael McSharry]

root@RPI4USB:~# cd /usr/local/HomeSeer
root@RPI4USB:/usr/local/HomeSeer# cd bin/mcsMQTT
root@RPI4USB:/usr/local/HomeSeer/bin/mcsMQTT# ls
AES.dll libpaho-mqtt3a.so paho.mqtt.c
AES.pdb M2Mqtt.Net.dll RESTARTHS.exe
BLEMQTT MCSMQTT_2020.dll Restart.Trace
BLEMQTTHS4 MCSMQTT_2020.pdb sqlite3.dll
BluetoothBeacon.dll MCSMQTTHS4_2020.dll System.Runtime.WindowsRuntime.dll
BluetoothBeacon.pdb MCSMQTTHS4_2020.pdb System.Web.DataVisualization.dll
Broadlink.dll MinimalisticTelnet.dll websocket-sharp.dll
Broadlink.pdb MinimalisticTelnet.pdb WindowsBase.dll
Eval3.dll Mono.Data.Sqlite.dll Windows.winmd
Eval3.pdb MySql.Data.dll
GnatMQ.Net.dll Newtonsoft.Json.dll

================ Attempt to start a non-existant file =================

root@RPI4USB:/usr/local/HomeSeer/bin/mcsMQTT# ./BLE
-bash: ./BLE: No such file or directory

================ Attempt to start BLEMQTTHS4 before it's properties where changed to 755 (all users executable)

root@RPI4USB:/usr/local/HomeSeer/bin/mcsMQTT# sudo ./BLEMQTTHS4 192.168.0.16
sudo: unable to resolve host RPI4USB: Name or service not known
sudo: ./BLEMQTTHS4: command not found

=============== Attempt after changing properties to 755, Ctl-C used to stop after it started  ====================

root@RPI4USB:/usr/local/HomeSeer/bin/mcsMQTT# sudo ./BLEMQTTHS4 192.168.0.16
sudo: unable to resolve host RPI4USB: Name or service not known
Starting Beacon Scan using Client 192_168_0_16
Connect with Broker count 0
Waiting for Subscribe
Successful connection isConnected= 1
Subscribe succeeded
^CBLEScan exit due to signal 2
exiting
Exiting BLEScan
root@RPI4USB:/usr/local/HomeSeer/bin/mcsMQTT#

Looking at BLEMQTTHS4.c I see the following at the startup. Before any file access is explicitly performed the message "Starting Beacon Scan using Client" is printed in the terminal window. Since this is not printed in your terminal window the problem is not after execution started but finding the file to execute.

int main(int argc, char* argv[]) //sudo ./BLEMQTT localhost [username password]
{
//printf("Starting Beacon Scan\n");
Settings.mqtt_log = MQTTASYNC_TRACE_ERROR;
signal(SIGINT, signal_callback_handler);

int fd;
struct ifreq ifr;

fd = socket(AF_INET, SOCK_DGRAM, 0);
/* I want to get an IPv4 IP address */
ifr.ifr_addr.sa_family = AF_INET;
/* I want IP address attached to "eth0" */
strncpy(ifr.ifr_name, "eth0", IFNAMSIZ-1);
ioctl(fd, SIOCGIFADDR, &ifr);
close(fd);
/* display result */
snprintf(Settings.IP,sizeof(Settings.IP),"%s", inet_ntoa(((struct sockaddr_in *)&ifr.ifr_addr)->sin_addr));
ReplaceAll(Settings.IP,'.','_');
printf("Starting Beacon Scan using Client %s\n",Settings.IP);

You can repeat what I did and capture the terminal window output that shows the filepath and listing of files. Based upon my test above it behaves as if BLEMQTTHS4 is not present from the context where the command to start it is being issued.

As another point of note on broker IP. If you want to use the broker that is on the RPi then use 127.0.0.1. If you want to use the broker that is on another computer, then use the IP address of that computer. In the mcsMQTT setup for Broker IP an entry of 127.0.0.1 means to start an internal broker. A remote computer cannot access the Windows loopback IP 127.0.0.1. You need to use the IP that is visible outside of the Windows computer such as 192.168.0.100. In your description you have not provided the IP of the Windows computer, only 192.168.4.109 for the RPi.

[Richel]

So, if I understand what you wrote, I need to change properties to 755 for file BLEMQTTHS4? I tried changing the properties with a command line that I found on the web and this did not work ("not permitted"). What did you use? Also, I do understand about the IP addresses. My Windows-based HS4 connects nicely to the RPi-based HS4 broker using its IP address (192.168.4.109). Thanks. Elliott

[Michael McSharry]

If you want the RPi to be your broker used for your network then all settings should use 192.168.4.109. I though your setting on Windows mcsMQTT was 127.0.0.1 which means it is a local broker and not a RPi broker.

I use WINSCP to manage Window/Linux cross-platform. I also use PuTTY for the terminal window. WINSCP is so easy to use as it makes the Linux file system look like part of the Windows file system.

You can use chmod 755 rather than chmod +x from a terminal window. The difference is that 755 covers all users while +x covers the current user.

If you are not logged in with elevated privledges then your commans will all need to be preceeded with sudo. If you are not permitted to use sudo then you need to log in with an account that the privledge.

[Richel]

If you want the RPi to be your broker used for your network then all settings should use 192.168.4.109. I though your setting on Windows mcsMQTT was 127.0.0.1 which means it is a local broker and not a RPi broker.

I use WINSCP to manage Window/Linux cross-platform. I also use PuTTY for the terminal window. WINSCP is so easy to use as it makes the Linux file system look like part of the Windows file system.

You can use chmod 755 rather than chmod +x from a terminal window. The difference is that 755 covers all users while +x covers the current user.

If you are not logged in with elevated privledges then your commans will all need to be preceeded with sudo. If you are not permitted to use sudo then you need to log in with an account that the privledge.

Michael:
Thanks. WINSCP is great! I have been using PuTTY for quite some time. So, looking at the properties of BLEMQTTHS4 with WINSCP under "Permissions" it states Octal 0755. so, I guess, it already has the correct permissions. Further, using WINSCP, I ran "execute." and once, again, got:
Command '"./BLEMQTTHS4"'
failed with return code 127 and the following error message.
-bash: line 7: ./BLEMQTTHS4: No such file or directory

Just a thought from that information: is it looking for another file somewhere, since BLEMQTTHS4 clearly exists?
Elliott

[Michael McSharry]

What I know about Linux comes from google searches. There are many "No such file or directory" hits such as [SOLVED] "No such file or directory" when file exists (linuxquestions.org) in which case the issues was 32 bit vs. 64 bit. I would like to have a specific answer to give, but have no more ideas.

BLEMQTT was originally a port of the code I did for ESP32 to run on RPi as part of HS3 BLE support for X/Y position of a BLE beacon. I tried to document the build process in mcsMQTT.pdf. Later when HS4 came into existence there was a modification made to make it work for the BLE support provided in HS4 which was a simple home/away rather than position. If I recall you did try to build BLEMQTT from sources, but don't recall the progress. The zip file attached in this thread does have the sources that I used for the most recent build that I had done.

[Richel]

What I know about Linux comes from google searches. There are many "No such file or directory" hits such as [SOLVED] "No such file or directory" when file exists (linuxquestions.org) in which case the issues was 32 bit vs. 64 bit. I would like to have a specific answer to give, but have no more ideas.

BLEMQTT was originally a port of the code I did for ESP32 to run on RPi as part of HS3 BLE support for X/Y position of a BLE beacon. I tried to document the build process in mcsMQTT.pdf. Later when HS4 came into existence there was a modification made to make it work for the BLE support provided in HS4 which was a simple home/away rather than position. If I recall you did try to build BLEMQTT from sources, but don't recall the progress. The zip file attached in this thread does have the sources that I used for the most recent build that I had done.

Michael:
Thanks. So, it could possibly be the OS? I am running Debian 11.4, but I don't know whether it is 32- or 64-bit. I am not running HS4 from their image file, because my understanding is that in that implementation, Bluetooth is turned off, because of interference with Z-Wave, which I would not be using. In any case, I couldn't get mcsMQTT to see BLE beacons on that version. So, I installed an OS on the RPi and then installed HS4.
I stopped trying to implement BLEMQTT because I got stymied at a step and decided to try again with HS4 and mcsMQTT. I think the step was that BLEMQTT would not stay connected to Bluetooth. Elliott

[Richel]

What I know about Linux comes from google searches. There are many "No such file or directory" hits such as [SOLVED] "No such file or directory" when file exists (linuxquestions.org) in which case the issues was 32 bit vs. 64 bit. I would like to have a specific answer to give, but have no more ideas...

So, Michael, it is a 32-bit versus 64-bit problem. My RPi was installed in a 64-bit OS. I installed the 32-bit OS, installed HS4 and ./BLEMQTTHS4 works (to some extent)!
It is still not quite functional. When I did ./BLEMQTTHS4, I got the following:
Starting Beacon Scan using Client 40_1_214_118
Failed to set scan parameters data.: Operation not permitted
Failed to set event mask.: Bad file descriptor
Failed to enable scan.: Bad file descriptor
Could not set socket options
: Bad file descriptor
Failed to disable scan.: Bad file descriptor
Exiting BLEScan
Where the heck is it getting Client 40_1_214_118 and how do I fix that and the other errors?
Elliott

EDIT: Well, son of a gun, despite the above, after disabling and enabling mcsMQTT on HS4 running on RPi, it is working! Beacons are being discovered. Hard to believe, finally! You need to add to your documentation not to use a 64-bit OS on an RPi, if you are using this for HS4.

[Michael McSharry]

The transition from a 32 bit OS to a 64 bit OS is often a slow one because of dependencies of libraries used by applications that have not been rebuilt and made available for use with the 64 bit OS. In many cases the authors have moved on to greener pastures and the library never gets rebuilt. Users of many applications will not be able to migrate to a 64 bit OS. I will add some words to mcsMQTT.pdf to make this awareness.

40.1.214.118 is the eth0 (wired ethernet) IP that BLEMQTT found for the RPi when it queried the OS
The "Failed to ..." and "Could not set..." messages are the error responses during initialization as BLEMQTT tries to configure the Host Controller Interface (HCI) for the Bluetooth stack. I do not know specifically why they exists, but best guess is a privilege issue so assure sudo is used when starting BLEMQTT to guarantee elevated privledge.