Announcement

Collapse
No announcement yet.

Location Isolation with ESP32 Bluetooth Low Energy

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • MattL0
    replied
    Thanks Michael!

    Leave a comment:


  • Michael McSharry
    replied
    I curious to know if you did pursue with thios with raspberry pi?
    I do not recall who was trying to use the RPi, but I did do an implementation on RPi of the same API used for ESP32. From mcsMQTT/HS perspective it is transparent if the end node is a RPi or ESP32 so a mixture could be used. It was posted at https://forums.homeseer.com/forum/li...eacon-tracking

    The RPi does make for a better BLE client which I suspect is due to the independent antennae between WiFi and BLE vs. ESP32 which shares one. The RPi CPU has more throughput than the ESP32 so that could contribute as well.

    I took a look at the mcsMQTT manual and did not see any update based upon the RPi and BLE. I normally document the projects there. Likely it did not have any following so never took the effort. Actually very little interest overall and I did not migrate BLE to the HS4 version of mcsMQTT. I believe multiple ESP32/RPi running to support close-location identification is too much complexity for the typical HS user. It was a good learning experience project, but not something that is for the mass market.

    Leave a comment:


  • MattL0
    replied
    hi Michael,

    I curious to know if you did pursue with thios with raspberry pi?

    I am now using this plugin ( via the jeedom hs3 plugin) https://jeedom.github.io/plugin-blea/en_US/index
    You will need to transalte this in english with google.


    The idea of this plugin is to monitor ble devices. And you can use others raspberry pi/ linux pc as ''entenna''. My main use is to detect if a device is present or not ( ex. my keys with a Tile pro ( bluetooth tracker) on them). When any of the entenna detect the bl;e devcie , it is shown as ''present'' with a value of 1. When any of the entanna havent seen the b;le device since a while ( user configurable ) it is show as not present with a value of 0. The main problem with jeedom is that all value of device a cached into ram... so when the the jeedom machine restart .. it is possible that some plugin takes a empty '''' string an interprete this as a value oif zero.

    anyways


    The plugin works pretty well, exept for when the jeedom machine is restarted.

    I would like to be able to send thoses data ( only presence detection) myself to my hs3 machine via mosquitto_sub.
    Then i guess i could elaborate a logic in hs3 after that .

    I am also curious if you did some research about this topic normal ble dongle on linux , using the data from hci0 scan?



    one last thing, the bluetooth dongle is praysed by the jeedom community, it provides more range and you can change the enttanna for 9dbi one or 12.

    Leave a comment:


  • Michael McSharry
    replied
    I was not aware of the project, but did a look at the documentation. The passive approach is common between Find3 and what I have done. Both use RSSI as the primary measurement.

    There are two primary differences. The obvious is WiFi vs. Bluetooth and this really make little difference for this application.

    The second is that Find3 translates the set of RSSI into a specific X,Y (or room) as a fingerprint. There is a setup phase where you move the smartphone into each room that is a candidate for tracking and use the set of RSSI measurements from each RPi as the fingerprint for that phone in that room. In my implementation the RSSI is used to trilaterate an X,Y position using an algorithm similar to what is used to isolate earthquake epicenters from multiple seismic station readings. There is no learning phase with this approach.

    WiFi broadcast intervals are measured in minutes and are variable. Bluetooth scanning intervals are measured in seconds with the RPi being more capable of short intervals than the ESP32. If one's use case is to know when the tracked device is starting to move, is moving or has stopped moving then the faster interval is needed.

    I know in my household smartphones are usually stationary when in the house at some known location so when time to use it then will remember where it is. Others may carry there phone on-person all the time. Carrying beacons on a keychain or similar has the same issue. For the tracking to work the user behavior has to be that the device is always on-person.

    The Find3 document does acknowledge IOS MAC spoofing and my experience has been that Android does this as well to protect the user from passive scanning. IOS and Android do this with Bluetooth too. Beacons used with Bluetooth tracking by definition do not spoof their MAC address.

    Find3 looks to be a nice project, but with fundamentally different approaches to location identification. I do not see how what I have done could be integrated other than having two answers that would then be used by a higher level integration application. For me the BLE tracking effort was done as a learning vehicle for ESP32, Bluetooth, kalman filtering, and location isolation algorithms. It was an excellent learning experience. The need for on-person devices eliminates a practical application for me. For others this constraint will not be a problem.

    Leave a comment:


  • OliP
    replied
    Just found this project, looks really interesting.
    Earlier this year i started looking at Find3 - https://github.com/schollz/find3
    Wondered if you were aware of it, or if any part of it could help or assist this project.

    I've paused work on implementing Find3 for now as i have too much on my plate, but am still interested in this area.

    Leave a comment:


  • Michael McSharry
    replied
    I am familiar with ESP-NOW which is a capability developed Espressif. I had always considered it's benefit to be with battery operation associated with deep sleep so the ESP could be awaken and the WiFi on-time was significantly reduced largely due to reduction in the overhead to establish a connection. I had not considered the application where one server acted as a gateway in a star configuration. This gateway node would then still need to communicate with an application server such as HomeSeer using some other protocol (such as MQTT).

    What I cannot envision is how this becomes integrated into other HA components. All ESP processor products use a single WiFi radio and even the ESP32 shares the Bluetooth and WiFi antenna. Once data has been collected by the ESP-NOW server I do not see how it gets communicated to other parts of the HA environment. I suspect it may be possible to use something like RPi that has both ethernet and Wifi, but then the software would need to be developed to implement the Expressif protocol on the RPi hardware.

    There are other variants of communication protocols that have been developed to satisfy a specific niche. These typically exploit some aspect of a standard protocol to remove a capability and optimize another. When the standard is sacrificed then the breath of application is reduced. These solutions have their place, but they do not replace the predecessor protocol.

    In the general case of low power IOT devices one expects the devices to either lose connection because they are low-cost marginal hardware designs or they intentionally go offline to conserve battery. A system designed needs to consider the Quality Of Service that is being supplied. For example, consider an IOT device that turns on a light and it is important that the light does turn on. The application (e.g. HomeSeer) commands the light to the on state, but the IOT device it not connected at the time the command is sent. The MQTT protocol assures a QOS so that the command is held back until the IOT device has advertised itself as now being awake and connected.

    Leave a comment:


  • kiley.jerry@gmail.com
    replied
    I have your mqtt plugin but I am not sure mqtt is the most efficient protocol to use. Have you seen the ESP-Now protocol. It is made for stuff like this. 40ms from sleep to transmitting. Hardly any overhead so you could do wireless reveivers and have amazing battery life or much shorter intervals. Once a device is paired to the master it is a persistent connection. The master can be directly connected to the home seer servers and have 20 receivers If you need more receivers add another master. It has CRC and you could pull individual sensors on different frequencies. I love mqtt but this is what this protocol is designed for. 250 byte packet size. Basically a wireless can-bus system. I am not a programmer or I would try to do it myself but I have used ESP-NOW with amazing results. Hopefully this suggestion doesn't annoy you but you did say you were trying different things.

    Leave a comment:


  • Michael McSharry
    replied
    While I gave up on nodejs due to abandoned libraries, I did find libraries that can be used with C on RPi. I can do continuous polling of BLE beacons and report any change in RSSI. I am moving down the path of trying to implement the same business logic that I used in the ESP32 in the RPi. Short of this it should be pretty easy to just do reporting of RSSI/Distance from the RPi. I suspect the same will work with USB dongles as with the BLE built into the RPi.

    Leave a comment:


  • Michael McSharry
    replied
    Extending BLE location monitoring to RPi using nodejs seemed like a very good excuse to learn nodes just as it was to learn ESP32 development. Unfortunately it looks as if the nodejs Bluetooth libraries have been abandoned as many others seem to have the same install issue I have been having. I may try again in the future, but it is not looking very positive as the issue surfaced years ago and it seems to still exist. There was a lot of interest when Bluetooth first became available with RPi, but now the interest seems to have dwindled.

    Leave a comment:


  • ts1234
    replied
    Michael,

    Thanks for doing all you do. This is very over my head. Hopefully in a year or so I'll be a little more fluid. I do think long and short range BT have a lot of uses...but since home automation to me is your home responding to you automatically...stable tools for "where" you are in your house as well as arrival / departure sensing are really core/key to everything. I'm continuously amazed that major manufactures have not focused on this as about everything else could really be done with simple timers and hard wiring. That's a bit of an exaggeration. no doubt. However, some of the biggest complaints I've see posted center around things especially like the flakiness of zware arrive sensors. For example, I want my garage to open right when I drive up...not a 1/2 a block away...or sitting there waiting for it to open. I don't want it to open at 2 AM because my zwave arrival sensor disappears wjile I'm sleeping and then comes back. Clearly BT has many applications, but most (really all) reasonably priced approaches today are simply not stable and depend on cell phones and/or wireless TCP/IP and really work perfectly only about 30%-50% of the time. Cell service in my neighborhood sucks which make this even worse. Some of this might be OK for somethings, but for unlocking your house or any security related function it simply isn't.

    Tom

    Leave a comment:


  • Michael McSharry
    replied
    I found a very promising approach using node.js on YouTube. While I have never developed anything in node.js, it is used for zigbee2mqtt, node-red and many others. I followed the YouTube video, but ran into a library install issue that appears to be related to the version npm/node.js that is current. Node.js should provide an easy port to the API used for ESP32 so should be able to mix and match RPi/ESP32 with mcsMQTT. I will see if I find anything else following this thread tomorrow.

    Leave a comment:


  • ts1234
    replied
    That is pretty awesome!!! In my testing I couldn't connect either. I am guessing you'd see a more gradual RSSI value change with some of the long range ones. I'm way over my head here...just starting to load Visual Studio ...and already digging since some of the menus have changed in 2019 it looks like, but can't even find the Homeseer templates or even know if it's ok to run VS2019 or I really need to find the 2017 version. ...Going to have to start reading.

    Leave a comment:


  • Michael McSharry
    replied
    I have been poking around and see that RPi has bluetooth built-in, but the same should work with dongles as well. At https://www.raspberrypi.org/forums/v...ic.php?t=47466 is a good discussion of using linux script for bluetooth tracking. The core of it is the hcidump command. I have been playing with it and can get "hcidump lescan" to show the bluetooth devices. If I run "hcidump lescan --duplicates" then it produces a constant stream of output showing the nearby bluetooth devices. This is the continuous scan that you desire.

    I was not able to get the beacon to connect which I suspect is normal for a beacon, but to use the rssi hcidump command one needs to be connected. In this same thread I found "sudo btmon |grep RSSI &" to spawn off a process that dumps rssi info from bluetooth. I then setup a whitelist with one beacon and used hcidump with duplicates to repeat the scan. This resulted in the image below.

    Click image for larger version

Name:	Capture.PNG
Views:	213
Size:	53.5 KB
ID:	1324088

    I then moved the beacon 10 ft away and got RSSI around -90. Another 10 ft and the RSSI only marginally change to -92 or so.

    What this shows me is that one should be able to get BLE RSSI info for beacons relatively easily with the drivers built into the OS. More google seaching may even find more complete projects that may be able to be easily adaptable to work with HS.

    Leave a comment:


  • ts1234
    replied
    Again, thanks for EVERYTHING!

    I find it almost funny that one of my jobs was taking care of Sun Microsystems data network for 7 years and yet in that time I never learned UNIX (sigh). It seems far easier to go the RPi route as you state. I'll look over the SDK, but guessing it's not the way I'd go as one of my weakest points has always been ...read the darn manual.

    Again,

    Thanks for all the help!...especially on Labor Day...

    Tom

    Leave a comment:


  • Michael McSharry
    replied
    The ESP32 power is not the limiting factor, but I suspect its architecture is. It has a single 2.4 GHz antenna that is used for both Wifi and Bluetooth. Since Wifi is its method of interface it needs to keep this part relatively responsive. Something like a RPi with Ethernet for the interface and USB for the Bluetooth should have plenty of horsepower. I do not have experience working with hardware devices on Linux, but you may be able to do it at the Linux rather than source code level such as shown https://raspberry-projects.com/pi/pi...tooth-commands

    If I was to go the RPi route then I would not write the HS plugin, but use MQTT as the communication protocol and let mcsMQTT handle interface with Homeseer. This should also be doable at the Linux level by piping the BLE output to mosquitto_pub application. Of course a tighter and more flexible approach would be to develop source code that communicated directly with the devices. You could even follow the API that is documented in mcsMQTT.pdf so that the additional BLE features of mcsMQTT could be utilized. It just depends upon how fancy you want to get. It does seem, however, that just basic info is all you need. Since the RPi has the horsepower it seems most appropriate to maximize its share of the processing and only communicate event-level information to HS.

    If you do want to write a plugin, but the Homeseer SDK download provides a template. There are also users that have improved upon the HS template. I have no specific recommendations, but a board search should find some hits.

    Leave a comment:

Working...
X