Announcement

Collapse
No announcement yet.

BLE - Flashed, Booted, MQTT'd, No data in BLE Plugin

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    BLE - Flashed, Booted, MQTT'd, No data in BLE Plugin

    After fighting to flash the ESP, I've done my due diligence by reading the documentation.
    I've gone through multiple iterations with the name/mqtt/etc. I cannot make the BLE page display any information.

    ESP32:
    Everything appears to be fine here, the fallback topic is the only thing I see that could be changed:

    Code:
    00:00:00 Project BLEScan BLEScan (Topic BLEScan/1, Fallback BLEScan_1/1, GroupTopic BLEScanners) Version 5.12.0d-STAGED
    00:00:00 CFG: Saved to flash at FFFFFFFF, Count 220, Bytes 3768
    00:00:00 WIF: Attempting connection...
    00:00:00 WIF: Connecting to AP1 IOTNetwork in mode 11
    00:00:01 WIF: Connected
    00:00:01 DNS: Initialized
    00:00:01 HTP: Web server active on BLEScan/1-2700.local with IP address 192.168.1.81
    00:00:01 APP: (UTC) Sun Aug 23 00:08:49 2020, (DST) Sun Aug 23 00:08:49 2020, (STD) Sun Aug 23 00:08:49 2020
    12:08:51 MQT: Attempting connection...
    12:08:51 MQT: Connected
    12:08:51 MQT: BLEScan/1/LWT = Online (retained)
    12:08:51 MQT: Subscribe to BLEScan/2/+
    12:08:51 MQT: Subscribe to BLEScan/3/+
    12:08:51 MQT: Subscribe to BLEScan/4/+
    12:08:51 MQT: Subscribe to BLEScan/5/+
    12:08:51 MQT: Subscribe to BLEScan/6/+
    12:08:51 MQT: Subscribe to BLEScan/7/+
    12:08:51 MQT: Subscribe to BLEScan/8/+
    12:08:51 MQT: Subscribe to BLEScan/9/+
    12:08:51 MQT: Subscribe to BLEScan/10/+
    12:08:51 MQT: BLEScan/1/cmnd/POWER =
    12:08:51 MQT: Subscribe to BLEScan/1/cmnd/#
    12:08:51 MQT: Subscribe to BLEScanners/cmnd/#
    12:08:51 MQT: Subscribe to cmnd/BLEScan_1/1/#
    12:08:51 MQT: BLEScan/1/INFO1 = {"Module":"WEMOS","Version":"5.12.0d","FallbackTopic":"BLESc an_1/1","GroupTopic":"BLEScanners", "UTimers":"8"}
    12:08:51 MQT: BLEScan/1/INFO2 = {"WebServerMode":"Admin","Hostname":"BLEScan/1-2700","IPAddress":"192.168.1.81"}
    12:08:51 MQT: BLEScan/1/INFO3 = {"RestartReason":"1"}
    12:08:51 MQT: BLEScan/1/66-e6-e6-9e-b9-0b = {"Characteristics":{"TxPower_1":60,"TxPower_10":100,"Blackli st":0}}
    12:08:52 MQT: BLEScan/1/4c-0f-9a-1b-79-50 = {"Characteristics":{"TxPower_1":60,"TxPower_10":100,"Blackli st":0}}
    12:08:52 MQT: BLEScan/1/77-71-dc-76-92-0a = {"Characteristics":{"TxPower_1":60,"TxPower_10":100,"Blackli st":0}}
    12:08:52 MQT: BLEScan/1/14-c3-41-30-3e-bb = {"Characteristics":{"TxPower_1":60,"TxPower_10":100,"Blackli st":0}}
    12:08:52 MQT: BLEScan/1/50-21-cd-53-c9-9a = {"Characteristics":{"TxPower_1":60,"TxPower_10":100,"Blackli st":0}}
    12:08:52 MQT: BLEScan/1/43-19-94-3b-81-61 = {"Characteristics":{"TxPower_1":60,"TxPower_10":100,"Blackli st":0}}
    12:08:52 MQT: BLEScan/1/62-e2-e2-a9-a8-eb = {"Characteristics":{"TxPower_1":60,"TxPower_10":100,"Blackli st":0}}
    12:08:57 Found Beacon: Name: , Address: 6a:aa:aa:e4:dc:83, manufacturer data: e000011aca6a6b35, serviceUUID: 0000fe9f-0000-1000-8000-00805f9b34fb
    And here's the BLE page, not a single thing on it.

    Looking at the BLEScan topic under the MQTT section, information is clearly being published.

    #2
    I will take a look tomorrow to assess why the BLE information is not being collected for the BLE page. It has been many versions ago that BLE was exercised so it is likely that something got missed in the updates for HS4 compatibility.

    Comment


      #3
      Originally posted by Michael McSharry View Post
      I will take a look tomorrow to assess why the BLE information is not being collected for the BLE page. It has been many versions ago that BLE was exercised so it is likely that something got missed in the updates for HS4 compatibility.
      As I seem to be the only one going down the road of BLE, I greatly appreciate you spending time to troubleshoot it. Thanks!

      Comment


        #4
        Turns out it was a trival error that is fixed in http://mcsSprinklers.com/mcsMQTT_5_5_1_6.zip. The only difficulty I noticed is following the rename of a beacom the topics received with the prior name are retained. This will confuse the BLE page. To overcome this use the Obsolete column checkboxes to remove the old topics.

        As an awareness, when only one scanner is used then the beacon location will always be at the scanner location since that will be the center of the circle of the RSSI-based radius. What will change is the distance from the scanner.

        Comment


          #5
          Originally posted by Michael McSharry View Post
          Turns out it was a trival error that is fixed in http://mcsSprinklers.com/mcsMQTT_5_5_1_6.zip. The only difficulty I noticed is following the rename of a beacom the topics received with the prior name are retained. This will confuse the BLE page. To overcome this use the Obsolete column checkboxes to remove the old topics.

          As an awareness, when only one scanner is used then the beacon location will always be at the scanner location since that will be the center of the circle of the RSSI-based radius. What will change is the distance from the scanner.
          I'm not seeing any changes with this version. I replaced all the files and the BLE page is still empty.

          Comment


            #6
            The fix allowed Scanner (1) to be recognized. Note in your screenshot the "View Beacon Info from Scanner Selected" row of "Page Viewing Options" has only "Stop Updates" radio available. This was the same for me prior to 5.5.1.6. If not able to select a scanner then there are no beacons that will be shown that are visible from the unselected scanner.

            Your settng has the radio to show only beacons with the H column selected. Since no beacons have been visbible it is not possible to selected the H column checkboxes so none of the beacons will be shown.

            If this does not help then zip up \data\mcsMQTT\mcsMQTT.db, \data\mcsMQTT\mcsMQTT Debug.txt, \config\mcsMQTT.ini and email to mcsSolutions at CenturyTel dot net. This will give me more visibility to help. Note that the debug file assumes you already had MQTT page, General Tab, Enable debug checkbox selected at the time that the ESP32 started transmitting and you tried to view the data with mcsMQTT.

            Click image for larger version

Name:	Capture.PNG
Views:	108
Size:	93.2 KB
ID:	1413847

            Comment


              #7
              Originally posted by Michael McSharry View Post
              If not able to select a scanner then there are no beacons that will be shown that are visible from the unselected scanner.

              Your settng has the radio to show only beacons with the H column selected. Since no beacons have been visbible it is not possible to selected the H column checkboxes so none of the beacons will be shown.
              I was wondering about that pip. I checked MQTT, the LWT is online (I don't know if the BLE portion communicates with MQTT at all, or if they are in essence two separate plugins, that both communicate by means of MQTT).

              I confirmed the installed version was 5.5.1.6. Unchecked the boxes so that my configuration looked like yours then I rebooted the server for good measure. There are still no beacons listed under the page viewing options.

              I will enable the debug log, and reboot the server. And confirm that BLEScan\1 is being seen then ship everything off to you.
              Thanks!

              Comment


                #8
                After initialization completed the first messages I saw were the config topic and characteristics for a beacon. This is good from the ESP32.

                BLEScan/1/CONFIG,Payload={"Location":{"X":10,"Y":20},"Filter":{"RSSI": "0","XY":0,"Zone":0,"Dropout":4},"Management":{"Receiver Gain ":100,"ScanInterval":60,"ScanDuration":2,"ReportingMode" :0," DisableNew":0,"MasterScanner":0,"SelectedScanner":1,"Retain" :0}}, Triggers=3

                BLEScan/1/76-76-76-72-c3-d9,Payload={"Characteristics":{"TxPower_1":60,"TxPower_10":1 00,"Blacklist":0}}, Triggers=3

                Subsequent to this I did not recognize any attempt to access the BLE page of the plugin. It may be that there is no debug in this sequence, but seems like something should exist. Did you try to access the page while the debug data was being collected and the ESP32 restarted?

                In your first screenshot it showed no scanners were observed. Are you now seeing scanner 1 as in my latest screenshot or are you still seeing none?

                There are many obsolete versions of BLE topics that I suggest are removed. The LWT topics are retained by the broker so these are harder to remove and not that significant, but the other can be done within mcsMQTT. What I see in your debug is a variety of formats
                tele/esp32/BLE-scan
                esp32/BLE-scan-1/tele
                1/BLE-Scan/tele
                BLE-Scan/1/tele

                From General Tab, Inbound Management, Obsolete Row enter tele/esp32/# to remove the first. Do similar for the others so all this obsolete data is removed from mcsMQTT.

                Comment


                  #9
                  Originally posted by Michael McSharry View Post
                  After initialization completed the first messages I saw were the config topic and characteristics for a beacon. This is good from the ESP32.

                  BLEScan/1/CONFIG,Payload={"Location":{"X":10,"Y":20},"Filter":{"RSSI": "0","XY":0,"Zone":0,"Dropout":4},"Management":{"Receiver Gain ":100,"ScanInterval":60,"ScanDuration":2,"ReportingMode" :0," DisableNew":0,"MasterScanner":0,"SelectedScanner":1,"Retain" :0}}, Triggers=3

                  BLEScan/1/76-76-76-72-c3-d9,Payload={"Characteristics":{"TxPower_1":60,"TxPower_10":1 00,"Blacklist":0}}, Triggers=3
                  From the ESP it seemed like everything was behaving.

                  Subsequent to this I did not recognize any attempt to access the BLE page of the plugin. It may be that there is no debug in this sequence, but seems like something should exist. Did you try to access the page while the debug data was being collected and the ESP32 restarted?
                  I have to believe so, because I went back several times to check it while the debug was running.


                  In your first screenshot it showed no scanners were observed. Are you now seeing scanner 1 as in my latest screenshot or are you still seeing none?
                  I still see nothing in the BLE Setup. Only "Stop Updates" exists.
                  Click image for larger version  Name:	ble_nothingnew.png Views:	0 Size:	76.2 KB ID:	1413992

                  There are many obsolete versions of BLE topics that I suggest are removed. The LWT topics are retained by the broker so these are harder to remove and not that significant, but the other can be done within mcsMQTT. What I see in your debug is a variety of formats
                  tele/esp32/BLE-scan
                  esp32/BLE-scan-1/tele
                  1/BLE-Scan/tele
                  BLE-Scan/1/tele

                  From General Tab, Inbound Management, Obsolete Row enter tele/esp32/# to remove the first. Do similar for the others so all this obsolete data is removed from mcsMQTT.
                  I will do that. I meant to do it previously. Those were from my previous experiments before I got the ESP32 BLE bin to flash successfully.

                  Comment


                    #10
                    I will extract pertinent messages from your debug and simulate your ESP32 and see what I learn. Likely tomorrow.

                    Comment


                      #11
                      Originally posted by Michael McSharry View Post
                      There are many obsolete versions of BLE topics that I suggest are removed. The LWT topics are retained by the broker so these are harder to remove and not that significant, but the other can be done within mcsMQTT. What I see in your debug is a variety of formats
                      tele/esp32/BLE-scan
                      esp32/BLE-scan-1/tele
                      1/BLE-Scan/tele
                      BLE-Scan/1/tele
                      I removed everything except BLEScan and BLEScanners. The BLE page hasn't changed. Do you want the same files again? Or just the debug?

                      Comment


                        #12
                        Originally posted by Michael McSharry View Post
                        I will extract pertinent messages from your debug and simulate your ESP32 and see what I learn. Likely tomorrow.
                        Other than being attached to MQTT and on my network, I reflashed the ESP and changed as little as possible.

                        Comment


                          #13
                          I believe the issue you are having is that you have disable new topic discovery. That means the topics that are being sent by ESP32 after you got the flash working correctly are not being retained so nothing appears to change. As a minimum this should be cleared until all the beacons of interest to you are showing on the BLE page.

                          Click image for larger version  Name:	Capture.PNG Views:	0 Size:	13.2 KB ID:	1414131

                          After changing this setting and the broker IP from your mcsMQTT.ini and then use your data from the debug file to first define the scanner
                          Code:
                          BLEScan/1/CONFIG,Payload={""Location"":{""X"":10,""Y"":20},""Filter"": {""RSSI"":""0"",""XY"":0,""Zone"":0,""Dropout"":4},""Managem ent"":{""ReceiverGain"":100,""ScanInterval"":60,""ScanDurati on"":2,""ReportingMode"":0,""DisableNew"":0,""MasterScanner" ":0,""SelectedScanner"":1,""Retain"":0}}
                          and then to define the beacon
                          Code:
                          BLEScan/1/76-76-76-72-c3-d9,Payload={""Scan"":{""Address"":""76:76:76:72:c3:d9"",""Ra wRSSI"":-85,""FilteredRSSI"":-85.00},""Location"":{""BeaconDistance"":11,""BeaconFOM"":11, ""BeaconX"":10,""BeaconY"":20,""Zone"":1020}}, Triggers=3
                          I obtain the following
                          Click image for larger version  Name:	Capture1.PNG Views:	0 Size:	70.3 KB ID:	1414132

                          I did not review the mcsMQTT document, but a scan duration of 2 seconds seems short to me. This setup indicates that a scan is made every minute and the window when listening for BLE is two seconds. The antenna is shared so most of the time it is being used for WiFi. What I am not certain is the spec on beacons for how long they have to respond to a announcement request. It seems to me I was giving it more time during my original development, but I may have learned that two seconds was enough.

                          Comment


                            #14
                            Originally posted by Michael McSharry View Post
                            I believe the issue you are having is that you have disable new topic discovery.
                            That's odd. I do not recall ever checking that box as I definitely want to discover new topics.

                            After changing this setting and the broker IP from your mcsMQTT.ini and then use your data from the debug file to first define the scanner
                            Code:
                            BLEScan/1/CONFIG,Payload={""Location"":{""X"":10,""Y"":20},""Filter"": {""RSSI"":""0"",""XY"":0,""Zone"":0,""Dropout"":4},""Managem ent"":{""ReceiverGain"":100,""ScanInterval"":60,""ScanDurati on"":2,""ReportingMode"":0,""DisableNew"":0,""MasterScanner" ":0,""SelectedScanner"":1,""Retain"":0}}
                            and then to define the beacon
                            Code:
                            BLEScan/1/76-76-76-72-c3-d9,Payload={""Scan"":{""Address"":""76:76:76:72:c3:d9"",""Ra wRSSI"":-85,""FilteredRSSI"":-85.00},""Location"":{""BeaconDistance"":11,""BeaconFOM"":11, ""BeaconX"":10,""BeaconY"":20,""Zone"":1020}}, Triggers=3
                            I made the "Discover new topics" change and didn't see anything until I executed those publist changes. Now the scanner and beacon list appears.
                            Oddly enough, before the scanner pip was there, beacons were showing under the location and distance pages. Unfortunately I did not take a screen shot.

                            In Location; There was single white circle which looked to have 20 MAC addresses stacked on top of it.
                            Under Distance; There was a dot and a mac address.

                            Looking at the beacons now, it appears the beacon information was being registered, but the scanner was not.

                            I did not review the mcsMQTT document, but a scan duration of 2 seconds seems short to me. This setup indicates that a scan is made every minute and the window when listening for BLE is two seconds. The antenna is shared so most of the time it is being used for WiFi. What I am not certain is the spec on beacons for how long they have to respond to a announcement request. It seems to me I was giving it more time during my original development, but I may have learned that two seconds was enough.
                            I'm not sure where that 2 second value came from. Prior to the publist changes, it was set at 30. After them, it appears as 2. This is a setting (whether in the ESP or the BLE console) I have not modified.

                            However, it appears to be up and running. Thank you! Will I need to execute publist's for additional scanners?

                            Comment


                              #15
                              I dont know what is in your publist, but I suspect it was not the publist that kicked things off, but that the ESP32 responded with the scanner information in the /CONFIG topic. This is done on each restart of ESP32 and if the restart is done after mcsMQTT finishes initialization then it should be seen.

                              With only one scanner the location of the beacon is a radius around the scanner of distance that is a function of RSSI. This means all beacons are centered at the scanner. Not until multiple scanners are are used will there be a mathematical means to asses another (x,y) center point for a beacon.

                              Review post #4 about removing obsolete topic if you change the name of beacon from its MAC address.

                              Comment

                              Working...
                              X