Announcement

Collapse
No announcement yet.

Current Queue Depth at 130K and Increasing!!!

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Current Queue Depth at 130K and Increasing!!!

    Have been fighting this issue where the Queue depth is LARGE and increasing. I am convinced it is my coding problem, as I have mostly custom coded devices attached to HS3. Pointers to how to find the mqtt messages that are flooding the system would be appreciated.

    System:
    • Three ESP brokers connected to mcsMQTT
    • Three SONOFF/Tasmota devices
    • Many CUSTOM coded ESP devices connected to the 3 brokers and do not have LWT defined
    • Many X10 devices connected through TW523
    Observation:

    Today, after I thought I had stabilized the problem saw the following.
    • 8pm last night restarted 3 brokers:
    • 7am today....
    • Queue Current/Max at 133,000 and rising.
    • Interesting, manually commanding from PC is successful... appears queue is delayed for local commanding????
    Action:
    • Disable/Enable plug-in
    • Queue went to MAX = 7 and has been there for over an hour.
    I am assuming a new burst will happen some time today; how can I catch what is causing the queue to increase??? Attached PDF has pictures showing full Statistics tab.
    Attached Files

    #2
    Suggest capturing the received messages in the History database or enabling debug that will provide a log in the lines starting with "AcceptMessageForTrigger". While the History can be viewed from the mcsMQTT UI, for this case of large amount of data it would probably be more efficient to look in \data\mcsMQTT\mcsMQTT History.db with SQLite DB Browser of other database viewing tool. The debug file is in the same folder as the database file.

    Comment


      #3

      Suggest capturing the received messages in the History database or enabling debug that will provide a log in the lines starting with "AcceptMessageForTrigger". While the History can be viewed from the mcsMQTT UI, for this case of large amount of data it would probably be more efficient to look in \data\mcsMQTT\mcsMQTT History.db with SQLite DB Browser of other database viewing tool. The debug file is in the same folder as the database file.

      Michael... thanks.. I am not as smart as you had hoped.

      #1 could use additional direction for implementing: enabling debug that will provide a log in the lines starting with "AcceptMessageForTrigger". Not in the manual, and not in my skill set... yet.

      #2 history.db I am starting to put ALL my devices into the history db. Would #1 allow me to avoid this step???

      #3 Only if you have time... when the queue is at 10K, and I issue a command from the UI. Why does the command get through before all the 10K items in the queue?? If I manually send a mqtt command, will IT then be stuck behind all the 10K messages waiting???

      thanks.







      Comment


        #4
        The file mcsMQTT Debug.txt contains much information to help isolate issues that may arise. Some of the line start as shown below where the Topic and Payload are shown for the received message. I highlighted in red the key to search to find these lines containing the received messages. The green highlight is the topic. You will be looking for some topic that repeats many, many times to fill the queue.
        Code:
        4/10/2020 4:47:14 PM 16216982 | ActoOnMessageForTrigger Topic FilteredWater/RESULT,Payload={"POWER":"ON"}, Triggers=0
        4/10/2020 4:47:14 PM 16217121 | ActoOnMessageForTrigger Topic FilteredWater/POWER,Payload=ON, Triggers=0
        4/10/2020 4:47:21 PM 16223839 | ActoOnMessageForTrigger Topic BedroomFireplace/SENSOR,Payload={"Time":"2020-04-10T16:47:20","ENERGY":{"TotalStartTime":"2019-01-03T16:50:56","Total":144.797,"Yesterday":0.023,"Today":0.016 ,"Power":1,"ApparentPower":4,"ReactivePower":4,"Factor":0.34 ,"Voltage":121,"Current":0.036}}, Triggers=0
        With the history database using DB Browser for SQLLite you will be looking in the Topic column of the Browse Data tab where the topic repeats many times in a short period of time. LastDate column is the time when it occurred. In the example below I have three topics that repeat, but they repeat every few minutes. In your case the repeat pattern will be many in the same second.

        Either technique of using the debug log or the database will produce the same result. Whichever one is easier for you would be appropriate to use.

        Click image for larger version

Name:	Capture.PNG
Views:	36
Size:	77.5 KB
ID:	1428104

        Comment

        Working...
        X