Announcement

Collapse
No announcement yet.

Plugin causing network trouble every 2 minutes

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Plugin causing network trouble every 2 minutes

    I'm setting up some RGB lighting and bought a Magic Home controller and loaded the trial of this plugin a couple of weeks ago. I didn't immediately connect the dots but have been having problems with connectivity between my Elk alarm and the HS3 UltraM1G3 plugin recently. I finally got down to business debugging this problem yesterday.

    Running continuous pings to multiple devices, I noticed that several of the devices would drop a packet or two every 2 minutes like clockwork. My Elk, however, would freak out and drop packets for about 20 seconds before recovering. After unplugging almost every device on my network one-by-one, the problem went away when I eventually unplugged my Homeseer NUC. I had added a couple of new plugins and updated a couple of others recently so started disabling plugins to narrow it down.

    Long story somewhat short. The Magic Home plugin was the culprit. I suppose it's going out every 2 minutes to try to discover any new controllers and whatever it's doing is wreaking havoc on my network and alarm panel.

    The Magic Home controller looks to be a piece of junk (at least the one I received) it will only connect to my WiFi if my LEDs are not plugged in. As soon as I connect the LEDs to the controller it drops the WiFi. I'm returning the controller so I won't be needing this plugin, but I figured I'd report this problem anyway in case the developer wants to try to resolve or lessen the network impact of the discovery attempts.

    #2
    Check for unauthorized API calls.

    Comment


      #3
      I've just come across the same issue. Started experiencing intermittent issues during conference calls and presumed it to be issues with ISP. I was able to see intermittent connection timeouts when pinging google.com every few minutes or so.

      After trouble shooting various things and shutting down homeseer I was then able to narrow it down to the Magichome plugin. Note that my strip lights were not even turned on.

      Comment


        #4
        I haven't seen this behavior at all under most operation, the plugin only does the discovery at startup and once every hour. the status of the devices is requested every few minutes, but this is no longer a broadcast but rather a targeted request to the device IP. I have however seen however once personally on a cheapo 8 port netgear managed switch that flow control was being erroneously triggered and was issuing pause frames to the connected devices on the switch, there was no further firmware to resolve but as soon as either flow control was disabled across all devices this resolved it. but as flow control was a required feature this single switch was replaced with a unifi 8 port switch with POE. not had any further issue at all.

        Don't suppose you guys have netgear switches do you?

        Comment


          #5
          I logged into the forms for the first time in years to tell the story of my life for the last 4 weeks. I can 1000% confirm, this plugin has caused me much grief. Watch this video. This is proof that this plugin causes issues in certain networking scenarios. Watch in ping plotter what happens when I enable the plugin. Instant network packet loss.

          https://streamable.com/i8qp2l


          I have an all Ubiquti Unifi network. Practically 100% up time and 0% issues. I installed these LED lights for my daughter and connected them to wifi. No issues. I wanted to control the lights from Homeseer and I saw this plugin. So I set it up. Worked like a champ. All of a sudden my Sonos in the living room which is linked to the sub over the network start distorting. My wifi calling on my cell phone starts dropping out. Video streams start pausing. I start getting PWNED in CoD MW. I was like what the hell. This plugin was the cause.

          So fuzzysb I enabled flow control on my 48 port switch and that prevents the plugin from causing the issue but still I am blown away this simple little plugin bought this 25 year network veteran to tears lol. CCIE and all. I was puzzled. I still won't be using the plugin I am going to burn it to the ground and derezz that SOB.

          Comment


            #6
            djdzam I am not experiencing this on my network, and like you i have Ubiquiti Unifi switches and Nano HD and Pro AP's. the router is Ubiquiti EdgeRouter 6p. I do not experience this issue. I did make improvements to the discovery in the beta available here I suggest you try this version, manually updating the files and let me know if this improves matters.

            As you pointed out the time of disruption is every 2 minutes. which does not coincide with any timer in the software, Every 60 mins for discovery and every 1 minute for status. What router are you using? I currently have flow control off on my network currently. you say that flow control fixes it, but i struggle to see how communication to and from a 2.4ghz Wifi led strip is maxing out a 1Gbps link to be resolved with the switch issuing pause frames. can you get the metrics for the number of pause frames that are being issued on the switchport your homeseer server is connected to (Assuming wired). i should be able to reproduce what you have approximately, i am presuming you as a network professional have a separate IoT VLAN as do i, if so how are you managing discovery and are you performing discovery across VLAN's? just so i can try and reproduce in some fashion.

            Just been running the same ping test on the router, also additionally a number of WAN IP endpoints with no drop in pings or performance when starting or every 2 minutes. one thing not 100% clear is what you were pinging to see the dropouts. is it your switch/router or other?

            Comment


              #7
              fuzzysb I am only responding and providing this information because I am now realizing you are the developer of the plugin so sorry for saying I was going to burn your plugin to the ground. I won't be doing any more troubleshooting forward because I spent way too much time diagnosing the problem for the last month. My issue has competently disappeared once I disabled the Magic Home plugin for Homeseer. That I can confirm. My network consists of 1 USG (not the PRO just the small firewall), the Gen 1 48 port PoE switch, and 5 UAP-AC-PRO access points. That's it. All running the latest firmware although I tried earlier versions too.I didn't put the led strip on a separate VLAN because I figured it has to be on the same vlan as my kids iPad so the app on the iPad could control it. I do have other VLANS on my network but homeseer and the led lights are on the same one. I will note that my Homeseer server runs as a KVM virtual machine on an unRaid server that is uplinked to the switch via 10GB twinax. Not sure if that has something to do with it. Could be an issue in the virtual switch stack in KVM that rolls down to the Ubiquiti side from the 10GB port. The dropouts were about 1-2 total pings loses but in most cases sub 1ms repleys to my local gateway became 30-50MS which is crazy for a device on the same network as your client. And the disruption wasn't exactly 2 mins to the number but it felt that consistent.

              Comment


                #8
                djdzam not a problem at all, sorry you were experiencing the issue, and if it was happening to me i would feel exactly the same way. I should to be honest be able to reproduce some portion of your setup to see if i can reproduce the issue, as it seems that you are not the only one with issues like this as this thread points out. it would be good to find a correlation as for most people using the plugin they don't see this. and it would be good to find a route cause. Ubiquiti routers have a way to perform broadcast discovery across VLAN's (bcast-relay) i use this currently, but as you say you were not. so i will work on this basis, i do have an Unraid server also on 10GB port, i can have a play on this and see if i can find the issue. just as a further check does your unraid server use jumbo frames anywhere on the link? as this could well be the cause when the broadcast is being sent, depending on configuration, should be an very easy one for me to test.

                Comment


                  #9
                  fuzzysb My MTU is set to 1500 on unRaid and Jumbo Frames was not enabled on my switch.

                  Comment


                    #10
                    ok leave it with me and i will test and feed back

                    Comment


                      #11
                      I have managed to recreate the issue, it seems the cause is when using network virtualization, the first broadcasts do not receive responses then the app uses the directed subnet broadcasts. this sends the broadcast directly to the local subnet and this is where the pings ont he router then get dropped whilst enumerating the replies.. I have tested by removing network virtualization from the the physical nic and the broadcast requests do not exhibit the same issue. With the newer beta version of the app i posted above discovery packets are now very infrequent anyway (once per hour) which would significantly improve the situation. I have pushed this version to be the default now so as soon as homeseer publish it everyone will get it. In the interim i can also recreate using Hyper-V vNics on my windows HS4 install as well as my HS3 Linux install. So i will see if i can figure out the cause of the timeout on a directed broadcast. when not using a vNic i don't get ping dropouts. which is very strange.

                      i am pondering changing the app to only do a broadcast on startup and then on demand using the config page should you introduce or remove lights. this would completely resolve on a day to day, but it is a workaround and not attributable to root cause. it is very strange as the packet i send is 64 bytes only and then i bind to the port waiting for responses for the length of time specified in the timeout value. its this bind which seems to cause havoc on the router when using a vNIC which is very strange as the responses returned are a max or 256 bytes.

                      I will continue investigating the root cause and see if there is a better way to work around it.

                      Comment

                      Working...
                      X