Excessive CPU and Excessive MQTT Topics

[mwolter]

[simplextech]

High CPU consumption here as well. I installed from plugins list and enabled... that's it. I never got around to configuring and I noticed my CPU usage for HS3 was really high from what it had been. Looking at the processes it was the mqtt plugin using the CPU. This boggles me as it was not configured and not doing anything.

Stats: HS3 (trial)
OS: Windows (may move to Linux later if I don't really need some plugins)

Any reason why it would be cycling and consuming CPU when nothing is configured? Is it doing a loop waiting to be configured ie searching for something that doesn't exist yet?

[Pete]

1 - Here install Homeseer in the root directory

/HomeSeer

2 - In Linux create my two homeseer users and give them sudo rights. These match the
two admin users in Homeseer (change the password to Pi if configured).

3 - chmod -R 777 or 775 /HomeSeer or /usr/local/HomeSeer or chmod -R a+rwX /HomeSeer or /usr/local/HomeSeer

[Michael McSharry]

Pete, can you provide more explicit direction about defining the users. I tried other suggestions above without luck. Also installed WebMin, but don't know how it can help me. I did a wireshark capture and see the that the HS server is responding to the HTTP request, but it is not returning anything and the browser eventually times out.

[Michael McSharry]

High CPU consumption here as well. I installed from plugins list and enabled... that's it. I never got around to configuring and I noticed my CPU usage for HS3 was really high from what it had been. Looking at the processes it was the mqtt plugin using the CPU. This boggles me as it was not configured and not doing anything.

Stats: HS3 (trial)
OS: Windows (may move to Linux later if I don't really need some plugins)

Any reason why it would be cycling and consuming CPU when nothing is configured? Is it doing a loop waiting to be configured ie searching for something that doesn't exist yet?

Also a mystery to me.
There is default startup initialization to enumerate all the HS devices. It will write to SQLite database (\Data\mcsMQTT\mcsMQTT.db). It will also open a debug file in the same folder mcsMQTT_Debug.txt. There will be some progress info in that file.

After this the only thing the plugin does is monitor the connection to HS. When the MQTT broker is defined then other things happen by collecting what it see's from the broker.

After Topics are accepted then it establishes a callback to listen to HS Events on Device changes. Before this the callbacks are not registered so will not be occurring.

Take a look at the \Data\mcsMQTT folder and see what is there. Look into mcsMQTT_Debug.txt. If something is there then it may mean something to me.

[simplextech]

Also a mystery to me.
There is default startup initialization to enumerate all the HS devices. It will write to SQLite database (\Data\mcsMQTT\mcsMQTT.db). It will also open a debug file in the same folder mcsMQTT_Debug.txt. There will be some progress info in that file.

After this the only thing the plugin does is monitor the connection to HS. When the MQTT broker is defined then other things happen by collecting what it see's from the broker.

After Topics are accepted then it establishes a callback to listen to HS Events on Device changes. Before this the callbacks are not registered so will not be occurring.

Take a look at the \Data\mcsMQTT folder and see what is there. Look into mcsMQTT_Debug.txt. If something is there then it may mean something to me.

Nothing much in the debug file. Here's the full contents:

4/6/2018 7:09:38 PM	0	| mcsMQTT Running at C:\Program Files (x86)\HomeSeer HS3, HS is at C:\Program Files (x86)\HomeSeer HS3 
4/6/2018 7:09:38 PM	82	| mcsMQTT InitHW ComputerName= john-pc, IOEnabled=False 
4/6/2018 7:09:38 PM	225	| mcsMQTT Debug InitHW Database Ready 
4/6/2018 7:09:38 PM	227	| mcsMQTT Debug Receive Ready 
4/6/2018 7:09:38 PM	228	| mcsMQTT Debug Trigger Ready 
4/6/2018 7:09:38 PM	255	| Spawning MQTT Threads  
4/6/2018 7:09:38 PM	255	| mcsMQTT Debug MQTT Ready 
4/6/2018 7:09:38 PM	255	| HW Init Complete  
4/6/2018 7:09:38 PM	256	| Background Init Started  
4/6/2018 7:09:38 PM	260	| Background Init Received  
4/6/2018 7:09:48 PM	9665	| Background Send  
4/6/2018 7:09:48 PM	9666	| Background Init Filters - Background Complete  
4/6/2018 7:10:08 PM	30255	| HS Request Name  
4/6/2018 7:10:38 PM	60259	| HS Request Name  
4/6/2018 7:11:08 PM	90281	| HS Request Name  
4/6/2018 7:11:38 PM	120294	| HS Request Name  
4/6/2018 7:12:08 PM	150311	| HS Request Name  
4/6/2018 7:12:38 PM	180317	| HS Request Name  
4/6/2018 7:13:08 PM	210333	| HS Request Name  
4/6/2018 7:13:38 PM	240334	| HS Request Name  
4/6/2018 7:14:08 PM	270347	| HS Request Name  
4/6/2018 7:14:38 PM	300350	| HS Request Name  
4/6/2018 8:42:48 PM	5589523	| Shutdown IO  
4/6/2018 12:50:37 PM	617	| mcsMQTT Debug Trigger Ready 
4/6/2018 12:50:37 PM	645	| Spawning MQTT Threads  
4/6/2018 12:50:37 PM	646	| mcsMQTT Debug MQTT Ready 
4/6/2018 12:50:37 PM	646	| HW Init Complete  
4/6/2018 12:50:37 PM	647	| Background Init Started  
4/6/2018 12:50:37 PM	648	| Background Init Received  
4/6/2018 12:50:37 PM	752	| HS Request InterfaceStatus  
4/6/2018 12:50:42 PM	5542	| HS Request InterfaceStatus  
4/6/2018 12:50:45 PM	8507	| Trying to remove expired web page 1 Collection was modified; enumeration operation may not execute. 0 Collection was modified; enumeration operation may not execute.
4/6/2018 12:50:45 PM	8534	| AddToMQTTSend Wait  
4/6/2018 12:50:45 PM	8578	| AddToMQTTSend Resume 4  
4/6/2018 12:50:52 PM	16040	| Background Send  
4/6/2018 12:50:52 PM	16041	| Background Init Filters - Background Complete  
4/6/2018 12:51:07 PM	30649	| HS Request Name  
4/6/2018 12:51:37 PM	60667	| HS Request Name  
4/6/2018 12:52:07 PM	90677	| HS Request Name  
4/6/2018 12:52:37 PM	120693	| HS Request Name  
4/6/2018 12:53:07 PM	150702	| HS Request Name  
4/6/2018 12:53:37 PM	180719	| HS Request Name  
4/6/2018 12:54:07 PM	210736	| HS Request Name  
4/6/2018 12:54:37 PM	240752	| HS Request Name  
4/6/2018 1:02:42 PM	725658	| Removed exppired web page  
4/6/2018 7:04:58 PM	22461976	| Shutdown IO

Not much in the data directory either. I'll have to setup something to post images as I don't currently use a image hosting.

[mwolter]

Most likely it's a permission issue. I'm assuming HS installed in /usr/local/HomeSeer, if not adjust the directory below.



Did you install Webmin? All of the commands below can be done through Webmin, but it will take some time to learn it. Command to install Webmin is below.

echo "deb http://download.webmin.com/download/repository sarge contrib" | sudo tee -a /etc/apt/sources.list; wget http://www.webmin.com/jcameron-key.asc; sudo apt-key add jcameron-key.asc; sudo apt-get update; sudo apt-get install webmin

Follow the steps below if you would like to use the console.



Set the root password

sudo passwd root

Change the directory permissions

sudo chown -R pi:root /usr/local/HomeSeer

Then use the ./go script to try it again.



if you cant access it through the web page. Use the top command to verify HSConsole is running.



If it's not running, use the following commands to view the startup logs.

sudo nano /usr/local/HomeSeer/Logs/Startup.log

or

sudo nano /usr/local/HomeSeer/Logs/PreviousStartup.log

Michael, did you try this?

[Pete]

@Michael,

I do not think it is a user issue or rights issue relating to using the homeseer gui.

When you start it manually in a terminal window you should see a bunch of dialog including any web server issues.

Are you using Homeseer Lite, Pro or Standard?

Are you using a static IP or DHCP?

SSH to you new box and type the following:

sudo netstat -plnt

Here did it for my HS3 Pro box (note that I have much more than this)

tcp 0 0 0.0.0.0:9999 0.0.0.0:* LISTEN 2473/mono

Here Homeseer is on port 9999. Note the Homeseer stuff shows up as mono.

Shut down Homeseer via the GUI and check the /HomeSeer/Config/settings.ini file

cat /HomeSeer/Config/settings.ini

You should see this in the middle or so of the file. If you do not insert the

gLastRunVersion=3.0.0.423
gWebSvrPort=9999

Note that my homeseer server port is set to 9999

Change this port to some other port for Homeseer use like 8088 or 82 or whatever you like. Restart Homeseer and you should be OK.

Not sure how you are starting homeseer automagically. There are a couple of ways to do this. Simplest is to add the start to the end of /etc/rc.local or a cron job.

exec sh /HomeSeer/autostart_hs

You can do this via ssh / nano or webmin.

Typically here create two users for homeseer use and linux use via ssh

1 - adduser homeseer
2 - answer all of the questions or just hit return until final question.
3 - add the homeseer user to sudo
usermod -aG sudo homeseer

I replicate the two above users (homeseer and personal name) in the Homeseer configuration name and password section and make them both admin. I also disable guest.

Webmin lets you totally manage your RPi with a web interface. It is a bit slow running on the RPi.

[Michael McSharry]

Yes I did. Same behavior. That is when I went to Wireshark to look a litter deeper and confirmed the HS server has the port that it claims during startup, but does not return any HTML. Only TCP level acknowledge.

What made matters worse is that the RPi I have Diet Pi installed with HS has also stopped working in the same manner. I have rebooted my router. I have tried a direct cable between Rpi and Windows laptop where the browser exits. Also tried Ipad browser.

Because of this common mode failure I just though that HS is calling home and when it finds two instances it stops the HTML which in essence make it unusable. To test it I powered down the new RPi that was just setup and my DietPi one comes back to life. I did did the other so now the new Mono install is working with HS.

That is the good news. The other I do not know is good or not, but when I bring up the mcsMQTT setup page I get just the HS header and otherwise blank. I will play a little and see what part of the setup page is causing issues with this Mono version.

[mwolter]

[Pete]

Do you see any mono errors in the HS logs?

Guessing you installed mono-complete and mono-vbnc eh?

I have myhomeseer stuff off by default and have disabled the Homeseer calling home script.

I get just the HS header and otherwise blank.

Here do not see this at all with: (see the HS header and rest of stuff)

[ATTACH]68009[/ATTACH]

Beta Update 3.0.0.423 on Homeseer Lite (zee2 OS) or Homeseer Pro running on

ARM - Pine64 2Gb Ubuntu 16.04 64 bit ==> Homeseer lite
Mono JIT compiler version 5.10.1.20

Intel - iSeries - Ubuntu 16.04 64 bit ==> Homeseer Pro
Mono JIT compiler version 5.10.1.20

I have another RPi and a test dual core AMD Homeseer server that I use for testing Homeseer 3 lite on. Will give your plugin a try on the RPi.

CPU Utilization is fine with the plugin here.

[ATTACH]68007[/ATTACH]

[Michael McSharry]

Thanks Pete, but the problem was two instances of HS which were running as trial. It appears they call home and more than one results in both not serving pages.

The problem with earlier Mono is the following:

        Dim oSortDictionary As Object

        If sDirection = "" Then
                oSortDictionary = New Generic.SortedDictionary(Of String, MqttReport)
        Else
                oSortDictionary = New Generic.SortedDictionary(Of String, MqttReport)(New ReverseSortComparer)
        End If

This version of Mono was not able to manage a generic object for the sorted dictionary during run time. If I define the type explicitly rather than using Object during compile time then all is good. What it means is that I need dedicated procedures for building a ascending and descending sort table. An extra page of code, but now it works.

I also have no issues with QOS being the defaults. The QOS seems more like a broker issue rather than a plugin issue. I did not test mosquitto on the newly built RPi with earlier Mono. I continued to test with mosquitto on Rpi with Stretch and latest Mono.

[Pete]

Ahh...

Good news Michael.

Here running Mosquitto / Node Red on 3-4 RPi2's which I recently updated to Stretch.

[mwolter]

Remotely upgraded the PI and it appears to be working. QOS exact is still having issues, but it doesn't crash the plugin, just disconnects from the broker. I'll work on the broker as soon as I return home.

Thanks for your hard work.

[Michael McSharry]

I observe now that my test setup was using At Most as the QOS. I thought it defaulted to Exactly, but I guess the default was changed somewhere along the way. Likely when I made it a user setting. When I change it to Exactly I also get broker offline with the older Mono.

I do not think I can do anything about this as all I do is pass 0, 1 or 2 as a parameter to the M2Mqtt.Net.dll. This dll was written in C# and I do not want to modify it.

This means that you do not need to play with your broker, but use one of the less demanding QOS values.