HS will not upgrade after the application was started as a service

[John245]

Awesome! I'll sleep better now - it's getting late here

Sleep well.

Know that your help is much appreciated.

And know that I test on my QA/Staging environment so if things does not go well I just reinstall this environment.

Will deploy it tonight to my production environment.

---
John

[avpman]

Hi,

I followed these instructions very carefully, including creating the systemd start up script. I manually ran the hsstop.sh to test it. It gets stuck trying (I assume) to stop the upbSPUD plugin, see screenshot. HS does stop responding after first running the hsstop.sh script until the "HSConsole -log" runs again on line 2304 of the output the it is running again. Double checked all my paths are correct for my system. Debian 9

Any help would be appreciated.

I cloned my test HS4-Linux (on rpi4) system so I could run some tests. I was successful in updating from 4.2.0.0 to beta 4.2.0.5, and back again. Ran the cycle three times to be sure.

I then substituted in @avpman's unit file, and things went sideways fast. After some thought it came back that, under systemd, default behavior is as soon as the main process terminates, systemd immediately clobbers all child processes. So when the HSConsole process ends, the update.sh script gets clobbered before it can do it's thing. There are some additional directives needed in the unit file to make this work as we want.

Below is my unit file (homeseer.service), with "/usr/local/HomeSeer" as the HS root. After editing files in /etc/systemd/system, you want to run

sudo systemctl daemon-reload

to prevent systemd from bellyaching about changed unit files.

[Unit]
Description=HomeSeer Home Automation
Documentation=https://homeseer.com/support-home/
After=network-online.target remote-fs.target time-sync.target
Before=multi-user.target

[Service]
WorkingDirectory=/usr/local/HomeSeer
ExecStart=/usr/bin/mono HSConsole.exe --log
SyslogIdentifier=HSConsole
StandardOutput=null
Restart=on-failure
RestartSec=30
KillMode=none
TimeoutStopSec=300
ExecStop=/usr/local/HomeSeer/hsstop.sh

[Install]
WantedBy=multi-user.target

Here is the hsstop.sh, which lives in the HS root. Presume this is similar to @avpman's "stop_homeseer.sh". Remember to make this executable.

#!/bin/bash
# hsstop.sh - stop the HS application
# supports: systemd service shutdown

# import login credentials used to login to web server
# these are ignored if password not required
inifile=$(dirname $0)/Config/$(basename $0 .sh).ini
login=
test -r $inifile && . $inifile

# extract web server port from settings.ini
hsroot=$(dirname $0) # where this script lives
webport=$(awk -F= '\
{
gsub("\015", "") # remove CR character
if ($1 == "gWebSvrPort") print $2
}
' $hsroot/Config/settings.ini)

# send application shutdown command
for i in $(seq 1 5)
do
curl -f -s -o /dev/null ${login:+-u} $login --data 'ConfirmShutdownhs=Yes' "http://localhost:$webport/LinuxTools"
rc=$?
test $rc -eq 0 && break
curl -f -s -o /dev/null ${login:+-u} $login --data 'action=shutdown_hs' "http://localhost:$webport/system.html"
rc=$?
test $rc -eq 0 && break
sleep 2
done

killmain()
{
test -n "$MAINPID" && kill -0 $MAINPID && kill $MAINPID
}

trap killmain EXIT

# if curl cmd unsuccessful, terminate main process
test $rc -ne 0 && killmain

# wait until all HomeSeer mono processes terminate, with timeout
maxwait=300
polltime=5
mono=$(which mono) || exit
for (( t=0; t<$maxwait; t+=$polltime ))
do
pgrep -afi $mono.'*'\(HSConsole\|HSPI_\) || break
sleep $polltime
done

[dbrunt]

Hi,

I followed these instructions very carefully, including creating the systemd start up script. I manually ran the hsstop.sh to test it. It gets stuck trying (I assume) to stop the upbSPUD plugin, see screenshot. HS does stop responding after first running the hsstop.sh script until the "HSConsole -log" runs again on line 2304 of the output the it is running again. Double checked all my paths are correct for my system. Debian 9

Any help would be appreciated.

Back when I ran more plug-ins, one of them would not terminate in a timely fashion when HS asked it too. I think it might have been Insteon but not sure now. The issue lies in the plug-in - if it never terminates then definitely needs fixing; if slow then need to adjust the waittime but 300 seconds should be way more than adequate. I suppose the script could be adjusted to kill -9 the stubborn mono process after the graceful timeout expires...

[zwolfpack]

Hi,

I followed these instructions very carefully, including creating the systemd start up script. I manually ran the hsstop.sh to test it. It gets stuck trying (I assume) to stop the upbSPUD plugin, see screenshot. HS does stop responding after first running the hsstop.sh script until the "HSConsole -log" runs again on line 2304 of the output the it is running again. Double checked all my paths are correct for my system. Debian 9

Any help would be appreciated.

Did you mean to include a screen shot? I don't see it.

When testing the hsstop.sh script, recommend starting HSConsole manually (via the ./go command) rather than as a service. That way eliminate possibility of intervention by systemd.

[avpman]

"...recommend starting HSConsole manually (via the ./go command) rather than as a service. That way eliminate possibility of intervention by systems."

But isn't the purpose of the script? To stop HS when it is run as a service? Screen shot below:

Did you mean to include a screen shot? I don't see it.

When testing the hsstop.sh script, recommend starting HSConsole manually (via the ./go command) rather than as a service. That way eliminate possibility of intervention by systemd.

[zwolfpack]

It looks like systemd is restarting the HSConsole process? What happens if you start HSConsole manually instead? Does the UPBSpud plugin eventually exit?

[zwolfpack]

But isn't the purpose of the script? To stop HS when it is run as a service?

Yes, of course. But for testing, I'm suggesting running standalone to gauge just it's effect, without intervention by other "forces" ...

[avpman]

Similar result. The UPBspud.exe won't terminate. Eventually the hsstop script exits. However, since I started HS from ./go and not the service HS doesn't restart. This is an old plugin, unlikely the dev will fix anything. But it's critical to my system. Did someone mention changing the kill command from -0 to -9? Or modify the script to use -0 in normal cases and -9 if/when a process won't kill. (Beyond my capabilities.)

Any further suggestions or help would be appreciated. Thanks!

Yes, of course. But for testing, I'm suggesting running standalone to gauge just it's effect, without intervention by other "forces" ...

[zwolfpack]

avpman --

hsstop.sh doesn't itself issue any kill signals except in abnormal situations. I'll have to think about this a bit.

However it may be that this particular issue won't affect your ability to perform an update. Reason for this is that the HS script that performs the update (update.sh) now only checks for a running HSConsole.exe before proceeding with the install. It ignores running plugins.

The primary "fix" for the failure to update is the addition to the unit file of the non-standard "KillMode". Without this, systemd kills the running update.sh when it detects that the main process HSConsole.exe has exited. This because update.sh is a "child" of HSConsole.exe. Under default KillMode, systemd kills all child processes once the main process terminates.

hsstop.sh comes into play when stopping the service. With the non-standard KillMode, systemd doesn't kill the main process, so its the responsibility of hsstop.sh to do that. All this is necessary because HSConsole.exe doesn't handle kill signals as a server application normally would. A proper server process would catch the TERM signal, perform orderly shutdown and exit. HSConsole.exe doesn't appear to implement signal handling, so receipt of the signal generates an exception and the program immediately dies.

What I'll recommend for now is to implement the unit file and script as presented and try running an update. Also test system halt/reboot. My concern with the latter is that hsstop.sh may delay shutdown due to the running UPBSpud process.

Note that in hsstop.sh, near the end is a setting "maxwait=300". This is the time (5 minutes) that the script will wait for all the plugins to exit. You may try cutting this down if system shutdown drags on.