Announcement

Collapse
No announcement yet.

HS will not upgrade after the application was started as a service

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • zwolfpack
    replied
    avpman --

    hsstop.sh doesn't itself issue any kill signals except in abnormal situations. I'll have to think about this a bit.

    However it may be that this particular issue won't affect your ability to perform an update. Reason for this is that the HS script that performs the update (update.sh) now only checks for a running HSConsole.exe before proceeding with the install. It ignores running plugins.

    The primary "fix" for the failure to update is the addition to the unit file of the non-standard "KillMode". Without this, systemd kills the running update.sh when it detects that the main process HSConsole.exe has exited. This because update.sh is a "child" of HSConsole.exe. Under default KillMode, systemd kills all child processes once the main process terminates.

    hsstop.sh comes into play when stopping the service. With the non-standard KillMode, systemd doesn't kill the main process, so its the responsibility of hsstop.sh to do that. All this is necessary because HSConsole.exe doesn't handle kill signals as a server application normally would. A proper server process would catch the TERM signal, perform orderly shutdown and exit. HSConsole.exe doesn't appear to implement signal handling, so receipt of the signal generates an exception and the program immediately dies.

    What I'll recommend for now is to implement the unit file and script as presented and try running an update. Also test system halt/reboot. My concern with the latter is that hsstop.sh may delay shutdown due to the running UPBSpud process.

    Note that in hsstop.sh, near the end is a setting "maxwait=300". This is the time (5 minutes) that the script will wait for all the plugins to exit. You may try cutting this down if system shutdown drags on.

    Leave a comment:


  • avpman
    replied
    Similar result. The UPBspud.exe won't terminate. Eventually the hsstop script exits. However, since I started HS from ./go and not the service HS doesn't restart. This is an old plugin, unlikely the dev will fix anything. But it's critical to my system. Did someone mention changing the kill command from -0 to -9? Or modify the script to use -0 in normal cases and -9 if/when a process won't kill. (Beyond my capabilities.)

    Any further suggestions or help would be appreciated. Thanks!

    Click image for larger version  Name:	hsstop2.JPG Views:	0 Size:	116.9 KB ID:	1497790
    Originally posted by zwolfpack View Post

    Yes, of course. But for testing, I'm suggesting running standalone to gauge just it's effect, without intervention by other "forces" ...

    Leave a comment:


  • zwolfpack
    replied
    But isn't the purpose of the script? To stop HS when it is run as a service?
    Yes, of course. But for testing, I'm suggesting running standalone to gauge just it's effect, without intervention by other "forces" ...

    Leave a comment:


  • zwolfpack
    replied
    It looks like systemd is restarting the HSConsole process? What happens if you start HSConsole manually instead? Does the UPBSpud plugin eventually exit?

    Leave a comment:


  • avpman
    replied
    "...recommend starting HSConsole manually (via the ./go command) rather than as a service. That way eliminate possibility of intervention by systems."

    But isn't the purpose of the script? To stop HS when it is run as a service? Screen shot below:

    Click image for larger version  Name:	hsstop.JPG Views:	0 Size:	124.3 KB ID:	1497770

    Originally posted by zwolfpack View Post

    Did you mean to include a screen shot? I don't see it.

    When testing the hsstop.sh script, recommend starting HSConsole manually (via the ./go command) rather than as a service. That way eliminate possibility of intervention by systemd.

    Leave a comment:


  • zwolfpack
    replied
    Originally posted by avpman View Post
    Hi,

    I followed these instructions very carefully, including creating the systemd start up script. I manually ran the hsstop.sh to test it. It gets stuck trying (I assume) to stop the upbSPUD plugin, see screenshot. HS does stop responding after first running the hsstop.sh script until the "HSConsole -log" runs again on line 2304 of the output the it is running again. Double checked all my paths are correct for my system. Debian 9

    Any help would be appreciated.


    Did you mean to include a screen shot? I don't see it.

    When testing the hsstop.sh script, recommend starting HSConsole manually (via the ./go command) rather than as a service. That way eliminate possibility of intervention by systemd.

    Leave a comment:


  • dbrunt
    replied
    Originally posted by avpman View Post
    Hi,

    I followed these instructions very carefully, including creating the systemd start up script. I manually ran the hsstop.sh to test it. It gets stuck trying (I assume) to stop the upbSPUD plugin, see screenshot. HS does stop responding after first running the hsstop.sh script until the "HSConsole -log" runs again on line 2304 of the output the it is running again. Double checked all my paths are correct for my system. Debian 9

    Any help would be appreciated.
    Back when I ran more plug-ins, one of them would not terminate in a timely fashion when HS asked it too. I think it might have been Insteon but not sure now. The issue lies in the plug-in - if it never terminates then definitely needs fixing; if slow then need to adjust the waittime but 300 seconds should be way more than adequate. I suppose the script could be adjusted to kill -9 the stubborn mono process after the graceful timeout expires...

    Leave a comment:


  • avpman
    replied
    Hi,

    I followed these instructions very carefully, including creating the systemd start up script. I manually ran the hsstop.sh to test it. It gets stuck trying (I assume) to stop the upbSPUD plugin, see screenshot. HS does stop responding after first running the hsstop.sh script until the "HSConsole -log" runs again on line 2304 of the output the it is running again. Double checked all my paths are correct for my system. Debian 9

    Any help would be appreciated.

    Originally posted by zwolfpack View Post
    I cloned my test HS4-Linux (on rpi4) system so I could run some tests. I was successful in updating from 4.2.0.0 to beta 4.2.0.5, and back again. Ran the cycle three times to be sure.

    I then substituted in @avpman's unit file, and things went sideways fast. After some thought it came back that, under systemd, default behavior is as soon as the main process terminates, systemd immediately clobbers all child processes. So when the HSConsole process ends, the update.sh script gets clobbered before it can do it's thing. There are some additional directives needed in the unit file to make this work as we want.

    Below is my unit file (homeseer.service), with "/usr/local/HomeSeer" as the HS root. After editing files in /etc/systemd/system, you want to run
    Code:
    sudo systemctl daemon-reload
    to prevent systemd from bellyaching about changed unit files.

    Code:
    [Unit]
    Description=HomeSeer Home Automation
    Documentation=https://homeseer.com/support-home/
    After=network-online.target remote-fs.target time-sync.target
    Before=multi-user.target
    
    [Service]
    WorkingDirectory=/usr/local/HomeSeer
    ExecStart=/usr/bin/mono HSConsole.exe --log
    SyslogIdentifier=HSConsole
    StandardOutput=null
    Restart=on-failure
    RestartSec=30
    KillMode=none
    TimeoutStopSec=300
    ExecStop=/usr/local/HomeSeer/hsstop.sh
    
    [Install]
    WantedBy=multi-user.target
    Here is the hsstop.sh, which lives in the HS root. Presume this is similar to @avpman's "stop_homeseer.sh". Remember to make this executable.
    Code:
    #!/bin/bash
    # hsstop.sh - stop the HS application
    # supports: systemd service shutdown
    
    # import login credentials used to login to web server
    # these are ignored if password not required
    inifile=$(dirname $0)/Config/$(basename $0 .sh).ini
    login=
    test -r $inifile && . $inifile
    
    # extract web server port from settings.ini
    hsroot=$(dirname $0) # where this script lives
    webport=$(awk -F= '\
    {
    gsub("\015", "") # remove CR character
    if ($1 == "gWebSvrPort") print $2
    }
    ' $hsroot/Config/settings.ini)
    
    # send application shutdown command
    for i in $(seq 1 5)
    do
    curl -f -s -o /dev/null ${login:+-u} $login --data 'ConfirmShutdownhs=Yes' "http://localhost:$webport/LinuxTools"
    rc=$?
    test $rc -eq 0 && break
    curl -f -s -o /dev/null ${login:+-u} $login --data 'action=shutdown_hs' "http://localhost:$webport/system.html"
    rc=$?
    test $rc -eq 0 && break
    sleep 2
    done
    
    killmain()
    {
    test -n "$MAINPID" && kill -0 $MAINPID && kill $MAINPID
    }
    
    trap killmain EXIT
    
    # if curl cmd unsuccessful, terminate main process
    test $rc -ne 0 && killmain
    
    # wait until all HomeSeer mono processes terminate, with timeout
    maxwait=300
    polltime=5
    mono=$(which mono) || exit
    for (( t=0; t<$maxwait; t+=$polltime ))
    do
    pgrep -afi $mono.'*'\(HSConsole\|HSPI_\) || break
    sleep $polltime
    done

    Leave a comment:


  • John245
    replied
    Originally posted by zwolfpack View Post

    Awesome! I'll sleep better now - it's getting late here
    Sleep well.

    Know that your help is much appreciated.

    And know that I test on my QA/Staging environment so if things does not go well I just reinstall this environment.

    Will deploy it tonight to my production environment.

    ---
    John

    Leave a comment:


  • zwolfpack
    replied
    Originally posted by John245 View Post

    I found the issue.
    ...
    ---
    John
    Awesome! I'll sleep better now - it's getting late here

    Leave a comment:


  • John245
    replied
    Originally posted by zwolfpack View Post
    The item in red is due to having installed mono via 'mono-complete' rather than 'mono-devel'. (mono-devel is what is directed in the HS install documentation).

    In previous versions of the update.sh this would cause an issue. However recently HS took a suggestion from me and made a change which should ignore this.

    In your HS root, please verify that update.sh looks like this:
    Code:
    #!/bin/bash
    # first make sure HS is shut down
    while pgrep -af mono.'*'\(HSConsole\)
    do
    sleep 5
    done
    
    # extract the update files and reboot
    echo extracting $1
    tar xvf $1
    rm $1
    echo "Update complete"
    reboot
    xsp4.exe is a documentation server which has no use in this application. You can safely remove it via
    Code:
    sudo apt remove mono-xsp --auto-remove
    This begs the question though, as to why HomeSeer continues to run once the update is commanded. Can you check if the process id of the HSConsole process stays constant?
    I found the issue. Although I followed all directions the homeseer.service file did not change in Webmin. I deleted it from Webmin and did set-up a new service. This solved the issue.

    Thanks for all the help, much appreciated.

    This should be part of HomeSeer's standard instruction on how to install Linux.

    ---
    John

    Leave a comment:


  • zwolfpack
    replied
    The item in red is due to having installed mono via 'mono-complete' rather than 'mono-devel'. (mono-devel is what is directed in the HS install documentation).

    In previous versions of the update.sh this would cause an issue. However recently HS took a suggestion from me and made a change which should ignore this.

    In your HS root, please verify that update.sh looks like this:
    Code:
    #!/bin/bash
    # first make sure HS is shut down
    while pgrep -af mono.'*'\(HSConsole\)
    do
        sleep 5
    done
    
    # extract the update files and reboot
    echo extracting $1
    tar xvf $1
    rm $1
    echo "Update complete"
    reboot
    xsp4.exe is a documentation server which has no use in this application. You can safely remove it via
    Code:
    sudo apt remove mono-xsp --auto-remove
    This begs the question though, as to why HomeSeer continues to run once the update is commanded. Can you check if the process id of the HSConsole process stays constant?

    Leave a comment:


  • SteveW
    replied
    zwolfpack, excellent information! Thanks.

    rjh, to your questions in another thread: personally, since I run HS4 on a dedicated RPi 4, I don't care how the software is started; as a task or as a service, and I don't mind having the upgrade procedure reboot the system. Of course, others will have different needs and preferences.

    IMO, one long-time issue with HS for Linux is that the installation procedure is not thoroughly documented, and it assumes some specific Linux concepts and commands that may be ambiguous or unclear to non Linux experts. Going back to 2019 or earlier, there are various long threads with various posters adding their own sets of commands, and some have typos, are incomplete, or are unclear. To the poster (you know who you are) who may reply with the typical Linux arrogance that if you don't know what you are doing, then don't use Linux (or "RTFM"), that's unhelpful. Linux is for everyone, and helping users learn is part of the fun of it.

    Leave a comment:


  • John245
    replied
    Originally posted by zwolfpack View Post
    Ok, looks like we'll need to dig deeper. First of all, what OS are you running on?
    Code:
    more /etc/os-release
    Next, paste this line in a command window and let it run. It will output a list of the running mono processes once every five seconds. As written it will run 200 times (about 17 minutes); change the 200 near the end to run longer.
    Code:
    seq -f "echo pass %g; pgrep -af mono; sleep 5" 1 200 | sh
    On my system this shows
    Code:
    pass 1
    822 /usr/bin/mono HSConsole.exe --log
    893 /usr/bin/mono /opt/HomeSeer/HSPI_Sonos.exe
    905 /usr/bin/mono /opt/HomeSeer/HSPI_SDJHealth.exe
    921 /usr/bin/mono /opt/HomeSeer/HSPI_ZWave.exe
    945 /usr/bin/mono /opt/HomeSeer/HSPI_ZWaveGui.exe
    954 /usr/bin/mono /opt/HomeSeer/HSPI_BLLAN.exe
    pass 2
    822 /usr/bin/mono HSConsole.exe --log
    893 /usr/bin/mono /opt/HomeSeer/HSPI_Sonos.exe
    905 /usr/bin/mono /opt/HomeSeer/HSPI_SDJHealth.exe
    921 /usr/bin/mono /opt/HomeSeer/HSPI_ZWave.exe
    945 /usr/bin/mono /opt/HomeSeer/HSPI_ZWaveGui.exe
    954 /usr/bin/mono /opt/HomeSeer/HSPI_BLLAN.exe
    One line for the HSConsole, one for each running plugin. The number on the left is the process id.

    With this running, try the update and see if the output changes.
    OS:
    NAME="Ubuntu"
    VERSION="20.04.3 LTS (Focal Fossa)"
    ID=ubuntu
    ID_LIKE=debian
    PRETTY_NAME="Ubuntu 20.04.3 LTS"
    VERSION_ID="20.04"
    HOME_URL="https://www.ubuntu.com/"
    SUPPORT_URL="https://help.ubuntu.com/"
    BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
    PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-poli
    cy"
    VERSION_CODENAME=focal
    UBUNTU_CODENAME=focal

    Running processes:
    pass 66
    644 /usr/bin/mono /home/dunnenj/HomeSeer/HSConsole.exe --log
    727 /usr/bin/mono /usr/lib/mono/4.5/xsp4.exe --port 8084 --address 0.0.0.0 --appconfigdir /etc/xsp4 --nonstop
    1194 /usr/bin/mono /home/dunnenj/HomeSeer/HSPI_EasyTrigger.exe
    1211 /usr/bin/mono /home/dunnenj/HomeSeer/HSPI_Chromecast.exe
    1226 /usr/bin/mono /home/dunnenj/HomeSeer/HSPI_SKWARE_DEVICE_HISTORY.exe
    1267 /usr/bin/mono /home/dunnenj/HomeSeer/HSPI_ZWave.exe
    1302 /usr/bin/mono /home/dunnenj/HomeSeer/HSPI_ZWaveGui.exe
    1332 /usr/bin/mono /home/dunnenj/HomeSeer/HSPI_ULTRANETATMO3.exe
    1347 /usr/bin/mono /home/dunnenj/HomeSeer/HSPI_Unifi.exe
    1368 /usr/bin/mono /home/dunnenj/HomeSeer/HSPI_Worx.exe
    1385 /usr/bin/mono /home/dunnenj/HomeSeer/HSPI_Tuya.exe
    1404 /usr/bin/mono /home/dunnenj/HomeSeer/HSPI_SmartThings.exe
    1414 /usr/bin/mono /home/dunnenj/HomeSeer/HSPI_SmartMeter.exe
    1444 /usr/bin/mono /home/dunnenj/HomeSeer/HSPI_BLPlex.exe

    I see no changes before during or after the update. The big difference compared with your output is in red.

    ---
    John

    Leave a comment:


  • zwolfpack
    replied
    Ok, looks like we'll need to dig deeper. First of all, what OS are you running on?
    Code:
    more /etc/os-release
    Next, paste this line in a command window and let it run. It will output a list of the running mono processes once every five seconds. As written it will run 200 times (about 17 minutes); change the 200 near the end to run longer.
    Code:
    seq -f "echo pass %g; pgrep -af mono; sleep 5" 1 200 | sh
    On my system this shows
    Code:
    pass 1
    822 /usr/bin/mono HSConsole.exe --log
    893 /usr/bin/mono /opt/HomeSeer/HSPI_Sonos.exe
    905 /usr/bin/mono /opt/HomeSeer/HSPI_SDJHealth.exe
    921 /usr/bin/mono /opt/HomeSeer/HSPI_ZWave.exe
    945 /usr/bin/mono /opt/HomeSeer/HSPI_ZWaveGui.exe
    954 /usr/bin/mono /opt/HomeSeer/HSPI_BLLAN.exe
    pass 2
    822 /usr/bin/mono HSConsole.exe --log
    893 /usr/bin/mono /opt/HomeSeer/HSPI_Sonos.exe
    905 /usr/bin/mono /opt/HomeSeer/HSPI_SDJHealth.exe
    921 /usr/bin/mono /opt/HomeSeer/HSPI_ZWave.exe
    945 /usr/bin/mono /opt/HomeSeer/HSPI_ZWaveGui.exe
    954 /usr/bin/mono /opt/HomeSeer/HSPI_BLLAN.exe
    One line for the HSConsole, one for each running plugin. The number on the left is the process id.

    With this running, try the update and see if the output changes.

    Leave a comment:

Working...
X