Concerning updater message in log

[Kbevo]

When restarting HS3 I received the following message in the log:
Feb-16 10:52:21 AM Updater An item in the Updater is temporarily unavailable: The operation has timed out Item: 'http://modernwarp.com/HomeSeer3/incontrol_winstore_updater.txt'


What is this message, why did it get it, and how do I get rid of it? In this day of viruses and other mischief this is concerning. First, as mentioned, I dont recognize the domain. Second it is an http not https. Please, someone explain.

Thanks
Gary

[S-F]

This just means that the site where a developer hosts their plugin/s is unresponsive.

[Kbevo]

This just means that the site where a developer hosts their plugin/s is unresponsive.

Thanks for the info. I am glad to know I am not the only one. Are you comfortable with your HS server issuing calls to insecure sites?

Thanks
Gary

[mrhappy]

Thanks for the info. I am glad to know I am not the only one. Are you comfortable with your HS server issuing calls to insecure sites?

Thanks
Gary

It does a GET on the updater text file on the developers server and then pulls the icon and plugin details into your updater page so really there is relatively little it is doing. When you want to download the file then obviously it is just downloading the zip file from the developers server. I don't know an incredible amount web hosting but I'm not sure it executes anything and just does a GET on a little bit of text and parses it from there so I am not sure it could cause an issue, I don't actually think HTTPS is supported by the updater anyway.

HS have suggested (and Rich said a couple of weeks ago that he wanted to get it resolved soon albeit it has been on the cards since last year) they will have a developer portal and host all plugins (if I read it right) which at least gives you some assurance if the developers server goes down and/or it is hosted (hopefully) appropriately. I don't know when or if that will come.

The issue for me would be that as I host my own plugins (or I have a hosting company to do it rather) then I have probably an accurate list in the web logs of all of the people to have a HS installation because just by going in the updater then it will have done a GET to my server and be recorded. Providing someone has recently gone in the updater I am going to have a list of IP's to peoples servers among other data, that's obviously not good if you are untrustworthy (...I'm at least security cleared for my job!!) and/or people run their HS servers on port 80 with the default admin accounts. Then someone with access to that list could create absolute havoc, I've mentioned this before but I'd really like to see that door closed up myself.

[Kbevo]

Thanks much MrHappy for the education. That insight is helpful. I never thought much about it, but it comes as a surprise that the updater is doing downloads from developer servers (vs the HS servers).

Now that I understand that part, its one thing to do the "Gets" for the updater information when I explicitly request it, and something else for it to be doing that communication behind the scenes (e.g. during HS startup) and without my knowledge.

I agree about your "absolute havoc" statement and feel HS needs to get control of the security aspects of the updater downloads fast, even if it comes at an additional cost to the clients (IMO). Smarter folks than me can work out the "how" to that issue but the downloads should at the least have some measure of virus scans and be distributed via secure connection.

Gary

[zwolfpack]

The updater control file is https://updatercontrol.homeseer.com/...s3/updater.txt. Many of the plugins are hosted on the HomeSeer server, but there is also an option for the developer to host. These are indicated by the lines starting with "@". These individual control files are accessed each time that the Manage Plugins page is refreshed. The plugin zip file is only downloaded when doing an install/update.

The hosts that are accessed can be listed via
curl -s https://updatercontrol.homeseer.com/...s3/updater.txt | awk -F/ '/^@/ { print $1 FS $2 FS $3 }' | sort -u

which today yields
@http://automatedhomeonline.com
@http://bbmessenger.hobby-site.com
@http://bladeplugins.no-ip.org
@http://donmor.ca
@http://download.casapiedrasoftware.info
@http://download.dedroog.com
@http://gaintechsolutions.com
@http://home.avglabs.net
@http://home.indigozest.net
@http://homeseer.azurewebsites.net
@http://homeseer.du-pre.com
@http://homeseer.dyndns-remote.com:8765
@http://hsupdater.exdivio.com
@http://kazteel.com
@http://kingfetty.com
@http://mcssprinklers.com
@http://mcsSprinklers.com
@http://meilibox.com
@http://modernwarp.com
@http://s652164905.websitehome.co.uk
@http://srv.rusnes.no
@http://tasker.ithemmet.se
@http://www.bobshome.net
@http://www.bobsplace.com
@http://www.domogeek.ca
@http://www.highpeak.co.za
@http://www.kazteel.com
@http://www.myautomatedhome.net
@http://www.passion4automation.be
@http://www.sandler.org
@https://dl.dropbox.com
@https://dl.dropboxusercontent.com
@https://skware1.blob.core.windows.net

There are some https:// entries indicating the updater is capable of supporting that.

[Kbevo]

The updater control file is https://updatercontrol.homeseer.com/...s3/updater.txt. Many of the plugins are hosted on the HomeSeer server, but there is also an option for the developer to host. These are indicated by the lines starting with "@". These individual control files are accessed each time that the Manage Plugins page is refreshed. The plugin zip file is only downloaded when doing an install/update.

Thanks zwolfpack for the feedback. That explains why I received the updater message during what I thought was HS3 startup. Apparently startup had already completed. I went into the Interfaces & Plugins page as soon as the Browser page loaded and that must have been when the updater started doing its thing.

In your opinion, is it unreasonable for HS to require the developers to support https, or for HS to host all plugins?

Gary

[zwolfpack]

Unreasonable - no ... unlikely - yes ...

[Michael McSharry]

It is my understanding that when a hosting site is used by 3rd party which is likely typical then to support https a 'SSL' certificate needs to be purchased. This for me is more onerous than the cost of the domain and hosting service.

[Krumpy]

Wonderful, another information security concern. But, then the risk is pushed back to the author of the plugin. I suppose that the “liability” is owned by the author of the plugin. The plugin author has vested interest in keeping it safe. HTTPS is a must to ensure that it is being hosted by the true intended source.

Interesting...

[Kbevo]

Clearly I do not understand all the issues here.

It is my understanding that when a hosting site is used by 3rd party which is likely typical then to support https a 'SSL' certificate needs to be purchased. This for me is more onerous than the cost of the domain and hosting service.

If cost is high, then it is unreasonable for the plugin developers to be required to go to https. To do so, I suspect would cause many developers to back away from distributing their handy-work entirely. That would hurt innovation, variety and function significantly.

Well we can take that position, but the client that gets hit is the one that suffers through the damage and cleanup. Not at all pleasant.

What is wrong with HS hosting ALL plugins that go through the updater? Even at increased cost?

Gary

[Krumpy]

[zwolfpack]

What is wrong with HS hosting ALL plugins that go through the updater? Even at increased cost?

I believe this is a developer's option. Allowing developers to host their own allows them the flexibility to instantly post updates. A classic convenience vs. security tradeoff. My observation has been than in HomeSeer's case, convenience usually wins these...

It should be noted that, per the updater.txt posted above, all zips (HomeSeer and 3rd party) are served unencrypted. So perhaps SSL fetch of zips isn't supported after all. (Some of the 3rd party updater.txt's are https, but within each of those, the zip URL is http).

To me, a bigger concern is that each one of the 3rd party servers listed above potentially has within its access logs a complete listing of every HS3 installation that has accessed the updater. This info could be used to launch a directed attack at any vulnerabilities in the HS3 application. Something to think about before opening up any outward facing ports...

[Kbevo]

I believe this is a developer's option. Allowing developers to host their own allows them the flexibility to instantly post updates. A classic convenience vs. security tradeoff. My observation has been than in HomeSeer's case, convenience usually wins these...

Perhaps the (paying) clients should have a say?

It should be noted that, per the updater.txt posted above, all zips (HomeSeer and 3rd party) are served unencrypted. So perhaps SSL fetch of zips isn't supported after all. (Some of the 3rd party updater.txt's are https, but within each of those, the zip URL is http).

From a novice observation, the updater payload appears to be downloaded, unziped, and perhaps even executed before the client has a chance to run any type of virus scan. Makes one rethink the updater altogether.

To me, a bigger concern is that each one of the 3rd party servers listed above potentially has within its access logs a complete listing of every HS3 installation that has accessed the updater. This info could be used to launch a directed attack at any vulnerabilities in the HS3 application. Something to think about before opening up any outward facing ports...

Whats the fix?

Gary

[zwolfpack]

Perhaps the (paying) clients should have a say?

Paying clients always have a say. Already paid ones perhaps not so much (unless enough of them raise a ruckus) ...

From a novice observation, the updater payload appears to be downloaded, unzipped, and perhaps even executed before the client has a chance to run any type of virus scan. Makes one rethink the updater altogether.

Probably overkill, but if you want to maintain your own library of plugin installers, the script below will do just that.

Not that I put a lot of faith in it, but Norton AV gives them all a clean bill of health!

Whats the fix?

That's a tough one - like I said, think twice, and then twice again before opening any outward facing ports. I use MyHS, which I doubt has been sufficiently vetted for vulnerabilities. However my assessment is that it's lower risk (and of course less work) than blazing my own solution.

#!/bin/bash
# script to download plugin installer archives

dest=zips

# download updater file if it has changed on the server
wget -N -nv https://homeseer.com/updates3/updater.txt || exit 1

# expand 3p (non-homeseer hosted) update info
# resulting file is same format as updater.txt
echo "Gathering 3p info..."
cat updater.txt | dos2unix | \
awk '/^@/ {
    sub("^@", "", $1)
    print "wget -nv --tries=3 " $1 " -O - && echo || echo unable to retrieve: " $1 " >&2"
}
' | sh | dos2unix > updater3p.txt

# extract URL of each plugin archive file and download it
echo "Downloading zip archives..."
(
    echo mkdir -p $dest
    echo cd $dest
    cat updater.txt updater3p.txt | awk -F, '\
    {
        if ($1 !~ "^\047" && NF >= 9) {
            if ($8 !~ "^/") $8 = "/"$8
            url=$7$8"/"$9
            gsub("[ \t]","",url)
            print "wget -N -nv --tries=3 " url
        }
    }
    '
) | sh

Run under Linux, or Cygwin on Windows.
Initial run will download all plugin zips (275 as of today); subsequent runs will download only updated entries.