Z-Wave Interface Backup Script

[zwolfpack]

I've written a Perl script to automate backup of the Z-Wave interface. As you know, this is normally done thru the HS3 web interface: Plug-ins->Z-Wave->Controller Management. Under Z-Wave Interfaces, expand an interface, then under Actions, select "Back Up this interface", change the filename if desired, then hit START.

The script performs essentially the same actions:
1) a GET to retrieve the form and element names
2) a POST to set the action to "Back Up this interface"
3) a GET to retrieve the default filename and submit button properties
4) a POST to submit the backup action

The script takes one parameter, the HS3 hostname. This defaults to localhost, so if run on the HS3 machine it can be omitted.

You will need to install Perl in order to use this...

Linux ==============================
Perl is installed by default in most Linux distributions. On the Raspberry Pi, some additional modules are required. At the shell prompt enter:

sudo apt-get update
sudo apt-get -y install libwww-perl

Windows =================================
On Windows, ActivePerl, Strawberry Perl and Cygwin are options.
http://www.activestate.com/activeperl/downloads
http://strawberryperl.com/
http://cygwin.com/

I'll recommend ActivePerl since it's default install is smallest. Download & run the installer appropriate for your architecture (32 or 64 bit). Select "typical" install and take all the defaults.

=============================
I developed this on the Raspberry Pi (a z-net and another one). Initially I was thinking of setting up a cron job to run the backup. However I needed to verify it would run on windows,so I tried it under Cygwin on my desktop box, then I installed ActivePerl on my Hometroller. Then I set up this event, which I think is the cleanest way:



Caveats:
I only have one interface so I can't test a multiple interface case. However it was written it with that in mind, and from the way the page is structured I think it has a fair chance of working.

EDIT 1: added revision 'r1' (2017-01-28). Based on new info, the backup file name in multiple interface installations can be problematic. This update attempts to address this.
EDIT 2: replaced previous revisions with 'r2' (2017-01-31). adds action => updatetime which seems to be required by the page.
EDIT 3: revision 'r3' (2017-03-03) adds basic authorization capability. Username & password can be hardcoded into the file or provided on command line via -l username/password.
EDIT 4: Improved procedure for Linux install.
EDIT 5: revision 'r4' (2017-07-21) has improved retry mechanism needed in multi-interface installations.
EDIT 6: Updated screenshot to show purge of old files.
EDIT 7: revision 'r5' (2018-04-05) fix seems to avoid zwave plugin losing its mind and pegging the CPU 10-30 minutes after backup.
EDIT 8: revision 'r6' (2018-12-12) fix for issue backing up Raspberry Pi 'internal' interface -- simplify file name generation.

zwbkup.zip

[lveatch]

Sweet!! Thank you. I've been meaning to do this for a year.

Confirmed to work on windows 8.1 pro with ActiveState Perl 5.16.3 build 1603. I too only have a single z-wave controller.

Nice work!

[randy]

I've written a Perl script to automate backup of the Z-Wave interface. As you know, this is normally done thru the HS3 web interface: Plug-ins->Z-Wave->Controller Management. Under Z-Wave Interfaces, expand an interface, then under Actions, select "Back Up this interface", change the filename if desired, then hit START.

The script performs essentially the same actions:
1) a GET to retrieve the form and element names
2) a POST to set the action to "Back Up this interface"
3) a GET to retrieve the default filename and submit button properties
4) a POST to submit the backup action

The script takes one parameter, the HS3 hostname. This defaults to localhost, so if run on the HS3 machine it can be omitted.

You will need to install Perl in order to use this...

Linux ==============================
Perl is installed by default in most Linux distributions. On the Raspberry Pi, some additional modules are required. At the shell prompt enter:

sudo cpan

The first time that CPAN is run, it will offer to perform automatic automatic configuration. Accept this.
The prompt cpan[1]> should appear -- enter:

install Bundle::CPAN

It will crank for awhile. Accept the default for any requested inputs.

When the cpan the prompt returns, enter exit or ^D. You should be ready to go.

Windows =================================
On Windows, ActivePerl, Strawberry Perl and Cygwin are options.
http://www.activestate.com/activeperl/downloads
http://strawberryperl.com/
http://cygwin.com/

I'll recommend ActivePerl since it's default install is smallest. Download & run the installer appropriate for your architecture (32 or 64 bit). Select "typical" install and take all the defaults.

=============================
I developed this on the Raspberry Pi (a z-net and another one). Initially I was thinking of setting up a cron job to run the backup. However I needed to verify it would run on windows,so I tried it under Cygwin on my desktop box, then I installed ActivePerl on my Hometroller. Then I set up this event, which I think is the cleanest way:

[ATTACH]59137[/ATTACH]

Caveats:
I only have one interface so I can't test a multiple interface case. However it was written it with that in mind, and from the way the page is structured I think it has a fair chance of working.

Thank you! This is the one backup that depends on my remembering - not a good thing
I have three interfaces, so I will give it a go and let you know.

The only problem I have now is that the first interface I backup sets the default file name preamble name for all of the interfaces. This is a HS bug. Here I backed up the Shed interface first, so the garage backup starts with "ZNet_Shed". We'll see if the script behaves the same.

[zwolfpack]

The only problem I have now is that the first interface I backup sets the default file name preamble name for all of the interfaces. This is a HS bug. Here I backed up the Shed interface first, so the garage backup starts with "ZNet_Shed". We'll see if the script behaves the same.

Hi Randy, I think this will be an issue. I noticed that the timestamp in the provided filename only gets updated the 1st time the form is called after a plugin restart - so I substitute in the current time. Factoring in the info you provided though, looks like the bug goes deeper. I'm now thinking that the script should just build the filename from scratch.

To this end, can you confirm/deny if the list of interfaces in the 'Z-Wave Networks and Options' section appears in the same order as that in the 'Z-Wave Interfaces' section? If the order always matches, it greatly eases getting the friendly names and Home ID's to use in building the filenames.

[randy]

Hi Randy, I think this will be an issue. I noticed that the timestamp in the provided filename only gets updated the 1st time the form is called after a plugin restart - so I substitute in the current time. Factoring in the info you provided though, looks like the bug goes deeper. I'm now thinking that the script should just build the filename from scratch.

To this end, can you confirm/deny if the list of interfaces in the 'Z-Wave Networks and Options' section appears in the same order as that in the 'Z-Wave Interfaces' section? If the order always matches, it greatly eases getting the friendly names and Home ID's to use in building the filenames.

Also, we'll need a table of the Interface Model abbreviations that appear at the beginning of the filename (ZNet_...) for example. Here's what I'll propose; any suggestions/corrections welcomed. (3rd column is the filename prefix for each interface model).

    0  => "None",        # None,
    1  => "ZNet",        # HomeSeer Z-NET Ethernet
    4  => "SmartStickP", # HomeSeer SmartStick +
    6  => "ZWaveMe",     # Z-Wave.me UZB
    2  => "ZStick",      # Aeon Labs Aeotec Z-Stick
    5  => "ZTroller",    # HomeSeer Z-Troller
    8  => "HomeProZC",   # ACT HomePro ZC Series
    11 => "ThinkStick",  # ControlThink ThinkStick
    14 => "AspUSB",      # Cooper Aspire USB
    17 => "AspRFUSB",    # Cooper Aspire RF-USB
    19 => "SmartStick",  # HomeSeer SmartStick Legacy
    20 => "CA8700",      # Intermatic CA8700
    23 => "HA22",        # Intermatic HA22
    25 => "SigmaUZB",    # Sigma Designs UZB
    29 => "WDUSB10",     # Wayne-Dalton WDUSB-10
    32 => "Serial",      # Generic Serial Controller
    35 => "Ethernet",    # Ethernet Interface

They are NOT in the same order.

I didn't have time today, I plan to test it tomorrow. I'll let you know as soon as I do.

I also forgot to mention that the Home ID is also carried over from the first backup requested after a restart or plug-in restart.

[zwolfpack]

OK thanks. I posted an 'r1' that hopefully will address the filename issues.

[randy]

FWIW I have 3 Z-Net V2 named "Primary", "Shed" and "Garage" running software 1.0.23. Each has a unique Home ID.

I am not familiar with perl, but I will do what I can to help figure this out. I ran this event:



The log showed that the event was triggered, but no backups were created. I don't know where to look for logging, so I ran it from a command prompt and here is the session:

Microsoft Windows [Version 10.0.14393]
(c) 2016 Microsoft Corporation. All rights reserved.

C:\Users\Randy>perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl"
GET http://localhost/ZWaveControllers
200 OK
POST http://localhost/ZWaveControllers
[
  OpSelect_27E33F441720 => 24
]
200 OK
POST http://localhost/ZWaveControllers
[
  OpSelect_489ED7854F04 => 24
]
200 OK
POST http://localhost/ZWaveControllers
[
  OpSelect_F1A4B1F6610A => 24
]
200 OK
GET http://localhost/ZWaveControllers
200 OK
POST http://localhost/ZWaveControllers
[
  OpSelect_27E33F441720 => 24
]
200 OK
POST http://localhost/ZWaveControllers
[
  OpSelect_489ED7854F04 => 24
]
200 OK
POST http://localhost/ZWaveControllers
[
  OpSelect_F1A4B1F6610A => 24
]
200 OK

C:\Users\Randy>

I don't know where to look for additional logging and no new backups were created.

[zwolfpack]

Z-Wave Interface Backup Script

Thanks for helping with this!
Here's how it works with one interface:

GET http://192.168.1.103/ZWaveControllers
200 OK
POST http://192.168.1.103/ZWaveControllers
[
  OpSelect_27E33F441720 => 24
]
200 OK
GET http://192.168.1.103/ZWaveControllers
200 OK
POST http://192.168.1.103/ZWaveControllers
[
  OpSelect_27E33F441720 => 24
  File_27E33F441720 => ZNet_House_EBCF7B0B_2017-01-29_12.53.04.ZWave
  name => GO_27E33F441720-24
  id => GO_27E33F441720-24_ID598385
  type => submit
]
200 OK

First the action is set to "back up this interface" for all the interface(s), then on the next pass, the filename fields and submit buttons are available and so are filled and posted. I'm thinking that perhaps only only one action can be in progress at a time...
What happens if you manually follow this same sequence: set the action on each interface, then fill in the filename and submit each one?
The attached "multi-test" version allows an optional 2nd command line parameter: the "form ID". This is the 12 digit hex string that shows right under the Action widget. With this you can try

perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 27E33F441720 
perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 489ED7854F04
perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost F1A4B1F6610A

[randy]

Thanks for helping with this!
Here's how it works with one interface:
First the action is set to "back up this interface" for all the interface(s), then on the next pass, the filename fields and submit buttons are available and so are filled and posted. I'm thinking that perhaps only only one action can be in progress at a time...
What happens if you manually follow this same sequence: set the action on each interface, then fill in the filename and submit each one?
The attached "multi-test" version allows an optional 2nd command line parameter: the "form ID". This is the 12 digit hex string that shows right under the Action widget. With this you can try

perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 27E33F441720 
perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 489ED7854F04
perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost F1A4B1F6610A

When you say "12 digit hex string that shows right under the Action widget", I don't know what you mean. Under controller actions the 8 character HomeID string is in the filename just under the action drop down, but I don't find any 12 digit hex string. What is the Action "widget"?

[randy]

Thanks for helping with this!
Here's how it works with one interface:
First the action is set to "back up this interface" for all the interface(s), then on the next pass, the filename fields and submit buttons are available and so are filled and posted. I'm thinking that perhaps only only one action can be in progress at a time...
What happens if you manually follow this same sequence: set the action on each interface, then fill in the filename and submit each one?
The attached "multi-test" version allows an optional 2nd command line parameter: the "form ID". This is the 12 digit hex string that shows right under the Action widget. With this you can try

perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 27E33F441720 
perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 489ED7854F04
perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost F1A4B1F6610A

I tried three command line entries you suggested as shown here

Microsoft Windows [Version 10.0.14393]
(c) 2016 Microsoft Corporation. All rights reserved.

C:\Users\Randy>perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 27E33F441720
Note: processing limited to form ID 27E33F441720...
GET http://localhost/ZWaveControllers
200 OK
POST http://localhost/ZWaveControllers
[
  OpSelect_27E33F441720 => 24
]
200 OK
GET http://localhost/ZWaveControllers
200 OK
POST http://localhost/ZWaveControllers
[
  OpSelect_27E33F441720 => 24
  File_27E33F441720 => Ethernet_Primary_003D3409_2017-01-29_17.31.28.ZWave
  id => GO_27E33F441720-24_ID598385
  type => submit
  name => GO_27E33F441720-24
]
200 OK

C:\Users\Randy>perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 489ED7854F04
Note: processing limited to form ID 489ED7854F04...
GET http://localhost/ZWaveControllers
200 OK
POST http://localhost/ZWaveControllers
[
  OpSelect_489ED7854F04 => 24
]
200 OK
GET http://localhost/ZWaveControllers
200 OK
POST http://localhost/ZWaveControllers
[
  OpSelect_489ED7854F04 => 24
]
200 OK

C:\Users\Randy>perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost F1A4B1F6610A
Note: processing limited to form ID F1A4B1F6610A...
GET http://localhost/ZWaveControllers
200 OK
POST http://localhost/ZWaveControllers
[
  OpSelect_F1A4B1F6610A => 24
]
200 OK
GET http://localhost/ZWaveControllers
200 OK
POST http://localhost/ZWaveControllers
[
  OpSelect_F1A4B1F6610A => 24
]
200 OK

C:\Users\Randy>

I got one backup from the first command "Ethernet_Primary_003D3409_2017-01-29_17.31.28.ZWave". It appears to be a valid backup. Do I need to look for different IDs for the other two controllers. ?

[zwolfpack]

No, the 2nd and 3rd just aren't behaving themselves. A couple of possibilities come to mind; (1) the field names for the filename and the submit button have a different naming convention for 2nd & subsequent forms; (2) there is some yet to be determined interaction between the forms. To investigate (2), can you stop & start the zwave plugin, then run the 2nd and/or 3rd command and see if you get a result. To investigate (1) I will need to add some additional debugging output.

[randy]

Restarting the plug-in between runs of the script created 3 correctly named backups, that appear to be correct.

[zwolfpack]

I have an idea - after the backup is completed, a CLOSE button appears. I'm not simulating that part. Let me try adding that.

[zwolfpack]

When you say "12 digit hex string that shows right under the Action widget", I don't know what you mean. Under controller actions the 8 character HomeID string is in the filename just under the action drop down, but I don't find any 12 digit hex string. What is the Action "widget"?

Missed this earlier - see below

[zwolfpack]

Z-Wave Interface Backup Script

I have to confess, my web programming skills are firmly rooted in the 1990's. That CLOSE button isn't a normal HTML form element -- its dynamically created by some sort of modern magic.
That being said, setting the action widget to (blank) seems to yield the same result... so try the attached. Again, it takes one or more form ID's on the command line, or if none provided it looks them up and runs them all in sequence.