Announcement

Collapse
No announcement yet.

Z-Wave Interface Backup Script

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Z-Wave Interface Backup Script

    I've written a Perl script to automate backup of the Z-Wave interface. As you know, this is normally done thru the HS3 web interface: Plug-ins->Z-Wave->Controller Management. Under Z-Wave Interfaces, expand an interface, then under Actions, select "Back Up this interface", change the filename if desired, then hit START.

    The script performs essentially the same actions:
    1) a GET to retrieve the form and element names
    2) a POST to set the action to "Back Up this interface"
    3) a GET to retrieve the default filename and submit button properties
    4) a POST to submit the backup action

    The script takes one parameter, the HS3 hostname. This defaults to localhost, so if run on the HS3 machine it can be omitted.

    You will need to install Perl in order to use this...

    Linux ==============================
    Perl is installed by default in most Linux distributions. On the Raspberry Pi, some additional modules are required. At the shell prompt enter:

    Code:
    sudo apt-get update
    sudo apt-get -y install libwww-perl
    Windows =================================
    On Windows, ActivePerl, Strawberry Perl and Cygwin are options.
    http://www.activestate.com/activeperl/downloads
    http://strawberryperl.com/
    http://cygwin.com/

    I'll recommend ActivePerl since it's default install is smallest. Download & run the installer appropriate for your architecture (32 or 64 bit). Select "typical" install and take all the defaults.

    =============================
    I developed this on the Raspberry Pi (a z-net and another one). Initially I was thinking of setting up a cron job to run the backup. However I needed to verify it would run on windows,so I tried it under Cygwin on my desktop box, then I installed ActivePerl on my Hometroller. Then I set up this event, which I think is the cleanest way:


    Click image for larger version

Name:	zwave backup event r1.jpg
Views:	64
Size:	39.5 KB
ID:	1268918
    Caveats:
    I only have one interface so I can't test a multiple interface case. However it was written it with that in mind, and from the way the page is structured I think it has a fair chance of working.

    EDIT 1: added revision 'r1' (2017-01-28). Based on new info, the backup file name in multiple interface installations can be problematic. This update attempts to address this.
    EDIT 2: replaced previous revisions with 'r2' (2017-01-31). adds action => updatetime which seems to be required by the page.
    EDIT 3: revision 'r3' (2017-03-03) adds basic authorization capability. Username & password can be hardcoded into the file or provided on command line via -l username/password.
    EDIT 4: Improved procedure for Linux install.
    EDIT 5: revision 'r4' (2017-07-21) has improved retry mechanism needed in multi-interface installations.
    EDIT 6: Updated screenshot to show purge of old files.
    EDIT 7: revision 'r5' (2018-04-05) fix seems to avoid zwave plugin losing its mind and pegging the CPU 10-30 minutes after backup.
    EDIT 8: revision 'r6' (2018-12-12) fix for issue backing up Raspberry Pi 'internal' interface -- simplify file name generation.
    Attached Files
    Last edited by zwolfpack; December 22nd, 2018, 10:46 PM.

  • #2
    Sweet!! Thank you. I've been meaning to do this for a year.

    Confirmed to work on windows 8.1 pro with ActiveState Perl 5.16.3 build 1603. I too only have a single z-wave controller.

    Nice work!

    Comment


    • #3
      Originally posted by zwolfpack View Post
      I've written a Perl script to automate backup of the Z-Wave interface. As you know, this is normally done thru the HS3 web interface: Plug-ins->Z-Wave->Controller Management. Under Z-Wave Interfaces, expand an interface, then under Actions, select "Back Up this interface", change the filename if desired, then hit START.

      The script performs essentially the same actions:
      1) a GET to retrieve the form and element names
      2) a POST to set the action to "Back Up this interface"
      3) a GET to retrieve the default filename and submit button properties
      4) a POST to submit the backup action

      The script takes one parameter, the HS3 hostname. This defaults to localhost, so if run on the HS3 machine it can be omitted.

      You will need to install Perl in order to use this...

      Linux ==============================
      Perl is installed by default in most Linux distributions. On the Raspberry Pi, some additional modules are required. At the shell prompt enter:

      Code:
      sudo cpan
      The first time that CPAN is run, it will offer to perform automatic automatic configuration. Accept this.
      The prompt cpan[1]> should appear -- enter:
      Code:
      install Bundle::CPAN
      It will crank for awhile. Accept the default for any requested inputs.

      When the cpan the prompt returns, enter exit or ^D. You should be ready to go.

      Windows =================================
      On Windows, ActivePerl, Strawberry Perl and Cygwin are options.
      http://www.activestate.com/activeperl/downloads
      http://strawberryperl.com/
      http://cygwin.com/

      I'll recommend ActivePerl since it's default install is smallest. Download & run the installer appropriate for your architecture (32 or 64 bit). Select "typical" install and take all the defaults.

      =============================
      I developed this on the Raspberry Pi (a z-net and another one). Initially I was thinking of setting up a cron job to run the backup. However I needed to verify it would run on windows,so I tried it under Cygwin on my desktop box, then I installed ActivePerl on my Hometroller. Then I set up this event, which I think is the cleanest way:

      [ATTACH]59137[/ATTACH]

      Caveats:
      I only have one interface so I can't test a multiple interface case. However it was written it with that in mind, and from the way the page is structured I think it has a fair chance of working.
      Thank you! This is the one backup that depends on my remembering - not a good thing
      I have three interfaces, so I will give it a go and let you know.

      The only problem I have now is that the first interface I backup sets the default file name preamble name for all of the interfaces. This is a HS bug. Here I backed up the Shed interface first, so the garage backup starts with "ZNet_Shed". We'll see if the script behaves the same.

      Click image for larger version

Name:	Capture.PNG
Views:	1
Size:	9.4 KB
ID:	1188722
      Last edited by rprade; January 28th, 2017, 11:29 AM.
      Randy Prade
      Aurora, CO
      Prades.net

      PHLocation - Pushover - EasyTrigger - UltraECM3 - Ultra1Wire3 - Arduino

      Comment


      • #4
        Originally posted by rprade View Post
        The only problem I have now is that the first interface I backup sets the default file name preamble name for all of the interfaces. This is a HS bug. Here I backed up the Shed interface first, so the garage backup starts with "ZNet_Shed". We'll see if the script behaves the same.
        Hi Randy, I think this will be an issue. I noticed that the timestamp in the provided filename only gets updated the 1st time the form is called after a plugin restart - so I substitute in the current time. Factoring in the info you provided though, looks like the bug goes deeper. I'm now thinking that the script should just build the filename from scratch.

        To this end, can you confirm/deny if the list of interfaces in the 'Z-Wave Networks and Options' section appears in the same order as that in the 'Z-Wave Interfaces' section? If the order always matches, it greatly eases getting the friendly names and Home ID's to use in building the filenames.

        Last edited by zwolfpack; December 22nd, 2018, 10:43 PM.

        Comment


        • #5
          Originally posted by zwolfpack View Post
          Hi Randy, I think this will be an issue. I noticed that the timestamp in the provided filename only gets updated the 1st time the form is called after a plugin restart - so I substitute in the current time. Factoring in the info you provided though, looks like the bug goes deeper. I'm now thinking that the script should just build the filename from scratch.

          To this end, can you confirm/deny if the list of interfaces in the 'Z-Wave Networks and Options' section appears in the same order as that in the 'Z-Wave Interfaces' section? If the order always matches, it greatly eases getting the friendly names and Home ID's to use in building the filenames.

          Also, we'll need a table of the Interface Model abbreviations that appear at the beginning of the filename (ZNet_...) for example. Here's what I'll propose; any suggestions/corrections welcomed. (3rd column is the filename prefix for each interface model).
          Code:
              0  => "None",        # None,
              1  => "ZNet",        # HomeSeer Z-NET Ethernet
              4  => "SmartStickP", # HomeSeer SmartStick +
              6  => "ZWaveMe",     # Z-Wave.me UZB
              2  => "ZStick",      # Aeon Labs Aeotec Z-Stick
              5  => "ZTroller",    # HomeSeer Z-Troller
              8  => "HomeProZC",   # ACT HomePro ZC Series
              11 => "ThinkStick",  # ControlThink ThinkStick
              14 => "AspUSB",      # Cooper Aspire USB
              17 => "AspRFUSB",    # Cooper Aspire RF-USB
              19 => "SmartStick",  # HomeSeer SmartStick Legacy
              20 => "CA8700",      # Intermatic CA8700
              23 => "HA22",        # Intermatic HA22
              25 => "SigmaUZB",    # Sigma Designs UZB
              29 => "WDUSB10",     # Wayne-Dalton WDUSB-10
              32 => "Serial",      # Generic Serial Controller
              35 => "Ethernet",    # Ethernet Interface
          They are NOT in the same order.

          I didn't have time today, I plan to test it tomorrow. I'll let you know as soon as I do.

          I also forgot to mention that the Home ID is also carried over from the first backup requested after a restart or plug-in restart.
          Randy Prade
          Aurora, CO
          Prades.net

          PHLocation - Pushover - EasyTrigger - UltraECM3 - Ultra1Wire3 - Arduino

          Comment


          • #6
            OK thanks. I posted an 'r1' that hopefully will address the filename issues.

            Comment


            • #7
              FWIW I have 3 Z-Net V2 named "Primary", "Shed" and "Garage" running software 1.0.23. Each has a unique Home ID.

              I am not familiar with perl, but I will do what I can to help figure this out. I ran this event:

              Click image for larger version

Name:	Capture.jpg
Views:	1
Size:	25.2 KB
ID:	1188732

              The log showed that the event was triggered, but no backups were created. I don't know where to look for logging, so I ran it from a command prompt and here is the session:
              Code:
              Microsoft Windows [Version 10.0.14393]
              (c) 2016 Microsoft Corporation. All rights reserved.
              
              C:\Users\Randy>perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl"
              GET http://localhost/ZWaveControllers
              200 OK
              POST http://localhost/ZWaveControllers
              [
                OpSelect_27E33F441720 => 24
              ]
              200 OK
              POST http://localhost/ZWaveControllers
              [
                OpSelect_489ED7854F04 => 24
              ]
              200 OK
              POST http://localhost/ZWaveControllers
              [
                OpSelect_F1A4B1F6610A => 24
              ]
              200 OK
              GET http://localhost/ZWaveControllers
              200 OK
              POST http://localhost/ZWaveControllers
              [
                OpSelect_27E33F441720 => 24
              ]
              200 OK
              POST http://localhost/ZWaveControllers
              [
                OpSelect_489ED7854F04 => 24
              ]
              200 OK
              POST http://localhost/ZWaveControllers
              [
                OpSelect_F1A4B1F6610A => 24
              ]
              200 OK
              
              C:\Users\Randy>
              I don't know where to look for additional logging and no new backups were created.
              Randy Prade
              Aurora, CO
              Prades.net

              PHLocation - Pushover - EasyTrigger - UltraECM3 - Ultra1Wire3 - Arduino

              Comment


              • #8
                Z-Wave Interface Backup Script

                Thanks for helping with this!
                Here's how it works with one interface:
                Code:
                GET http://192.168.1.103/ZWaveControllers
                200 OK
                POST http://192.168.1.103/ZWaveControllers
                [
                  OpSelect_27E33F441720 => 24
                ]
                200 OK
                GET http://192.168.1.103/ZWaveControllers
                200 OK
                POST http://192.168.1.103/ZWaveControllers
                [
                  OpSelect_27E33F441720 => 24
                  File_27E33F441720 => ZNet_House_EBCF7B0B_2017-01-29_12.53.04.ZWave
                  name => GO_27E33F441720-24
                  id => GO_27E33F441720-24_ID598385
                  type => submit
                ]
                200 OK
                First the action is set to "back up this interface" for all the interface(s), then on the next pass, the filename fields and submit buttons are available and so are filled and posted. I'm thinking that perhaps only only one action can be in progress at a time...
                What happens if you manually follow this same sequence: set the action on each interface, then fill in the filename and submit each one?
                The attached "multi-test" version allows an optional 2nd command line parameter: the "form ID". This is the 12 digit hex string that shows right under the Action widget. With this you can try

                Code:
                perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 27E33F441720 
                perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 489ED7854F04
                perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost F1A4B1F6610A
                Last edited by zwolfpack; March 25th, 2017, 03:44 AM.

                Comment


                • #9
                  Originally posted by zwolfpack View Post
                  Thanks for helping with this!
                  Here's how it works with one interface:
                  First the action is set to "back up this interface" for all the interface(s), then on the next pass, the filename fields and submit buttons are available and so are filled and posted. I'm thinking that perhaps only only one action can be in progress at a time...
                  What happens if you manually follow this same sequence: set the action on each interface, then fill in the filename and submit each one?
                  The attached "multi-test" version allows an optional 2nd command line parameter: the "form ID". This is the 12 digit hex string that shows right under the Action widget. With this you can try

                  Code:
                  perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 27E33F441720 
                  perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 489ED7854F04
                  perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost F1A4B1F6610A
                  When you say "12 digit hex string that shows right under the Action widget", I don't know what you mean. Under controller actions the 8 character HomeID string is in the filename just under the action drop down, but I don't find any 12 digit hex string. What is the Action "widget"?
                  Randy Prade
                  Aurora, CO
                  Prades.net

                  PHLocation - Pushover - EasyTrigger - UltraECM3 - Ultra1Wire3 - Arduino

                  Comment


                  • #10
                    Originally posted by zwolfpack View Post
                    Thanks for helping with this!
                    Here's how it works with one interface:
                    First the action is set to "back up this interface" for all the interface(s), then on the next pass, the filename fields and submit buttons are available and so are filled and posted. I'm thinking that perhaps only only one action can be in progress at a time...
                    What happens if you manually follow this same sequence: set the action on each interface, then fill in the filename and submit each one?
                    The attached "multi-test" version allows an optional 2nd command line parameter: the "form ID". This is the 12 digit hex string that shows right under the Action widget. With this you can try

                    Code:
                    perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 27E33F441720 
                    perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 489ED7854F04
                    perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost F1A4B1F6610A
                    I tried three command line entries you suggested as shown here
                    Code:
                    Microsoft Windows [Version 10.0.14393]
                    (c) 2016 Microsoft Corporation. All rights reserved.
                    
                    C:\Users\Randy>perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 27E33F441720
                    Note: processing limited to form ID 27E33F441720...
                    GET http://localhost/ZWaveControllers
                    200 OK
                    POST http://localhost/ZWaveControllers
                    [
                      OpSelect_27E33F441720 => 24
                    ]
                    200 OK
                    GET http://localhost/ZWaveControllers
                    200 OK
                    POST http://localhost/ZWaveControllers
                    [
                      OpSelect_27E33F441720 => 24
                      File_27E33F441720 => Ethernet_Primary_003D3409_2017-01-29_17.31.28.ZWave
                      id => GO_27E33F441720-24_ID598385
                      type => submit
                      name => GO_27E33F441720-24
                    ]
                    200 OK
                    
                    C:\Users\Randy>perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost 489ED7854F04
                    Note: processing limited to form ID 489ED7854F04...
                    GET http://localhost/ZWaveControllers
                    200 OK
                    POST http://localhost/ZWaveControllers
                    [
                      OpSelect_489ED7854F04 => 24
                    ]
                    200 OK
                    GET http://localhost/ZWaveControllers
                    200 OK
                    POST http://localhost/ZWaveControllers
                    [
                      OpSelect_489ED7854F04 => 24
                    ]
                    200 OK
                    
                    C:\Users\Randy>perl "C:\Program Files (x86)\HomeSeer HS3\scripts\zwbkup.pl" localhost F1A4B1F6610A
                    Note: processing limited to form ID F1A4B1F6610A...
                    GET http://localhost/ZWaveControllers
                    200 OK
                    POST http://localhost/ZWaveControllers
                    [
                      OpSelect_F1A4B1F6610A => 24
                    ]
                    200 OK
                    GET http://localhost/ZWaveControllers
                    200 OK
                    POST http://localhost/ZWaveControllers
                    [
                      OpSelect_F1A4B1F6610A => 24
                    ]
                    200 OK
                    
                    C:\Users\Randy>
                    I got one backup from the first command "Ethernet_Primary_003D3409_2017-01-29_17.31.28.ZWave". It appears to be a valid backup. Do I need to look for different IDs for the other two controllers. ?
                    Randy Prade
                    Aurora, CO
                    Prades.net

                    PHLocation - Pushover - EasyTrigger - UltraECM3 - Ultra1Wire3 - Arduino

                    Comment


                    • #11
                      No, the 2nd and 3rd just aren't behaving themselves. A couple of possibilities come to mind; (1) the field names for the filename and the submit button have a different naming convention for 2nd & subsequent forms; (2) there is some yet to be determined interaction between the forms. To investigate (2), can you stop & start the zwave plugin, then run the 2nd and/or 3rd command and see if you get a result. To investigate (1) I will need to add some additional debugging output.

                      Comment


                      • #12
                        Restarting the plug-in between runs of the script created 3 correctly named backups, that appear to be correct.
                        Randy Prade
                        Aurora, CO
                        Prades.net

                        PHLocation - Pushover - EasyTrigger - UltraECM3 - Ultra1Wire3 - Arduino

                        Comment


                        • #13
                          I have an idea - after the backup is completed, a CLOSE button appears. I'm not simulating that part. Let me try adding that.

                          Comment


                          • #14
                            Originally posted by rprade View Post
                            When you say "12 digit hex string that shows right under the Action widget", I don't know what you mean. Under controller actions the 8 character HomeID string is in the filename just under the action drop down, but I don't find any 12 digit hex string. What is the Action "widget"?
                            Missed this earlier - see below
                            Attached Files

                            Comment


                            • #15
                              Z-Wave Interface Backup Script

                              I have to confess, my web programming skills are firmly rooted in the 1990's. That CLOSE button isn't a normal HTML form element -- its dynamically created by some sort of modern magic.
                              That being said, setting the action widget to (blank) seems to yield the same result... so try the attached. Again, it takes one or more form ID's on the command line, or if none provided it looks them up and runs them all in sequence.
                              Last edited by zwolfpack; March 25th, 2017, 03:44 AM.

                              Comment

                              Working...
                              X