Backup strategies

[randy]

Backup! Backup! Backup!

There are so many things that can render your HomeSeer system non-functional, a thorough backup routine is imperative. In my system there are several thousand Devices and 1600+ Events. There is no way in the world I would want to build it again from scratch. In nearly 10 years of HS3/HS4 usage I have never had an unrecoverable failure. Take a look at the assumptions \below and come up with your own plan.

All of this is from a Windows O/S perspective, but much would apply to Linux versions as well.

My Backup plan consists of:

• A full HomeSeer directory backup to a local NAS using the BLBackup plug-in, taken at 1:00AM and 1:00PM each day. I retain 60 copies. The NAS is in a different physical location in the house from the rest of my servers and network devices.
• A full metal backup of the HS server using Macirum Reflect. This stores a full image of the system once a month plus differential copies on Mondays and incremental copies each day. These are also stored on the NAS..
• Before I install any plug-in or new HomeSeer version, I shut down HomeSeer and take a copy of the HomeSeer directory structure, placing it on the server desktop. This gives me working copy to roll back to in the event an update has an issue..
• I don't worry about offsite backups for HomeSeer, since a catastrophic event (fire, flood) in my house would likely make HomeSeer useless.

There a number of things to be considered and understood:

• HomeSeer is all contained within a single directory structure. This directory structure is portable in that it contains a fully functional HS implementation
• This directory structure can be copied to another Windows computer and will run without issue as long as the OS (64 or 32 bit) and file placements are similar.
• The built in backup utility only backs up 4 of the folders in this directory. It does backup data, config, scripts and HTML directories, it does not copy HomeSeer or plug-ins. To restore using these backups, you must first install HomeSeer and all plug-ins before restoring these files. The HS data files will backup, but many plug-ins use various database files. Those will be in use and will not backup with the built-in utility.

There are some backup utilities available as a plug-in. I use Blades BLBackup with a twice a day backup and 30 days of retention. If space is a premium, Blade's backup can be put in compressed files. It suffers from the same problems as the built-in as it relates to 3rd party database files.

There are a number of PC based backup programs. Many of these utilize MS Shadow Copy features and can take a snapshot of the drive prior to backup. This snapshot will capture even 3rd party databases that are in use. I chose Macirum Reflect years ago and stick with it. It can do a full metal restore of the server from a networked backup by booting off a thumb drive. This has saved me once when I had an SSD fail on my HS server. It only took replacement of the SSD and about an hour to get my system back to where it was at the last backup. There are many other choices, but a full metal backup ought be in place for anyone who counts on HS to be running. .

In the HomeSeer data directory you will find the HS database files.

For HS3 it was stored in a single SQLite database with an .hsd extension. Along with the primary data file, there will be 10 or more copies stored with _xx appended to the filename. This file is backed up in this fashion each time HS3 is started. Note: the number appended to each file is not sequential. As you can see in the screenshot below, _21 is the newest file and _23 is the oldest. You must pay attention to the timestamps if you want to restore from one of these backups. This one database contains all Devices, Events, Counters and Timers.



For HS4 the database files are JSON backups all stored in a single directory under the \HomeSeer HSx\Data directory. There are 5 JSON files and 5 backup files. Their names explain what is stored in each file. These files contain every Device, Event, Timer and Counter, but each group in their own file. .




Z-Wave

Again some concepts are important.

• The Z-Wave Node Information is all stored within the interface's flash memory.
• A Backup of each interface is created each time a Node is included and Excluded. With multiple interfaces, each interface has its own backup. These backups are stored in the \HomeSeer HSx\Data\Z-Wave directory.
• A new or existing interface can be overwritten with the backup and restored to a fully functional Z-Wave network.
• Z-Wave interfaces each have a unique HomeID. The devices are written with this HomeID when they are included.
• HomeSeer addresses each Z-Wave device by its HomeID and Node number. Each Z-Wave device communicates with the Interface bearing the correct HomeID and the interface communicates with HomeSeer devices through the HomeID and Node number.
• A manual backup can be initiated from the plug-in menus for each interface.
• A manual Restore can be initiated from the plug-in menus.
  • When a controller is restored it is first erased, then it is restored from the backup. The restored interface will then have the HomeID of the backup
  • You cannot have two interfaces with the same HomeID on the same system. If you restore a new controller from a backup, the old controller must be completely offline
  • Some interfaces do not support backup and/or restore operation
• HomeSeer and the interfaces must be in sync. If you restore an Interface from an earlier backup there may not be a node in the controller for a device that exists in HomeSeer. If you restore from an older copy of HomeSeer, there may be Nodes in an interface but no corresponding HomeSeer device.
• When you backup the \HomeSeer HSx\Data directory, you are also backing up the Z-Wave backups.

The Z-Wave backups are stored in the \HomeSeer HSx\Dat\Z-Wave directory and a pair of files are created for each interface.



Each HS Device references the HomeID and Node of the controller.

[drhtmal]

This is great documentation!!! Thanks for posting it.

[mikee123]

Very useful, thank you Randy.
What I would like to know is what is stored in the .hsd files, and can I restore by renaming a older version to the newer by just renaming it to the newest file name? Same question for the events and counter backups, how do I restore from those?

[kenm]

I've tried a couple of times to wrap my head around BLBackup but was never able to get it to make a backup? Should I ask questions here or in the BLBackup sub-forum?

BTW, I currently use Windows File History with 30 days of retention for the HomeSeer HS4 directory and I also use Macrium Reflect for full system backup.

[Kevb]

I've tried a couple of times to wrap my head around BLBackup but was never able to get it to make a backup? Should I ask questions here or in the BLBackup sub-forum?

BTW, I currently use Windows File History with 30 days of retention for the HomeSeer HS4 directory and I also use Macrium Reflect for full system backup.

I didn't either, but learned you have to create an event to have BLBackup create or abort backups. Haven't done it yet, though, just been doing manuals.

[randy]

Very useful, thank you Randy.
What I would like to know is what is stored in the .hsd files, and can I restore by renaming an older version to the newer by just renaming it to the newest file name? Same question for the events and counter backups, how do I restore from those?

For the hsd files, they are SQLite. You can find many free utilities to read and repair SQLite databases. Everything is stored, Events, Devices, Counters and Timers. HS settings are stored in settings.cfg. If you want to restore from one of the backups, rename the primary without the number appended, then remove the appended underscore and number and restart HomeSeer. Alternately you can just select the database even with the appended number by selecting it in setup. The downside is if you have data, data_1, data_2, etc and you switch to data_2, the backups will be data_2_1, data_2_2, etc.

[randy]

I've tried a couple of times to wrap my head around BLBackup but was never able to get it to make a backup? Should I ask questions here or in the BLBackup sub-forum?

BTW, I currently use Windows File History with 30 days of retention for the HomeSeer HS4 directory and I also use Macrium Reflect for full system backup.

I would post in the BLBackup forum. I can respond with help.

[randy]

I didn't either, but learned you have to create an event to have BLBackup create or abort backups. Haven't done it yet, though, just been doing manuals.

It is configuration of backup directories in setup to determine the source, frequency, retention and optional Virtual Device.



An optional virtual device can show backup status, mine is RefID 6537. I also added controls to the Virtual Device allowing me to start a backup, either compressed or not.





The Event dictates the destination, type and format. The Event also triggers from the control I added to the Virtual Device.

[kenm]

Kevb The light bulb has come on, although still a little dim. I wondered why nothing happened after I set everything up on the plugin directories page. It seems like I have to specify much of the same information in the event that I did on the directories page so I'm still somewhat confused about some of the setup. I did setup a manual event and finally got a backup to run.

[ecetron]

• You cannot have two interfaces with the same HomeID on the same system. If you restore a new controller from a backup, the old controller must be completely offline

This makes absolutely no sense to me. Did you mean that you can't have two interfaces with the same HomeID and Node number/ID? I had in my old house 1 Hometroller PI plus 3 Z-nets all on the SAME HomeID. Worked like a charm. I can't reproduce this at the new house because of the limitations of the 4.x plug-in, but it would work fine as long as they all have different Node numbers

[kenm]

I would post in the BLBackup forum. I can respond with help.

randy I posted my questions here.

[kenm]

Any plus to using BLBackup versus manually coping the directory and doing a daily HS4 backup with it's tool? I too never got BLBackup to work. Would be nice if there was a tutorial on how to set it up.

It used to be, not sure if it still is, but a manual copy can't access files that are "in use". Again, might have changed.

Once I get BLBackup figured out, I'm happy to take a stab at a tutorial.

[randy]

Any plus to using BLBackup versus manually coping the directory and doing a daily HS4 backup with it's tool? I too never got BLBackup to work. Would be nice if there was a tutorial on how to set it up.

A manual backup, HS backup or BLBackup will not backup in use database files with integrity. It depends on how often they are written to. The only way I know to backup in use files reliably is using the MS Shadow Copy mechanism. This has to be at the O/S level like Macirum, Acronis or the rest.

If you set up BLBackup like I showed in post 8 above, it will work. It just depends on your databases. It is only 3rd party databases that seem to be at issue. I have Jon00's Graphing and Shill's Device History running which write very frequently. Even if that is an issue, the backup will still restore an otherwise working system.

[jordonmusser]

Could you not run a windows task schedule where it shuts down HS4, copies the whole directory and then restarts HS4?

[randy]

Could you not run a windows task schedule where it shuts down HS4, copies the whole directory and then restarts HS4?

Yes, you could. Using Acronis or Macirum Reflect, both of which support shadow copies, do the same without having to shut down HS. Your suggestion or these products will get you an intact backup of the entire HS ecosystem. It's up to you to decide the best method.