Announcement

Collapse
No announcement yet.

Extremely slow backup

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • TomB
    replied
    That did it, it took 1 min, 10 secs to do a full backup after I deleted the database and reinstalled the plugin. That's more like the speed I would expect

    Thank you so much to all for the input, glad we got there in the end.

    Over the network to SMB/NAS it took 3 mins 50 secs, so very quick indeed.
    Last edited by TomB; January 21st, 2018, 04:43 AM.

    Leave a comment:


  • Blade
    replied
    Originally posted by TomB View Post
    Here is the log-file you asked for now running with SSD backing, and taking about 40-45 minutes. It's a dropbox link due to size.

    https://www.dropbox.com/s/e30f92xws6...ailed.log?dl=0

    Edit: It seems it process each file a ton of times, but I am not sure if that's how it should be or not. If you have time to check it out I would be most grateful.
    Your database does not look right to me
    I would try this:

    1. Disable BLBackup (make sure the HSPI_BLBackup.exe process is not running - if it is then kill it)
    2. Delete the BLBackup.s3db
    3. Re-install the plugin
    4. Enable the plugin

    See if that helps

    Leave a comment:


  • Blade
    replied
    Originally posted by TomB View Post
    Quick question on the BLBackup.s3db file, is that necessary to keep for a long time or can I delete it and let it refresh? I have had it set to daily backup and 60 backups to keep, not sure if that is the reason but that file is now 160MB.

    Don't want to remove it if it's vital to the operation of BLBackup though
    Mine is only 29MB
    The database is used by the plugin when running a backup. It will cause issues if you delete it

    Leave a comment:


  • TomB
    replied
    Here is the log-file you asked for now running with SSD backing, and taking about 40-45 minutes. It's a dropbox link due to size.

    https://www.dropbox.com/s/e30f92xws6...ailed.log?dl=0

    Edit: It seems it process each file a ton of times, but I am not sure if that's how it should be or not. If you have time to check it out I would be most grateful.
    Last edited by TomB; January 20th, 2018, 04:04 PM.

    Leave a comment:


  • TomB
    replied
    Quick question on the BLBackup.s3db file, is that necessary to keep for a long time or can I delete it and let it refresh? I have had it set to daily backup and 60 backups to keep, not sure if that is the reason but that file is now 160MB.

    Don't want to remove it if it's vital to the operation of BLBackup though

    Leave a comment:


  • TomB
    replied
    Originally posted by SteveMSJ View Post
    Tom, I use BLBackup to backup my entire HS3 folder to the hard disk on another computer (not SSD) over the local network and it takes a fraction over 2 minutes for 2403 files. In my case I have it set to create a zip file so it is only writing one big file over the network rather than thousands of small files. That is probably the difference.

    Steve
    If you monitor your ZiP-file while the backup is going I assume you as well see it gradually increasing, its not doing local ZIP and then one file move, it adds to the remote ZIP-file the whole process.

    But that's not whats making it slow on my side at least since it's the same speed locally as well.

    Leave a comment:


  • TomB
    replied
    Yes, as one mentioned, I am also creating just one ZIP-file.

    The system is quick as any, it caps the Gbit NIC when doing SMB copying both to and from the system and my NAS (where I run my backups to), so it is just with BLBackup it is this slow. There was very little difference when I do file-structure instead of ZIP, just less CPU on the system itself.

    A manual copy of the whole HS3 folder with about 3300 files takes less than 30 seconds so it does not seem to be any network issues.

    The only thing common between the old and new system is the HS3 folder itself. I installed Windows 10, did all updates, then installed HS3 from fresh file, renamed the folder and copied in the old HS3 folder and run it.

    It is fine though, I can live with the 45 minutes time, it just seems odd if should take so long. It was worse when it took 5+ hours but this is okay. I have like 20 SSDs in the house so why I decided to go with the spin-disk for this machine I don't really know, I won't ever do it again

    Leave a comment:


  • SteveMSJ
    replied
    Yes, I should read more carefully. However, mine is a 99MB zip file so something is up with Tom's system.

    Steve

    Leave a comment:


  • rprade
    replied
    Originally posted by SteveMSJ View Post
    Tom, I use BLBackup to backup my entire HS3 folder to the hard disk on another computer (not SSD) over the local network and it takes a fraction over 2 minutes for 2403 files. In my case I have it set to create a zip file so it is only writing one big file over the network rather than thousands of small files. That is probably the difference.

    Steve

    Leave a comment:


  • rprade
    replied
    My backup is folder based for the entire HS3 directory and all subdirectories. It is over 5gb to a Synology NAS over the network. It currently takes just under 15 minutes.

    Jan-20 1:29:46 AM BLBackup Info The backup process (Folder Tree - Full) has completed at 1/20/2018 1:29:46 AM
    Jan-20 1:15:00 AM BLBackup Info The backup process (Folder Tree - Full) has started at 1/20/2018 1:15:00 AM...

    Leave a comment:


  • SteveMSJ
    replied
    Originally posted by TomB View Post
    Ok, so I found the reason for the extremely slow performance. It was apparently the disk, not that the disk itself was slow, but it seems when doing backup in this way of Homeseer, it opens one session for each and every small file, and the result is slower access with spin-disk. I changed to a SSD drive again (cloned system-drive) and now I am back to about 45 minutes backup window, and I can live with it.

    However, normal copy in Windows is a LOT faster, so I am not sure how you are doing the backup with your plugin as the performance is so much slower within than using normal copy/zip.

    Either way, I am back to where I was before, so I guess that's as good as it gets right now
    Tom, I use BLBackup to backup my entire HS3 folder to the hard disk on another computer (not SSD) over the local network and it takes a fraction over 2 minutes for 2403 files. In my case I have it set to create a zip file so it is only writing one big file over the network rather than thousands of small files. That is probably the difference.

    Steve

    Leave a comment:


  • TomB
    replied
    Ok, so I found the reason for the extremely slow performance. It was apparently the disk, not that the disk itself was slow, but it seems when doing backup in this way of Homeseer, it opens one session for each and every small file, and the result is slower access with spin-disk. I changed to a SSD drive again (cloned system-drive) and now I am back to about 45 minutes backup window, and I can live with it.

    However, normal copy in Windows is a LOT faster, so I am not sure how you are doing the backup with your plugin as the performance is so much slower within than using normal copy/zip.

    Either way, I am back to where I was before, so I guess that's as good as it gets right now

    Leave a comment:


  • Blade
    replied
    I use a zip backup and I backup many folders from different computers and I have not seen it take an extended amount of time
    Try and get me a debug log of the backup so I can see if something is taking longer than it should

    Leave a comment:


  • TomB
    replied
    Just update, took 5 hours 10 minutes with a local backup even.

    The new computer is an Intel I5 with 16GB ram while the old was a Intel Celeron with 8GB ram, but the old one had a 256GB SSD while the new has a 750GB hybrid disk. But it should not be that big a difference, backing up 1,5GB shouldn't really take 45 minutes on the old one either.

    What would you say is the normal backup period? As Zip or as folder, what do you usually use? I have always used ZIP, but the CPU is only at 10% or so when running backup.

    Leave a comment:


  • TomB
    replied
    I set it in debug mode and then ran the backup, but since it ran for over 6 hours and we got company, I forgot to get the file, and it ran once more during the night. The issue now is that the log-file is actually 1GB and I think I cannot attach that here. So thinking I should delete it again and then maybe try one more.

    Right now I am trying a local backup, but it is running just as slow.

    Leave a comment:

Working...
X