Announcement

Collapse
No announcement yet.

Extremely slow backup

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Extremely slow backup

    Hello!

    I have been using your plugin for roughly a year, and I am running it on a Windows 10 install of Homeseer. I recently changed computers but it is still the same setup, Windows 10 with Gbit connection to my network, and a SMB share where I point the backups to.

    Before the change of computer the backup took about 45 minutes, something which is still very long time for about 130MB zipped file, but it was just steadily increasing over time so I expected it to be okay. However, after change of computer it has now taken more than 8 hours, and in the end I have just stopped the service. The file is then up to about 115MB, so its still 15MB to go.

    Do you have any tips on why this is slow? I have turned off Real Time Inspection and I do not have any other anti-virus running. I have turned off TCP/UDP offloading on the NIC, and normal copy runs at close to 1Gbit/s speed.

    Bit at a loss here, and would very much like to get this sorted.

    #2
    That seems odd but I would say it is the new computer in some way causing this. The plugin did not change.
    You could turn on debug logging in the plugin and then run the backup and when it is done, turn off debug logging and attach the BLBackup-Detailed.log zipped here and I can see if anything jumps out at me
    Cheers,
    Bob
    Web site | Help Desk | Feature Requests | Message Board

    Comment


      #3
      Thanks for getting back to me.

      Yes, I know it will seem like it's the machine, but right before I changed computers (same day) I also upgraded to the newest version 2.0.50.0, as I was on an the 2.0.41.0 before.. I did not run a backup after that, so cannot say if it was slow or not on the old computer.

      I put another backup on at 08:01 this morning, it is now 12:46 and it is currently at about 119.5MB. It goes quite fast in the beginning and just slows off more and more towards the end.

      Do you think it would make a difference to run to local disk instead of network share? I have never tried it as it defies the purpose of backup, I want it in another place.

      I will run another backup after this finishes with debug mode on. Do I set that in Config in the UI or do I need to edit a ini-file somewhere?

      Comment


        #4
        You could try it to local disk and see if it makes a difference

        There is an options page in the plugin web pages where you will see debug logging
        Turn that on, run the backup and then turn it off

        Attach the BLBackup-Detailed.log zipped
        It is in the HS3 root folder
        Cheers,
        Bob
        Web site | Help Desk | Feature Requests | Message Board

        Comment


          #5
          I set it in debug mode and then ran the backup, but since it ran for over 6 hours and we got company, I forgot to get the file, and it ran once more during the night. The issue now is that the log-file is actually 1GB and I think I cannot attach that here. So thinking I should delete it again and then maybe try one more.

          Right now I am trying a local backup, but it is running just as slow.

          Comment


            #6
            Just update, took 5 hours 10 minutes with a local backup even.

            The new computer is an Intel I5 with 16GB ram while the old was a Intel Celeron with 8GB ram, but the old one had a 256GB SSD while the new has a 750GB hybrid disk. But it should not be that big a difference, backing up 1,5GB shouldn't really take 45 minutes on the old one either.

            What would you say is the normal backup period? As Zip or as folder, what do you usually use? I have always used ZIP, but the CPU is only at 10% or so when running backup.

            Comment


              #7
              I use a zip backup and I backup many folders from different computers and I have not seen it take an extended amount of time
              Try and get me a debug log of the backup so I can see if something is taking longer than it should
              Cheers,
              Bob
              Web site | Help Desk | Feature Requests | Message Board

              Comment


                #8
                Ok, so I found the reason for the extremely slow performance. It was apparently the disk, not that the disk itself was slow, but it seems when doing backup in this way of Homeseer, it opens one session for each and every small file, and the result is slower access with spin-disk. I changed to a SSD drive again (cloned system-drive) and now I am back to about 45 minutes backup window, and I can live with it.

                However, normal copy in Windows is a LOT faster, so I am not sure how you are doing the backup with your plugin as the performance is so much slower within than using normal copy/zip.

                Either way, I am back to where I was before, so I guess that's as good as it gets right now

                Comment


                  #9
                  Originally posted by TomB View Post
                  Ok, so I found the reason for the extremely slow performance. It was apparently the disk, not that the disk itself was slow, but it seems when doing backup in this way of Homeseer, it opens one session for each and every small file, and the result is slower access with spin-disk. I changed to a SSD drive again (cloned system-drive) and now I am back to about 45 minutes backup window, and I can live with it.

                  However, normal copy in Windows is a LOT faster, so I am not sure how you are doing the backup with your plugin as the performance is so much slower within than using normal copy/zip.

                  Either way, I am back to where I was before, so I guess that's as good as it gets right now
                  Tom, I use BLBackup to backup my entire HS3 folder to the hard disk on another computer (not SSD) over the local network and it takes a fraction over 2 minutes for 2403 files. In my case I have it set to create a zip file so it is only writing one big file over the network rather than thousands of small files. That is probably the difference.

                  Steve

                  Comment


                    #10
                    My backup is folder based for the entire HS3 directory and all subdirectories. It is over 5gb to a Synology NAS over the network. It currently takes just under 15 minutes.

                    Jan-20 1:29:46 AM BLBackup Info The backup process (Folder Tree - Full) has completed at 1/20/2018 1:29:46 AM
                    Jan-20 1:15:00 AM BLBackup Info The backup process (Folder Tree - Full) has started at 1/20/2018 1:15:00 AM...
                    HS4 Pro, 4.2.19.0 Windows 10 pro, Supermicro LP Xeon

                    Comment


                      #11
                      Originally posted by SteveMSJ View Post
                      Tom, I use BLBackup to backup my entire HS3 folder to the hard disk on another computer (not SSD) over the local network and it takes a fraction over 2 minutes for 2403 files. In my case I have it set to create a zip file so it is only writing one big file over the network rather than thousands of small files. That is probably the difference.

                      Steve
                      HS4 Pro, 4.2.19.0 Windows 10 pro, Supermicro LP Xeon

                      Comment


                        #12
                        Yes, I should read more carefully. However, mine is a 99MB zip file so something is up with Tom's system.

                        Steve

                        Comment


                          #13
                          Yes, as one mentioned, I am also creating just one ZIP-file.

                          The system is quick as any, it caps the Gbit NIC when doing SMB copying both to and from the system and my NAS (where I run my backups to), so it is just with BLBackup it is this slow. There was very little difference when I do file-structure instead of ZIP, just less CPU on the system itself.

                          A manual copy of the whole HS3 folder with about 3300 files takes less than 30 seconds so it does not seem to be any network issues.

                          The only thing common between the old and new system is the HS3 folder itself. I installed Windows 10, did all updates, then installed HS3 from fresh file, renamed the folder and copied in the old HS3 folder and run it.

                          It is fine though, I can live with the 45 minutes time, it just seems odd if should take so long. It was worse when it took 5+ hours but this is okay. I have like 20 SSDs in the house so why I decided to go with the spin-disk for this machine I don't really know, I won't ever do it again

                          Comment


                            #14
                            Originally posted by SteveMSJ View Post
                            Tom, I use BLBackup to backup my entire HS3 folder to the hard disk on another computer (not SSD) over the local network and it takes a fraction over 2 minutes for 2403 files. In my case I have it set to create a zip file so it is only writing one big file over the network rather than thousands of small files. That is probably the difference.

                            Steve
                            If you monitor your ZiP-file while the backup is going I assume you as well see it gradually increasing, its not doing local ZIP and then one file move, it adds to the remote ZIP-file the whole process.

                            But that's not whats making it slow on my side at least since it's the same speed locally as well.

                            Comment


                              #15
                              Quick question on the BLBackup.s3db file, is that necessary to keep for a long time or can I delete it and let it refresh? I have had it set to daily backup and 60 backups to keep, not sure if that is the reason but that file is now 160MB.

                              Don't want to remove it if it's vital to the operation of BLBackup though

                              Comment

                              Working...
                              X