Jump to content

  • Log in with Twitter Log in with Windows Live Log In with Google      Sign In   
  • Create Account

Photo

Backing up large amounts causes server to "freeze"


  • Please log in to reply
7 replies to this topic

#1 WHS_Owner

WHS_Owner

    Advanced Member

  • Members
  • PipPipPip
  • 30 posts

Posted 17 June 2016 - 11:02 AM

Hi,

 

I'm using CloudDrive to backup files to Google Drive. I was using "Allway Sync", but had the issues I will describe below, so switched to "GoodSync", but this issue persists so I have to assume it's related to CloudDrive?

 

I'm using an empty 3TB drive as the cache location (so space is not an issue), I have Cloudbit settings as default (cache 1GB but allowing it to grow as needed). If I set up a sync with either of the above programs for a smaller selection of files like My Pictures (couple of GB) it will all copy to the cache drive quickly then happily saturate my internet upload speed and be uploaded with no problems. As soon as I try and sync a larger selection of folders, say My Movies, I can see it copying to the cache drive, but the next morning I can no longer RDP to my server. 

 

The server is still running, I can see a list of the shared folders from my PC over my home network, Team Speak is running as I can see the entry on my PC, but trying to read/write or RDP to the server is sooo slow to the point it deosn't happen. The server runs WHS 2011, I have the latest beta version of Cloudrive & Drivepool. The files I'm copying are on the Pool but as mentioned the cache drive is separate and not used for anything else. The only way I've been able to get back into the server previously is to hard reboot it by holding down the power button...which can cause problems of it's own!

 

I'm running out of ideas as to what the problem is? Any ideas? Getting pretty frustrated as I know this should work but it's not :-( 



#2 modplan

modplan

    Advanced Member

  • Members
  • PipPipPip
  • 100 posts

Posted 17 June 2016 - 01:59 PM

Windows uses RAM as a cache when copying files between drives. My guess is that your large transfer eventually saturates your computer's RAM before that RAM can be destaged to the cache drive. Since your computer is now starving for RAM, other tasks begin to fail/take forever.

 

My server has 16GB of RAM and I saw something similar. I would suggest using UltraCopier to throttle the copy. I set the throttle to the same as my upload speed and I had no more issues trying to copy a 4TB folder to my CloudDrive (other than it taking a few weeks) :)


  • Christopher (Drashna) likes this

#3 WHS_Owner

WHS_Owner

    Advanced Member

  • Members
  • PipPipPip
  • 30 posts

Posted 17 June 2016 - 02:21 PM

Windows uses RAM as a cache when copying files between drives. My guess is that your large transfer eventually saturates your computer's RAM before that RAM can be destaged to the cache drive. Since your computer is now starving for RAM, other tasks begin to fail/take forever.

 

My server has 16GB of RAM and I saw something similar. I would suggest using UltraCopier to throttle the copy. I set the throttle to the same as my upload speed and I had no more issues trying to copy a 4TB folder to my CloudDrive (other than it taking a few weeks) :)

 

Right ok, that makes sense...the one thing I didn't take into consideration!! Guess I can wait until it's all being copied to the cache and see if the server comes back to "life". This problem will only happen the first time I start these huge uploads anyway then, as after that they will just be backing up any new or different files.

 

Thanks for the info :-)



#4 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,196 posts
  • LocationSan Diego, CA, USA

Posted 17 June 2016 - 08:52 PM

The newest builds (620+) include some changes to the memory management and may help reduce the memory usage.  It may be worth trying it out. 

 

http://dl.covecube.c...24_x64_BETA.exe


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#5 WHS_Owner

WHS_Owner

    Advanced Member

  • Members
  • PipPipPip
  • 30 posts

Posted 18 June 2016 - 03:04 PM

The newest builds (620+) include some changes to the memory management and may help reduce the memory usage.  It may be worth trying it out. 

 

http://dl.covecube.c...24_x64_BETA.exe

 

Ok thank you, would I still need to use something like Ultracopy to limit copy speed to upload speed do you think?



#6 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,196 posts
  • LocationSan Diego, CA, USA

Posted 19 June 2016 - 12:26 AM

Ok thank you, would I still need to use something like Ultracopy to limit copy speed to upload speed do you think?

 

It wouldn't be a bad idea. 


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#7 WHS_Owner

WHS_Owner

    Advanced Member

  • Members
  • PipPipPip
  • 30 posts

Posted 19 June 2016 - 08:52 AM

It wouldn't be a bad idea. 

 

Update! It's fixed. I limited the upload speed to slightly less than my internet upload speed and it's happily chugged away all night. 

 

So the upshot of this is, I don't need a 3TB drive as a cache drive (it'll never get used) as the real limiting factor seems to be memory! In 4 days when this lot is uplloaded I'll switch back to a much smaller cache drive somewhere and make better use of that 3TB!

 

Thanks :-)


  • Christopher (Drashna) likes this

#8 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,196 posts
  • LocationSan Diego, CA, USA

Posted 19 June 2016 - 06:48 PM

Update! It's fixed. I limited the upload speed to slightly less than my internet upload speed and it's happily chugged away all night. 

 

So the upshot of this is, I don't need a 3TB drive as a cache drive (it'll never get used) as the real limiting factor seems to be memory! In 4 days when this lot is uplloaded I'll switch back to a much smaller cache drive somewhere and make better use of that 3TB!

 

Thanks :-)

 

I'm glad to hear that! 


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users