Jump to content
Covecube Inc.
  • 0
Tell

Local cache disk filling up despite "Fixed" cache size

Question

Hi all,

 

I've run into a situation where the local cache drive is getting filled up despite having a fixed local cache. Configuration is:

  • StableBit CloudDrive v 1.0.0.634 BETA
  • 4 TB drive created on Amazon Cloud Drive with 30 MB chunk size, custom security profile
  • Local cache set to 6 GB FIXED on a 120 GB SSD (the SSD is exclusive to CloudDrive - there's absolutely nothing else on this drive)
  • Lots of data to upload

When the local cache is filled up (6 GB) CD starts throttling write requests, as it should be doing (hooray for this feature, by the way). However, when the total amount of data uploaded is nearing the size of the cache drive, CloudDrive starts slowing down until it completely stops accepting writes and throws a red warning message saying that the local cache drive is full.

 

This is the CloudPart-folder after a session of having uploaded approx 30 GB of data.

 

post-2445-0-50523000-1469530493_thumb.png

 

This is the local cache disk at the same time as the screenshot above. Remember, there is absolutely nothing on this drive other than the CloudPart-folder.

 

post-2445-0-99852600-1469530502_thumb.png

 

Selecting "Performance --> Clear local cache" does nothing. Detaching and re-attaching the drive clears and empties the local drive, reducing the "Used space" to almost nothing, and I can again start filling the cloud drive with data until the cache drive runs full again.

 

As is obvious, a discrepancy exists between the amount of data reported as "Used space" on the SSD and the "Size on disk" of the CloudPart folder. My guess is that this is some sort of bug related to the handling of NTFS sparse files. Any ideas?

Share this post


Link to post
Share on other sites

Recommended Posts

  • 0

After some testing, I can confirm that the .777 build does NOT resolve this issue.

 

I understand that the issue is caused by how NTFS handles sparse files. Why is this issue so "rare"? Is it only a small subset of users that experience this bug? Does not everybody see this happen when they copy more data to cloud drive than the size of the cache?

 

For me, this makes CloudDrive incredibly difficult to use – I'm trying to upload about 8 TB of data, and given a 480 GB SSD as a cache drive, you can do the math on how many reboots I have to do, how many times I have to re-start the copy and how often I need to check the entire CloudDrive for data consistency (as CloudDrive crashes when the cache drive is full).

 

The reserved space is released when the CloudDrive service is restarted. This indicates that the deallocated blocks are freed when the file handles are closed from CloudDrive. To me, it seems like a piece of cake to just make CloudDrive  release and re-attach cache file handles on a regular basis – for example, for every 25% of cache size writes and/or when the cache drive nears full or writes are beginning to get throttled.

Share this post


Link to post
Share on other sites
  • 0

Not sure if this is related but maybe it will be helpful to someone

 

 

I had an issue where my cache drive was filling up and it ended up being because the ACD software was running and going crazy trying to sync on it's own and had made a boat load of its own files that filled the drive. I exited the amazon software (In fact i ended up uninstalling it) deleted all of the random files it made and havent had an issue since.

Share this post


Link to post
Share on other sites
  • 0

Not sure if this is related but maybe it will be helpful to someone

 

 

I had an issue where my cache drive was filling up and it ended up being because the ACD software was running and going crazy trying to sync on it's own and had made a boat load of its own files that filled the drive. I exited the amazon software (In fact i ended up uninstalling it) deleted all of the random files it made and havent had an issue since.

Thank you for your input! Unfortunately, the ACD software is not related to the issue being discussed in this thread. This thread details a problem that appears when an amount of data greater than the size of the drive the cache resides on is written (copied) to the CloudDrive drive.

Share this post


Link to post
Share on other sites
  • 0

Thanks Christopher. While most of that was understood, I only find fault with categorizing an issue that fills up a 120GB SSD and knocks all of my drives offline in 6-12 hours an issue that "generally resolves itself" unless by resolving itself you mean forcing the dismount of my drives so that the clusters are finally freed....

 

For me, and it looks like others, it makes CloudDrive utterly useless.

 

Alex may have a fix out for this soon. Or at least a change to the code that *may* help.  

 

I'll try to let you know when he's posted it. 

Share this post


Link to post
Share on other sites
  • 0

Alex may have a fix out for this soon. Or at least a change to the code that *may* help.  

 

I'll try to let you know when he's posted it. 

 

Thanks Chris, any updates since last month? Would like to get back to using CloudDrive, not seeing anything obviously relevant in the changelog.

Share this post


Link to post
Share on other sites
  • 0

At this point, no, not yet. 

 

I believe that Alex will deal with this later, but not now.   Once there is a stable release, he may look into it.

 

That said, I've stuck up a post-it not, so I can bug Alex about this, "directly"

 

Bummer, since you thought he may have a fix way back in December. Unfortunate, but thanks for the update.

Share this post


Link to post
Share on other sites
  • 0

If the SSD is getting filled up a way would be to calculate depending how much space is left and how much it take to fill and lower it by that amount so it adjusts your speed so it doesn't get lower. As of now it just dismounts. 

Share this post


Link to post
Share on other sites
  • 0

I've watched this thread for nearly a year when the same thing started happening to me and there have been no further updates or indication that this has any priority.  I too am going to have to abandon using CloudDrive for this continual dismount issue caused by the host disk being filled and dismounting the CloudDrive until the system is rebooted.  I have a daily reboot configured to try to mitigate this but even that is not frequent enough at times.  It is sad though, due to the fact that it does work and perform very well for my needs otherwise.  I still will use DrivePool and Stablebit Scanner on my other systems as they have been very dependable for me.  I have CloudDrive implemented with DrivePool and the plugin to control file placement so that my single host disk acts as the initial local writable space  in the DrivePool until the balancer moves the data to CloudDrive.  If the host disk did not continually get full with sparse files which triggers the crash and dismount of the CloudDrive it would be perfect for my needs.  I've had to resort to building a much more complex Linux solution that offers nearly the same functionality I need using rclone and unionfs-fuse.  The only really bad thing is that trying to get a large amount of data out of CloudDrive will only trigger the sparse file dismounts more frequently.

Share this post


Link to post
Share on other sites
  • 0
6 minutes ago, benjamato said:

I've watched this thread for nearly a year when the same thing started happening to me and there have been no further updates or indication that this has any priority

The problem is that this really isn't a "CloudDrive" issue, but more of an issue with NTFS itself.  

We use sparse files for the cache, and sparse files are pretty common, and well documented.  And we're not doing anything complex here with Sparse files.  

So, tracking down the cause, is difficult, at best.  And any fix... most likely wouldn't be with our code.   
So not only is there only so much we can do here, but there is only so much looking that we can do before we have to move on to other issues. 

 

That said, I'll mention this to Alex. 

Share this post


Link to post
Share on other sites
  • 0

Wow a blast from the past. Still following this thread apparently, got an email notification. I abandoned cloud drive over a year ago since this issue made it utterly useless for me and no progress was being made on a resolution. Sad to see this hasn't been fixed yet as I would still find my purchase useful if it had been fixed and could begin use cloud drive again.

Chris, if this is an issue with sparse files on NTFS, perhaps cloud drive should not be using them and use something else instead? I write software for a living, and if a dependency is broken in a way that I cannot fix and that negatively impacts my software, I find a workaround by using something else. period.

Share this post


Link to post
Share on other sites
  • 0
2 hours ago, modplan said:

Chris, if this is an issue with sparse files on NTFS, perhaps cloud drive should not be using them and use something else instead? I write software for a living, and if a dependency is broken in a way that I cannot fix and that negatively impacts my software, I find a workaround by using something else. period.

Sparse files are well documented, and well used.  The Page File is one, VHD(x)s use them, Windows Backup uses them.  

So, the question is, what do we use instead?  And will that alternative work any better, and will it have any other impacts (performance, functionality, resource usage, etc).   

And the reason that we chose sparse files in the first place, was that it was the best option, by far, for what we're doing. 

So, the option would be to rebuild a huge section of the code to ... potentially not even fix the issue, or leave it be (and maybe/hopefully revisit it later).


And... unfortunately (because I really, REALLY hate to say this), is that it seems to affect a very, very small percentage of systems. And it doesn't appear to be any specific trigger for the issue (eg, there is no way to reliably reproduce the issue).  So, because we are a small company with limited resources, it doesn't make sense to rebuild like 1/3 of the product to fix the issue.  Especially since it took so long to get a release out, anyhow.

Share this post


Link to post
Share on other sites
  • 0
2 hours ago, modplan said:

Wow a blast from the past. Still following this thread apparently, got an email notification

If you want/need to, scroll to the top, and click on the "Following" button/badge/thing.  That will unsubscribe you from the thread.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...