Jump to content
Covecube Inc.
  • 0
Tell

CloudDrive is syncing, but data is still shown as local

Question

Hi all!

 

First-time user of CloudDrive, still in the trial. Considering going for the bundle but want to make sure that it works before I purchase.

 

I'm wondering wether CloudDrive is actually moving any data or not. I'm trying to get this to work with Amazon Cloud Drive (yes, experimental and throttled, I know). I've created a 1 TB drive with 20 MB chunk size and a 100 GB local cache. After the drive was created I almost filled it up, most of which I then deleted whereafter I have almost filled the drive to about half capacity (the point here is that I'm estimating that I've totalled more than 1 TB of writes to it). The last writes to the drive finished about five hours ago.

 

CloudDrive has been showing various upload speeds between 1 Mbit/s and 10 Mbit/s since I started writing to the drive. The cloud provider's web interface confirms that I've so far managed to upload about 80 GB of chunks. The right-hand pie chart in CloudDrive is still showing all of my data as "Local" and nothing as stored in the cloud. The left-hand pie chart shows "To upload: 1 TB". I've tried turning "Background I/O" both on and off, but I haven't seen any difference (not entirely sure what this toggle does anyway). Should I not see about 80 GB of data starting to appear in the cloud here?

 

Thank you guys for any feedback.

Share this post


Link to post
Share on other sites

8 answers to this question

Recommended Posts

  • 0

This is a rather complicated question, because there are a lot of moving parts here.

 

 

However, specifically, how much data resides on the "physical" drive as reported by Windows ("My Computer"/Computer/This PC)?

 

Also, any files uploaded to the provider should appear in a "StableBit CloudDrive" folder, and you can view them there.

 

Also, especially for the Amazon Cloud Drive provider, we use upload verification. This means that after a chunk is uploaded, the software automatically re-downloads the file and checks it.  If it failed or doesn't match the checksum, the file is reuploaded again, until this completes successfully.  

 

If this runs into a lot of issues, it may not have uploaded much. And it sounds like this may be the case, if none (or almost none) for the data has been uploaded to the provider (Amazon Cloud Drive). 

 

 

Also, Also, what version of StableBit CloudDrive are you using (it's listed at the bottom of the UI)?

Share this post


Link to post
Share on other sites
  • 0

Hi Drashna!

 

Thanks for getting back to me.

 

I actually didn't make CloudDrive format it and assign a drive letter; I had the volume full-disk-encrypted with VeraCrypt which I then loaded with some test data. I take it that this means that absolutely every bit on the physical drive has been touched at least once in addition to the data I later wrote to it. The volume has been unmounted (untouched) for the last 24 hours. Is CloudDrive filesystem-aware, or is the drive treated as a block storage device?

 

The hidden "CloudPart.xxx-"-folder on one of my drives shows that both "Size" and "Size on disk" is the same and equals the drive size at 1 TB.

 

CloudDrive has been running continuously since my previous post. It has shown some "blue" and some "orange" notifications, but as they are infrequent I assume that these are the result of normal network bottlenecks and the like. The right-hand pie chart still shows all data as "Local". The Amazon Cloud Drive web interface shows that I have uploaded about another 80 GB of data for a total of 160 GB used space in the cloud.

 

I'm running version 1.0.0.463 BETA.

 

Thanks a lot!

Share this post


Link to post
Share on other sites
  • 0

Is CloudDrive filesystem-aware, or is the drive treated as a block storage device?

 

Yes.  By default, it's treated as a block storage solution.  However, the software is file system aware (at least for NTFS) and attempts to pin the MFT and other NTFS data (including directory entries). 

 

However, using VeraCrypt may interfere with this, depending on exactly what is going on. 

 

I'd say it may be an interaction, but if the drive has been unmounted (in VeraCrypt) and essentially untouched, it should be uploading, even slowly. 

 

 

 

The hidden "CloudPart.xxx-"-folder on one of my drives shows that both "Size" and "Size on disk" is the same and equals the drive size at 1 TB.

 

You have said that the drive itself is 1TB and that the cache size is 100GB, right? I'm pretty sure that's the case (as listed in the first post). 

 

The size on disk is right, if it's not really uploading, or has only uploaded very small amount, as it will be kept on disk until it's able to be uploaded. 

 

 

Though, a good question here, is what is your upload speed? 

Share this post


Link to post
Share on other sites
  • 0

Yes.  By default, it's treated as a block storage solution.  However, the software is file system aware (at least for NTFS) and attempts to pin the MFT and other NTFS data (including directory entries). 

 

However, using VeraCrypt may interfere with this, depending on exactly what is going on. 

 

I'd say it may be an interaction, but if the drive has been unmounted (in VeraCrypt) and essentially untouched, it should be uploading, even slowly. 

 

The VeraCrypt volume should still resemble an NTFS volume in layout, so if CloudDrive pins the area (beginning?) of the block device that is normally used for the MFT and other metadata it should give the sought-after performance improvement. The VeraCrypt documentation states that encryption offsets the cleartext drive blocks by 131072 bytes in the ciphertext. There is also some trim from the end of the volume. Anyway.

 

You have said that the drive itself is 1TB and that the cache size is 100GB, right? I'm pretty sure that's the case (as listed in the first post). 

 

Correct.

 

The size on disk is right, if it's not really uploading, or has only uploaded very small amount, as it will be kept on disk until it's able to be uploaded. 

 

The Amazon Cloud Drive web interface reports that I have uploaded 340 GB now (my Amazon Cloud Drive is empty save for this CloudDrive disk). This has increased from the original 80 GB i reported in the first post. Clearly, CloudDrive has spent a lot of time and bytes uploading something, but apparently it's not reported in the UI or acknowledged by reducing the "size on disk" of the CloudPart.xxx-folder.

 

The StableBit CloudDrive UI still shows no data in the cloud (right pie chart is all-local) and the left pie chart shows "To upload: 1 TB".

 

Though, a good question here, is what is your upload speed? 

 

80 Mbit/s from the ISP. The upload speed to Amazon in the StableBit CloudDrive UI varies between 2 and 15 Mbit/s.

 

At this point I will destroy this disk and re-create it to test if there is some improvement from a second attempt. Maybe I will test with a smaller disk to see if there is any change.

Share this post


Link to post
Share on other sites
  • 0

I hate to ask, but could you see if this behavior occurs on a disk that isn't protected by VeraCrypt or the like? 

 

If it doesn't, then it may be an interaction between the two that is causing the odd behavior.  

I'll be seeing about replicating this issue, as well, but on the chance that it's hardware/ISP specific...

 

And if it is still showing the "to upload" and no cloud content, then something is clearly going wrong here. 

Could you enable logging and let it run for a bit?

http://wiki.covecube.com/StableBit_CloudDrive_Log_Collection

Share this post


Link to post
Share on other sites
  • 0

I hate to ask, but could you see if this behavior occurs on a disk that isn't protected by VeraCrypt or the like? 

 

I'll test that.

If it doesn't, then it may be an interaction between the two that is causing the odd behavior.  

I'll be seeing about replicating this issue, as well, but on the chance that it's hardware/ISP specific...

 

I created a new 500 GB disk without formating or assigning a drive letter, then whole-disk-encrypted it with VeraCrypt again. I did not fill the encrypted volume with data. This means that the entire CloudDrive has had ciphertext (of cleartext 0) written to it exactly once. It began uploading immediately, and the StableBit CloudDrive UI reports that over 91 GB has been uploaded since my previous post.

 

To isolate the problem further, I will destroy this drive and then re-create and fill it with data (so that it is written to twice). In the interest of time I think I will do it with a 250 GB disk.

 

And if it is still showing the "to upload" and no cloud content, then something is clearly going wrong here. 

Could you enable logging and let it run for a bit?

http://wiki.covecube.com/StableBit_CloudDrive_Log_Collection

 

I will do that after conducting the 250 GB filled cleartext test.

Share this post


Link to post
Share on other sites
  • 0

I created a test with a 200 GB disk. I the 200 GB disk on Amazon Cloud Drive, with a local cache size of 1 GB and the other options set at their defaults except for not formating and assigning a drive letter.

 

I then formatted the it as a block device with VeraCrypt (one full pass of ciphertext) and filled the disk with cleartext data (a second almost-complete pass of ciphertext). StableBit CloudDrive began uploading, and the Amazon Cloud Drive web interface confirms that I have so far uploaded about 90 GB of chunks. This gave the same behaviour in the CloudDrive UI as seen previously - take a look at http://imgur.com/Zqhlo7H

 

The screenshot is taken after about 24 hours of uploading since the last time I unmounted the VeraCrypt volume.

 

I will destroy the drive, enable logging and recreate the problem.

Share this post


Link to post
Share on other sites
  • 0

Okay, I've reproduced the issue, and I can definitely see what you mean. 

 

It is like it's getting stuck uploading.

 

 

However,as for the pinning the metadata, it won't. This is because the actual drive appears as an uninitialized disk in Disk Management (or diskpart), and VeraCrypt does some "trickery" to display the drive. 

 

 

I've flagged the issue for Alex (the Developer) to take a look into. 

https://stablebit.com/Admin/IssueAnalysis/26031

 

 

In the meanwhile, StableBit CloudDrive does feature full disk encryption:

http://community.covecube.com/index.php?/topic/1269-full-drive-encryption/

 

And you could use it with BitLocker (which we have confirmed does work with CloudDrive. 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...