Jump to content
Covecube Inc.
  • 0

DrivePool integration questions


Question

How does CloudDrive integrate with DrivePool with regards to read striping on duplicated files? Will CloudDrive be pulling in data to try to satisfy uncached read requests from DrivePool, despite the whole file being allready available on another disk? Also should the network connection go down, assuming the CloudDrive is only being filled with duplicated files, would there be any errors thrown, or would all writes just go into the cache for when the network comes back up, and reads go to the local duplicated copy of the file? Hope that makes sense!

 

Also, would sudden power loss or a system lock up cause any loss of data, moreso than a standard NTFS disk on power loss, or would the cache just carry on from where it left off?

Link to post
Share on other sites

9 answers to this question

Recommended Posts

  • 0

http://community.covecube.com/index.php?/topic/1226-how-can-i-use-stablebit-drivepool-with-stablebit-clouddrive/

 

That said, if you create a CloudDrive on the pool, you can absolutely use duplication still, and gain it's benefits.  In fact, the read striping feature would still help out here.

 

As for the network going down, it depends on the provider and the settings.

However, the cache would be used still, until it's able to upload the files. 

The cache is an absolute fallback, in case we can't access the provider. 

 

As for power loss, BSOD and the like.... yes, it would just carry on from there. And if needed, a simple chkdsk may fix most issues that would occur. This was one of the things we definitely tested and made sure that it behaved in a reliable way.

Link to post
Share on other sites
  • 0

Well something went wrong overnight last night. The computer was semi-unresponsive and needed to be hard reset when I checked on it this morning. My 3TB CloudDrive with about 250GB left to upload is now only a 50GB drive with 800mb left to upload, and DrivePool is reporting the disk as missing causing the pool to be read only. I'm trying to work out what happened atm (unsure if the CloudDrive disconnected overnight, leaving the pool read only and then causing the computer to hang).

 

After detaching (not forefully) the CloudDrive (the computer kept failing to respond with it still attached) and reattaching it, it now says its a 3TB drive again, and is accessable through explorer. I imagine there is significant data loss though now that the 250GB of 'to upload' data is gone. Prior to detaching it there was still a ~300GB cache folder which disappeared when it was detached, which I assume stored the data to upload as the cache size was only set to 20GB.

 

I'm not sure if things went wrong before or after the reset. Yesterday evening the clouddrive was a 3TB disk, and after resetting this morning it was saying it was only 50GB (though after reattaching it reports as being 3TB again). The log files for both CloudDrive and DrivePool were last written to at 1.04am, however the computer remained powered on until about 9am this morning (though no further disk writes had occured anywhere else either). The computer didn't completly lock up, (moust was moveable) though it didn't respond to ctrl+alt+del (still responed to pings, though couldnt remote desktop in, or remotely connect to clouddrive/drivepool). I'm wondering if it was waiting for a disk read/write that couldn't be fufilled or something.

 

Summary:

* Last disk write was around 1am, though computer remained semi-responsive.

* Unsure why it locked up - possibly waiting for disk read/write as no other files were written beyond 1.04am

* Hard reset at 9am this morning - 3TB clouddrive (250GB left to upload) is now 50GB (800mb left to upload).

* Clouddrive is 3TB after reataching, though I assume significant data loss.

 

FWIW The clouddrive was originally set at 50GB when created, though then had been expanded to 3TB and had been at 3TB for several days. Unsure if this might be a potential culprit.

Link to post
Share on other sites
  • 0

I'm not sure if this is the exact order of events that caused my problem last night, but I've just reproduced something similar.

 

* Create a new 1GB cloud drive

* Copy ~900mb of files onto it

* Resize to 10GB while the 900MB is still uploading

* Wait for resize to complete, but while its still uploading the 900MB, hard reset the computer

* CloudDrive now reports the drive as being 1GB - though in this instance windows explorer reports it as being 10GB still. I'm not sure of data loss.

 

I'm now trying to reproduce it with larger sizes to see if I can get the 'to upload' to be lost again.

Link to post
Share on other sites
  • 0

Tried reproducing it again with more data - this time it fails to reattach after recovery.

 

* Create 10GB new cloud drive

* Copy 9GB onto it

* Resize to 30GB while the 9GB is still uploading

* Copy another 9GB onto it (so theres more than 10GB waiting to upload)

* Hard reset while its still uploading

* Drive recovers (saying its a 10GB drive)

* Drive fails to attach, saying its allready attached.

 

Seems something goes wrong after a hard restart after a drive is resized and data is still being uploaded. I'm not sure if it also happens if all data has first been uploaded after the resize but before the hard restart. Hopefully you guys can reproduce it.

 

*edit* Also had 2 BSODs today while playing with clouddrive. Both were apc_index_mismatch errors, and while Im not certain they were caused by clouddrive, the same error occured on two different devices - I've never had that error before. The first happened during a normal restart after windows updates were installed, and the second upon destroying a cloud drive.

Link to post
Share on other sites
  • 0

I have a similar issue with my computer freezing, though I have not resized the drive.

Windows 7, 64 bit, stablebit cloud drive version 1.0.0.283.  16GB RAM and 8core intel cpu. Using amazon clouddrive as the target.

Have tried it with default worker / maintance process and with increments of 100 up to 1000.  Have default cache size and all the way up to 10GB of cache, on a local SSD.

Files are being copied from a 7200 RPM sata HD. Cloud drive size doesn't seem to matter, 10GB, 50GB, 1TB, 5TB, 10TB.

Have 950mbit download, 650mbit download capabilities.

Encryption is enabled, default settings.

 

Process to recreate, and have done this on two different computers:

  • Create cloud drive
  • Copy 900MB video file
  • Successfully uploads and can retreive fine

 

  • Create cloud drive
  • Copy >10 x 900MB video file
  • Can see in Amazon web gui that it starts uploading the files
  • Windows explorer crashes and and does not recover (Ctrl-alt-del and starting explorer.exe does not recover it)
  • Non-microsoft applications freeze, IE firefox and stablebit cloud drive, showing "Not Responding" over the window.
  • Wait 15-30 minutes and it does not recover
  • Hard reset computer
  • Stablebit recovers the drive fine, shows data in the "to be uploaded" but it does not continue  (if add new data it will upload the new data but not the old copy data before hard reset)

 

Have this problem on any data larger than about 6GB if it is more than 1 file.

 

Have uninstall/reinstall stablebit and performed reboots between those actions.

Tried it with creating a new stablebit clouddrive as well.

 

Only have amazon clouddrive to test.

Link to post
Share on other sites
  • 0

Would you like logfiles of the drive resize/hard reset issue too? It's not a crash as such, and is easy enough to reproduce.

 

I haven't been able to reproduce the semi-responsive crash that happened though (its happened once since). I'll try sbcd's steps later to see if it can trigger it. Also I haven't been able to trigger the apc_index_mismatch BSOD either - all I know is it occured once when destroying a cloud drive, and once when restarting.

Link to post
Share on other sites
  • 0

Regarding the drive resize issue, 'enable disk tracing' turns itself off after the hard restart, so Im unsure how complete logs would be. It does reproduce reliably every time on multiple devices, so hopefully you can reproduce it allright.

 

As for the system lock up (assume its a different issue), I think it only happens when the clouddrive is added to the drivepool. I can't reproduce the crash sbcd had - I assume mine is different. Its only a semi crash - for instance the mouse keeps moving, media keeps playing, and the system responds to pings, howver I cant ctrl+alt+delete or remote desktop in to recover it and need to hard restart. I can't reliably reproduce it. The last one happened while duplicating - I have the cloud drive set up as a duplicate only drive though am unsure if this is relevant.

 

After recovery completed following the previous lock up and clouddrive was 'pinning data' I did a normal restart and got an apc_index_mismatch BSOD prior during shutdown - not sure if this is related or another bug. I'm not sure if its drivepool or clouddrive at fault. Regardless, I'm currently uploading the BSOD memory dump via the linked form.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...