CloudDrive with more than one computer.

[EldonMcGuinness]

Assuming one has enough licenses, why is it that CloudDrive can only be accessed with a single computer? I've utilized other cloud apps, rclone, plexdrive, cloudxtender, etc. and they all allow for more than a single computer to interact with the cloud data. Is this something systemic to the CloudDrive application or perhaps I am misunderstanding this limitation.

[Talyrius]

They don't create and mount a drive image...

The easiest solution is to share access to the drive over the LAN or through a VPN.

[EldonMcGuinness]

@Talyrius ahh, so DrivePool CloudDrive creates an actual drive image that it then works with which is stored on the cloud? I would have thought it just saved the files in a sub-directory or something on the cloud drives.

[Jaga]

It creates a larger virtual volume, breaks that volume up into equally sized chunks, and stores those chunks on your cloud provider.  When files/folders change on the virtual volume it updates whichever chunks need to be changed.  When one computer makes changes to the data on the cloud drive, you'd have to have a way to tell the other computer the changes happened, or the drive in the filesystem wouldn't reflect those changes.

It's a little like the difference between POP3 and IMAP email methods.  When you're viewing a HTML interface for cloud storage space, you're just being shown what's out there.  IMAP email connections do largely the same thing - give you a view of what's there.  CloudDrive has to be able to tell the file system what's there, keep a cache locally on it, keep the metadata (file/folder info) on it locally, etc.  That's more similar to POP3, which actually holds email data on the local machine.  One downloads data off the server (POP3), one just shows you what's out there (IMAP).  Due to the way CloudDrive has to work with the OS, it needs to directly interface with the data and keep portions of it locally.

I think it would be troublesome at best to try and deploy CloudDrive on two machines against the same virtual cloud disk.  And we're not even considering features like volume encryption.

[Viktor]

4 hours ago, EldonMcGuinness said:

@Talyrius ahh, so DrivePool creates an actual drive image that it then works with which is stored on the cloud? I would have thought it just saved the files in a sub-directory or something on the cloud drives.

Don't mix up CloudDrive and DrivePool. CloudDrive provides harddrive-like images located in the cloud, while CloudPool is based on hidden directories on its pooled hard or cloud drives.
Like a physical drive a cloud drive can be attached to only one computer at the same time. But you can move it from one to another computers (detach it here and attach it there).

[EldonMcGuinness]

19 hours ago, Viktor said:

Don't mix up CloudDrive and DrivePool. CloudDrive provides harddrive-like images located in the cloud, while CloudPool is based on hidden directories on its pooled hard or cloud drives.
Like a physical drive a cloud drive can be attached to only one computer at the same time. But you can move it from one to another computers (detach it here and attach it there).

Indeed, that was my typo, guess I should not ask questions after my shift until I've had a java recharge.

21 hours ago, Jaga said:

It creates a larger virtual volume, breaks that volume up into equally sized chunks, and stores those chunks on your cloud provider.  When files/folders change on the virtual volume it updates whichever chunks need to be changed.  When one computer makes changes to the data on the cloud drive, you'd have to have a way to tell the other computer the changes happened, or the drive in the filesystem wouldn't reflect those changes.

It's a little like the difference between POP3 and IMAP email methods.  When you're viewing a HTML interface for cloud storage space, you're just being shown what's out there.  IMAP email connections do largely the same thing - give you a view of what's there.  CloudDrive has to be able to tell the file system what's there, keep a cache locally on it, keep the metadata (file/folder info) on it locally, etc.  That's more similar to POP3, which actually holds email data on the local machine.  One downloads data off the server (POP3), one just shows you what's out there (IMAP).  Due to the way CloudDrive has to work with the OS, it needs to directly interface with the data and keep portions of it locally.

I think it would be troublesome at best to try and deploy CloudDrive on two machines against the same virtual cloud disk.  And we're not even considering features like volume encryption.

1

Thanks for all the information. I'm a regular user of rclone and know that it can be used with multiple devices at the same time, encryption or no, and was just surprised that CloudDrive is not the same way. I wonder if I can add a rclone mount to a DrivePool scheme, guess it is time to google-fu.

[Christopher (Drashna)]

Yeah, RCLONE and other tools deal with individual files themselves, reading and writing to the provider IMMEDIATELY. 

With StableBit CloudDrive, yeah, it's more like a disk image (akin to a VHD/VMDK, or the like).  So data is stored in the cache, and then uploaded when it's able to. 

So because of that, what is stored locally may not be in sync with what is on the Cloud provider, which could lead to situations where data corruption would not only be possible, but highly likely.

[EldonMcGuinness]

@christ

1 hour ago, Christopher (Drashna) said:

Yeah, RCLONE and other tools deal with individual files themselves, reading and writing to the provider IMMEDIATELY. 

With StableBit CloudDrive, yeah, it's more like a disk image (akin to a VHD/VMDK, or the like).  So data is stored in the cache, and then uploaded when it's able to. 

So because of that, what is stored locally may not be in sync with what is on the Cloud provider, which could lead to situations where data corruption would not only be possible, but highly likely.

Any chance one can add a mounted rclone drive to a DrivePool instance?

[Christopher (Drashna)]

That depends on how the RCLONE drive works, but I suspect that the answer is "no". 

[darkly]

On 8/2/2018 at 3:25 PM, Christopher (Drashna) said:

which could lead to situations where data corruption would not only be possible, but highly likely.

Can you explain this further?

[Christopher (Drashna)]

When you write data to the drive, it's not immediately written to the Cloud provider,  It's cached locally first, and then uploaded.

That alone is enough.  Until that data is uploading (which can include file system data), the data is out of sync on the provider.   You could have partial files uploaded, given you partial content.  Eg corrupt data. 

[darkly]

On 8/10/2018 at 1:29 PM, Christopher (Drashna) said:

When you write data to the drive, it's not immediately written to the Cloud provider,  It's cached locally first, and then uploaded.

That alone is enough.  Until that data is uploading (which can include file system data), the data is out of sync on the provider.   You could have partial files uploaded, given you partial content.  Eg corrupt data. 

I understand the cache and upload process, but I was wondering what you meant by "situations where data corruption [ . . . ] is highly likely". As long as the data syncs correctly, there would be no issues right? So do you just mean cases where the cached data fails to upload correctly to the cloud, or are there other issues that could present themselves even if the data appears to sync properly. I know there's the upload verification option. Are there any situations that would fall outside of its scope?

[Christopher (Drashna)]

"situations where data corruption" ... if you have any "to upload" data, then corruption can occur, if you try reading the data in the cloud.  

 

Eg, when you have two (or more) systems reading and writing data from the cloud, you'd have to ensure that ALL of the data is written to the cloud before any other system could read from it.  Otherwise, you'll get incomplete data.   And if that incomplete data was the file system, that could show up as a RAW partition. Or, only part of an image or video.   Etc.

Basically, what is on the cloud is NOT the completed data, if there is any "To Upload" data.  So, that really means that any time the drive is written to, it means that the cloud copy is no longer the correct or viable copy of data. 

[darkly]

Ohh, I just realized you were speaking hypothetically as if CloudDrive could mount to multiple machines... I get what you're saying now lol

[Christopher (Drashna)]

Sorry for not making that more clear! 

[EldonMcGuinness]

Just for posterity, there is not a way to have CloudDrive utilize a cloud provider directly, sans container if you will, correct?  This is a major point to me as I would rather be able to manipulate the files directly, instead of having them in a container which prevents me from accessing the data via another platform.

[darkly]

8 minutes ago, EldonMcGuinness said:

Just for posterity, there is not a way to have CloudDrive utilize a cloud provider directly, sans container if you will, correct?  This is a major point to me as I would rather be able to manipulate the files directly, instead of having them in a container which prevents me from accessing the data via another platform.

I feel like this kinda defeats the purpose of CloudDrive. I will say there are other tools that do this already (most obviously, something like Google Drive File Stream, but there are other 3rd party, paid options that work on other cloud providers). Are those not options for you?

To address your issue directly though, I did provide my workaround for this issue on another thread, and the first response on this thread covers part of that: Network share your drive over LAN. What I do in addition to this is run openVPN on my Asus router to access my drive away from home if need be. I also use Solid Explorer on Android to access my files from my phone as well (after using the Android openVPN client to connect).

[Jaga]

1 hour ago, EldonMcGuinness said:

Just for posterity, there is not a way to have CloudDrive utilize a cloud provider directly, sans container if you will, correct?  This is a major point to me as I would rather be able to manipulate the files directly, instead of having them in a container which prevents me from accessing the data via another platform.

Since Clouddrive works on a user-set block size for custom chunks, faster manipulation/transfer, encryption, etc. you'll need to change how this is approached.  CloudDrive isn't emulating space - it's emulating a physical drive, and all of the features it uses really require that it handles this in a structured format (by blocks) and not by files.

What you'd want to do instead is use file duplication/synchronization software (like SyncToy) and have it mirror whatever folders/files from your system that you specify directly to your cloud space.  You'll want to do it by mounting the space as a Drive letter, using the Cloud access software.

[EldonMcGuinness]

15 minutes ago, Jaga said:

Since Clouddrive works on a user-set block size for custom chunks, faster manipulation/transfer, encryption, etc. you'll need to change how this is approached.  CloudDrive isn't emulating space - it's emulating a physical drive, and all of the features it uses really require that it handles this in a structured format (by blocks) and not by files.

What you'd want to do instead is use file duplication/synchronization software (like SyncToy) and have it mirror whatever folders/files from your system that you specify directly to your cloud space.  You'll want to do it by mounting the space as a Drive letter, using the Cloud access software.

Indeed, I just wanted to leave that note so that there was a final answer if or when it gets indexed. I know I asked a few people about this and none of them really knew if it would work. As it sits now I use a combination of rclone, syncbackpro, and drivebender to achieve the desired outcome of having local duplication with a remote backup which can be utilized and accessed from multiple locations. I was just hoping the combo of clouddrive and drivepool would allow me to do the same with fewer components.

[Christopher (Drashna)]

On 8/24/2018 at 12:33 PM, EldonMcGuinness said:

Just for posterity, there is not a way to have CloudDrive utilize a cloud provider directly, sans container if you will, correct?  This is a major point to me as I would rather be able to manipulate the files directly, instead of having them in a container which prevents me from accessing the data via another platform.

No, not currently.