Ok, I really wanted to like CloudDrive. The concept is pretty awesome, but unfortunately, it's just too unstable for my liking to consider using it any longer. In the two months I've been using it, I've had numerous "failures":
I've had my drive (Using Google Drive) require a complete check and reupload of all the cached data (Which took the drive offline for ~17 hours), and since that is part of my DrivePool, it then made the entire DrivePool read-only.
It has also disconnected from Google Drive three times, and it doesn't/didn't auto reconnect, so my server was offline until I could get back to it.
Once it went into a never ending loop of trying to move files off the drive that hosted the write cache for the CloudDrive to the CloudDrive to make space (which then just puts it in the write cache for the cloud drive which is on the same physical disk, so no free space was ever gained). So then it'd try another file. And another. Until I finally caught it with ~4TB of cached data to upload and disabled all the balancers.
Tonight, I went to rearrange one of my raid drives that hosted the cache portion of the cloudrive, and I couldn't use any of the normal utilities to change the cluster size because I'm guessing it didn't like a 63TB worth of sparse files on the drive. It only dawned on me that was the reason the utilities were failing because one of them was saying not enough free space when the drive only had a few TB used on a 9TB parition, and after I gave up, I went to go move the files temporarily to another drive so I could reformat it and put them back and... Explorer came back and said my drive with 6TB free space couldn't hold it. I'll admit that the utilities and file explorer should be able to handle sparse files, but programs that actually use it are so rare, most things don't account for it, and want to write the entire 63TB instead of the 2-3TB that is actually used.
So I gave up, and figured I'd just detach the drive and reattach it on a different drive temporarily instead of copying over the cache files... And when I reattached it, one of the folders was "corrupt". DrivePool told me it was corrupt, chkdsk told me it was corrupt, and now half the 10TB of data I had in the cloud is borked because the folder metadata is semi-scrambled and it can't read the entire folder or any of it's subfolders. CHKDSK can't fix it, and none of the NTFS utils I tried can fix it, but a few can actually see the correct files, but they all want me to copy them to another drive... UGH. It took me 3 months to get that data up in to the cloud, and now half of it has to come back down. Even when I get that half down, I'm still sort of borked because no utility I have can actually DELETE the folder, so it's going to drive DrivePool crazy. The only way to "fix" it (other than sitting down with a disk hex editor and trying to fix it by hand) is to download the ENTIRE 10TB, wipe the drive, and then re-upload it AGAIN.
So far I've tried Paragon's utilities, Acronis's, EASE US's drw, Recovery4all, Auslogics File Recovery, Active Parition Recovery, NTFS Recovery Toolkit, and cgsecurity's Testdisk. Some can see the files, but will only restore to a different drive (Meaning it has to be downloaded then reuploaded), and some will try to find them by doing a "deep scan" (which requires reading the entire 63TB drive). Considering that I only got ~10TB up over a few months, once it's downloaded, I've just decided it isn't worth the headaches, and I'll just buy another 10-16TB drive and be done with it, without any of the extra headaches that come with the cloud drive's quirks.
Oh, and Plex doesn't like it when half it's library disappears/turns read-only, and NzbDrone/Sonarr/Radarr really don't like it -- it sees the files as gone, and then dutifully tries to go get them again, queueing up a zillion downloads that I already have until I catch it and shut everything down until CloudDrive comes back and then spend half a day trying to clean up the mess.
It's possible I would have better luck if I had an array of CloudDrives all with redundant copies, and a symmetrical connection, but even then, it'd have to contend with Google throttling issues (I could only ever get ~150mbs download speed from the drive on a 330mbs/30mbs connection), and with that the drive was very laggy, and causing some other system stability issues -- Opening file explorer taking a whole minute, copies to OTHER drives somehow getting delays occasionally, some random performance issues copying to the write cache occasionally, etc etc. It's CLOSE, but just not close enough for my use case. Again, it might work better as a real-time backup using DrivePool to put multiple copies up on many different CloudDrives to keep the impact of losing a drive lower, and increase the performance with smaller chunk sizes rather than a tiered storage solution I was going for SSD -> RAID -> Cloud automatically moving seldom used files (minus metadata) that it can to slower and slower tiers and moving them back up as necessary. I had hoped with DrivePool's balancing API could I throw together a quick balancer for my needs to do just that and handle some personal edge cases like never ever put 4K video in the cloud because it can't download it fast enough to serve it, and try to keep unwatched stuff to local, and only push watched stuff go to the cloud.
I hope this saves someone the pain I did, and gets them to really think about how they want their system to work, and how CloudDrive might (or might not) fit in that design the right way rather than all the wrong ways I did it.
Question
KingMotley
Ok, I really wanted to like CloudDrive. The concept is pretty awesome, but unfortunately, it's just too unstable for my liking to consider using it any longer. In the two months I've been using it, I've had numerous "failures":
So far I've tried Paragon's utilities, Acronis's, EASE US's drw, Recovery4all, Auslogics File Recovery, Active Parition Recovery, NTFS Recovery Toolkit, and cgsecurity's Testdisk. Some can see the files, but will only restore to a different drive (Meaning it has to be downloaded then reuploaded), and some will try to find them by doing a "deep scan" (which requires reading the entire 63TB drive). Considering that I only got ~10TB up over a few months, once it's downloaded, I've just decided it isn't worth the headaches, and I'll just buy another 10-16TB drive and be done with it, without any of the extra headaches that come with the cloud drive's quirks.
Oh, and Plex doesn't like it when half it's library disappears/turns read-only, and NzbDrone/Sonarr/Radarr really don't like it -- it sees the files as gone, and then dutifully tries to go get them again, queueing up a zillion downloads that I already have until I catch it and shut everything down until CloudDrive comes back and then spend half a day trying to clean up the mess.
It's possible I would have better luck if I had an array of CloudDrives all with redundant copies, and a symmetrical connection, but even then, it'd have to contend with Google throttling issues (I could only ever get ~150mbs download speed from the drive on a 330mbs/30mbs connection), and with that the drive was very laggy, and causing some other system stability issues -- Opening file explorer taking a whole minute, copies to OTHER drives somehow getting delays occasionally, some random performance issues copying to the write cache occasionally, etc etc. It's CLOSE, but just not close enough for my use case. Again, it might work better as a real-time backup using DrivePool to put multiple copies up on many different CloudDrives to keep the impact of losing a drive lower, and increase the performance with smaller chunk sizes rather than a tiered storage solution I was going for SSD -> RAID -> Cloud automatically moving seldom used files (minus metadata) that it can to slower and slower tiers and moving them back up as necessary. I had hoped with DrivePool's balancing API could I throw together a quick balancer for my needs to do just that and handle some personal edge cases like never ever put 4K video in the cloud because it can't download it fast enough to serve it, and try to keep unwatched stuff to local, and only push watched stuff go to the cloud.
I hope this saves someone the pain I did, and gets them to really think about how they want their system to work, and how CloudDrive might (or might not) fit in that design the right way rather than all the wrong ways I did it.
--Robert
Link to comment
Share on other sites
3 answers to this question
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.