ironhead65

I feel like I'm not helping myself enough. I'm not able to locate this, so maybe it doesn't exist...although the last few times I've looked and posted about various issues, Chris was able to point to some documentation... Is there a way to control functions / features using command line for CloudDrive? Specifically I want to be able to toggle the checkbox for the Upload Threads Count. My thought is, until the Global Controls are added, when I know I have a HUGE dump of files, I can just toggle ALL Upload Threads to 0...well, unchecked. Then, proceed to enable the drives one at a time. I suppose if I could get the statistics of each drive, I could automate this. Basically, check the drives status (how much left to upload). Once that is 0, then enable the next drive. That would be easy enough to do in script form. Thanks!

Another update, I can't believe how FAST this is going now! I waited at least 1 week, I am thinking closer to TWO weeks and got a total of maybe 50GB uploaded to the cloud services I'm tied to. At this time, I've been at this with the "upload threads" unchecked for 6 hours and I've dumped well over 3GB! That is more progress in so many hours than I could have HOPED for in 1/2 a week before!! --This is assuming: 50GB / 14 days = 3.57GB/day BEFORE 3GB/6hour * (24 hour / day) = 12GB/day NOW! This is AMAZING!!! I can now actually WATCH the progress update in the Drive PIE diagrams!! Now I REALLY want have to figure out how to maintain my sanity as I wait for the "Global Threading" or "Global Round Robin" feature to be added! So exciting!! I feel that I am mamually simulating what the feature would end up doing!

Wow, thanks again to both of you! I am not making some SERIOUS headway in getting my Cloud Seeded! I had 5 drives with about 80-100MB of data waiting to be uploaded. By targeting those first, they finished within MINUTES!! Each being given 5+Mbps and they just RIPPED through that data! Now on to the other drives one at a time! This is a beautiful setup now!!

OK, I just remoted back into that Virtual Machine - having left most of the Upload Threads Unchecked. Anything that was unchecked was NOT uploading any longer. The few that were, were certainly splitting up my bandwidth MUCH better (I'm guessing), since scrolling through (left/right arrows showing the different drives) showed only those two with the Uploads selected as actually uploading. They also showed about 5Mbps between the two of them. Normally, I only see about 300-500kbps on maybe 7 drives - which is a bit low for my link of 5Mbps Total. Hence the assumption that I'm being killed in overhead traffic at the moment. I suppose that I was too impatient and since they are THREADS, they just completed before they actually were removed? That makes sense to me at least! I also will say that this seems to have made that Virtual Machine happier as well. I can now remote into it without a TON of lag. The machine seems peppier and looking at the Task Monitor, the CloudDrive service is not pegging the processor to 100% any longer. Maybe I had multiple issues? Not enough System resources as well as low link speed? I could see the AES engine needing a LOT of effort due to the MASSIVE overhead I was probably accumulating? I do realize that this was probably more intended to link 1-2 drives / services...and I am abusing that a BIT as I pooled together to many services to get a large drive pool. Thanks for confirming the behavior. I guess I would have seen that next time I remoted into the machine.

Viktor, That did not actually seem to do anything. I'm guessing by unchecking it, the drive was allowed to make many threads instead of being limited to 2? I was not able to enter a 0 into that box. I can also confirm that the Write Enabled checkbox can not seemingly be toggled.

Viktor, That is great! Thanks. I was cautious about playing in the secondary window. However, as this is RIGHT in the main drive pie screen, I'll give it a try. Thanks!

Hello all, In my continued efforts to get the first SEED out, with the combination of all the drives I have as well as my lack of upload speed, first, thanks to Chris (config/settings) and the rest of the community (settings) and I think the latest release (algorithm changes?), at this time, instead of getting ALL of my Google Drives needing to be reauthorized, I get one every say...week or so. That is VERY happy! I was again looking through the options of the CloudDrive software, as I am still waiting on ~100GB of data to be uploaded (a LOT of it in cache at this time). As I wait for the feature request to be implemented (I'm being patient, really!) - where you can set a GLOBAL upload thread limit or at least potentially get a ROUND robin style upload (work one provider / drive, process some amount of data, then move on to the next - which I would HOPE will provide more targeted bandwidth instead of fracturing my meager upload speed (5mbps/50mbps cable modem) into nothingness as I get overtaken by overhead). Regardless, I noticed that under Technical Details, located next to the drive selection pulldown there is a checkbox for allowing "writing to this drive". As this has been running for over a week and I do not want to bugger anything up, I thought I would ask what this does. If I go through ALL my drives and UNSELECT this, will this put the processing of the drives in a paused state? This would allow me to go through one at a time enabling this. Let that one drive finish, move on to the next, etc. Then, once the seeding is done, enable all drives and let it manage things as it needs. Backups shall not be very large as it will be differences (~20-50 MB at a time, vs the 177GB I'm seeding with). Does this sound like a reasonable action? Or is this an abuse of a troubleshooting tool that will hose things? Just as a reminder, I have 20 Cloud Drives. Using BOX, Google Drive (not Enterprise Drive), Dropbox. Before the latest release I also had Mediafire in there. Those drives are pooled together using Drivepool. As I understand it, DrivePool is happy as the cache is taking up the slack. From CloudDrive's perspective, it is trying to upload as fast as it can across EVERY drive, which I think at this point, my drives are tolerant enough (again, thanks to the settings from the community), but I think the overhead of hitting EVERY drive is slowing things to a crawl. If we had the global threading, I would just set that to 1 - 3 to start with. As we do not, I'm trying to see if that checkbox will give me this option at least to finish up this initial seeding. Thanks again to everyone for their help!

Along with this, is there a way to setup a round robin of sorts? I have been working to overcome some strange lagginess and believe that through my testing that lag will go away once I have finished seeding my drivepool (made from Clouddrive drives). Until then, instead of having to re-authorize my Google Drive or other drives once a day (as the pool continues to seed), can I setup a global threads limit for upload? This way all other drives will just have to wait their turn, instead of splitting down to <100kbps each, then erroring out due to "slow" speeds. I realize this will KILL my performance overall - however it will keep the drives just happily dumping their cache to the drives in the cloud. Since my home network is slower than most on here (5/50), I do not think I'll run into issues such as listed above. For me, I am using this as a backup, so performance is NOT high on my list and due to the advanced caching, even while the drives are "waiting" to finish dumping, I can seemingly continue to run my backup every day and the Clouddrive just seems to be able to handle the differences - all the while trying to catch up with the filled up cache. If needed, I'll certainly submit for a feature request on this one.

I shall also state again that I have not played with this in a long time. It looks like I got an answer about the freezing here: http://community.covecube.com/index.php?/topic/2517-file-copy-very-slow/page-1&do=findComment&comment=18476 Although I think moving it to a separate machine where it can freeze all it likes and it won't matter (per the above description). I'll mess with the settings just to see if they help the freezing as that would be the best option from a resource utilization perspective.

Sorry for my late reply: It was specific to MediaFire. I have multiple Google Drives, Multiple DropBox drives. I have not tried this with the most recent version of the CloudDrive install. This was a beta from back when I posted this question. Has anything changed that I should try to test this out?

Hi there, I've been trying to organize my CloudDrive and Drivepool among my servers and struck upon what I thought was a good solution. I have two licenses of DrivePool, one CloudDrive and two Scanner installs. One File Main Server. It runs Windows Home Server 2011. I have DrivePool on that, with scanner. I have 8 SATA drives and 1 iSCSI target (Nas4Free). These are pooled into 1 drive. The machine is pretty powerful resource-wise (Xeon X3440 @ 2.53GHz), 16GB ram, running ESXi. The WHS install has 8 GB and 6 cores associated with it. Along with being my data-tank for all my computers in the house, this machine is also my Serviio video transcoder over DLNA. It works really well in this setup. A while ago, I purchased CloudDrive. I installed this on this machine - having made a snapshot prior to installing as we were still in beta at the time. I played around with it and had setup enough CloudDrives using Google Drive, Box, DropBox, etc, in order to see the value. I saw someone post saying they could use CloudDrives INSIDE DrivePool. This was a great idea. I pooled my drives together and saw that I had a massive Cloud based Drive! What I saw happening though, is at least in how I had things setup the drive would constantly require re-authorization of my drives (beta?), but the issue I ran into was the system PAUSING. So much so that it was difficult to watch ANYTHING transcoded. The system would just pause and you wouldn't have access to the video stream. Nor would you have access to any shared folders from the server. I turned the CloudDrive service off in the services manager. This resolved ALL my pausing issues. I set this aside for a while as I did not have time to mess with it. Come back to this a few days ago. I added a new server into my servers. On this new machine, I also installed ESXi. I added a 10G link between the two machines and setup a new LAN link in ESXi. The intention is to move the iSCSI drives from the OLD Asus Easy Home machine into this new machine, giving my existing drivepool a much needed speed boost (the 1GB link on this seems to cause most writes to the file pool to be as slow as that one link). On this new machine, I created a new VM for Windows Server 2008r2. I installed CloudDrive and Drivepool. This would hopefully allow me to have the pool, but have any performance issues with it NOT cause any issues with my existing setup. Along the way I wanted to mess with StarWind's iSCSI target, which can do RAW drives. My hope was to be able to use an iSCSI target to connect this pool, or the CloudDrives themselves back to the original WHS server so I can manage my pool there - or at least have access to the drives directly. So, here is the question, in StarWind's software, it is NOT detecting any of the CoveCube drivers / drives / etc. as a real drive. It only sees the virtual drive the OS is installed on. Is there a reason for this? I thought the drives were as real as you can get without actual hardware? Maybe this is a question to pose to StarWind as to WHY their software does not see the StableBit drives? Maybe my pausing issues were caused by needing to create a large cache? As I intend this pool to just be a backup of all my most precious data, My settings for ALL drives were Minimum and FIXED. I did not want ANYTHING cached as I wanted it ALL out on the cloud.

ufi56, With the last release I am seeing similar - where my machine seems to freeze...A LOT. When I FIRST purchased the software, I had set it up and was seeing some throttling issues. I attributed that to the size of my Cache drive and the throttling that needed to happen to keep things happy. Then, sometime last week (I do not recall the day), I was offered a new version. Realizing this is still beta and that we were coming close to the cycle - I figured why not? In my setup: I have 13 clouddrives setup. 1 box.com, 2 dropbox, 10 Google Drive (not the paid cloud drives, but the provided ones for personal use). I pooled them using DrivePool and disabled any replication / balancing / etc. Just act as a single large drive putting files where ever there was space. With this version, I now have to re-authenticate to each drive nearly every 12 hours! v1.0.0.846 My fileserver (Windows Home Server 2012) was getting so bogged down and seemingly SLOW that I actually just let all the drives finish asking for re-authentication. At this point, more than 1/2 of my drives are sitting waiting to be authenticated again - BUT my machine is now stable or responsive again. I am assuming something in the kernel level was waiting on something and instead of giving up the processor while it waited, it just sat there hogging things. BTW, DrivePOOL is unhappy with me at this time as most drives are not there. Good thing this is not critical data! --Dan

Christopher, As always, thank you! The awesome support provided by yourself and anyone else I've worked with on the forums is one of the reasons why I continue to purchase this software! --Dan

I'm hoping this thread can "pin" together all the best settings that everyone finds works for them. If I leave anything out, please comment and let me know. I think that knowing the number of threads (upload/download), speed of your internet connection (upload/download), anything special about it (in my case I have two connections that are pooled using pfSense), size of the drive, chunk size, if you are using drivepool, number of drives in the pool, services. The reason I think this is useful, I have essentially left everything at defaults (but for a few things) and am getting the sense that with my exact configuration a 10MB chunk might be too large. Maybe it is the VOLUME of data I tried to push at one time, but using the Technical Details page, I see a lot of chunks get 10-20% uploaded, then fail. It "SEEMS" that the chunk is resumed, but again, I get 10-20% more and it fails again. I'm not sure this is good for the system, so would a 1MB chunk work better? I plan to test this in the future, but at this time, I'm still trying to get 1 final successful DUMP to see how well my backup strategy will work using my various cloud drives pooled together. Update: after STOPPING the data flowing INTO the cache (stopped my copy of files) waiting for drive response - maybe your internet is too slow throttling messages went away. I guess the defaults seem ok for now, it just does not like to be FLOODED with a TON of data all at once. I think what also helped, as I am using this as a CloudPool (layering Drivepool on top of these drives), I disabled all balancers for this drivepool. I thought, I'd try that as it might have been trying to move files around while it was still uploading the flood of data into the pool. I decided that I could always re-enable them later on if I desired. For now, I just wanted the logical "one drive access" to the entire pool of the many cloud drives I have setup. So, is there settings that you have found that seem to work really well for you? If so, could you please post them? I'd certainly like to mess around with the settings in order to tune for optimal use. I thought I would start by standing on the shoulders of giants, listen to those that have experience with this already! Thanks!

I was playing around with settings and noticed that I could enable beta providers. I see that MediaFire is now included. I tried to enter more than 1 account and it failed. Is this intentional at this time, or shall I just wait for the next beta / submit this as a problem ticket? Thanks again for such awesome software!

Hi there, Awesome software - as always! I've been using Drivepool and Scanner since nearly the beginning. Currently I have something like 22 drives in my main drive pool. They range from iSCSI, SATA-attached, USB (depending on the pool), Ximeta-attached (they have a custom network thing they do), or even virtualized under ESXi. Anything that shows up to Windows as a physical drive can just be pooled. I love it! Recently purchased Clouddrive, after messing around and some Google Searching of this forum, I think I'm fairly setup well. I have 13 clouddrives setup. 1 box.com, 2 dropbox, 10 Google Drive (not the paid cloud drives, but the provided ones for personal use). I used all defaults for everything, except set the cache to MINIMIAL+ Encrypted it, set it to auto login to that encrypted drives (as I only care that the CLOUD data is encrypted...I mean, you want to look at my pictures of my kids THAT bad and break into my PC to do it...okay, enjoy, you earned it). Pointed to a specific 100GB hard drive partition that I could dedicate to this (using drivepool on all other drives and one specific thing mentioned was the cache drive could NOT be part of a drivepool) renamed it removed the drive letter and set it to a folder name (use this with drivepooling to cut down on the displayed drive letters for a clean look) I am getting a slew of "throttling due to bandwidth" issues. I admit that my cache drive is probably too small for the amount of data I DUMPED and will continue to monitor this as I do not feel that I was getting those messages when I did not just DUMP enough data to fill the ENTIRE CloudPool in one shot. So, my request is to have a view in the program to look at all drive upload/download at the same time. Maybe even space? I love the existing charts. They are easy to look at, easy to read and understand. I also like the "Technical Details" page as that shows a TON of information, such as the file - or chunk - and how much of it is uploaded / downloaded. I'm wondering if there is a way to view all drives at once? I would use this to get a sense of the overall health of the system. That is, if I have to scan through all 13 drives, I do not see where my bandwidth is being consumed to understand if the cache drive is FULL or if I am having Upload/Download issues. This reason, by the time I click through each drive, I do not see where the bandwidth is being consumed as the bandwidth seems to shift between drives fast enough that I do not see a true representation of what is going on. I'm sure part of that is redrawing the graphs. I find the technical details page much more useful, as I do not see what is LEFT to upload, but I get a much faster idea of what is going on and although annoying to click through ALL the drives, it seems to be giving me a better idea of what is going on. I think that having an overall page would be fantastic. Thank you again for continuing to show what is possible! --Dan

Going along with the theme of this and not finding my specific answer I'll ask here. I'm a long time user of StableBit Drive Pool and Scanner. Best money I've spent on my home server. In the past 6 months, I've used GDrive and other types of programs to access Google Drive / DropBox using scripting to try to pool multiple accounts together to do an off-site snapshot. I found managing the pool difficult, however LOVED that I controlled every aspect. I was able to pre-encrypt the data using OpenSSL as well as copy files around from my Local Drive Pool, to the other drives. I found that my algorithm was lacking and tests showed that I could create multiple links to multiple accounts to increase speed, however keeping the data balanced was really hard. I give you credit for handling that! I decided I just filled them 1 drive at a time and decided speed was not really a factor for me. The only thing I needed to worry about was making certain I did not try to stick a file on the cloud drive that was too large for the remaining capacity. Having had that experience, I know that at least GDrive has had to have an update on numerous times due to Google Changing their API for accessing the Drive (unlike WebDAV which ... well all WebDAVs I've used by anyone seems to behave the same). Thinking about how paranoid I get about recovering data, I'd like to toss out a situation that I hope NEVER happens. Consider that I would like to use a mix of 1-2 DropBox and 5-6 Google Drive accounts. I would map all of these into DrivePool on a new Virtual Machine setup JUST for cloud access (we all bought at least two licenses of this, didn't we!? I know I did and the second one sits doing nothing since I bought it). Thinking about MY setup with the scripting, if I need to access a file and GDRIVE or whatever I was using in the script stopped working (API change), I would first figure out WHICH file I needed to access (remember encrypted), then I could download it like a normal Google Drive / DropBox access. Lastly, once it hit my hard drive, I could decrypt it. Assuming that StableBit cannot or will not, for whatever reason (directive from Google due to their license changing?), update the access to Google Drive, can I access my data files through the Google Drive interface? I'm assuming YES. Now, let's further this if I use the FILE or FOLDER level encryption, can I download the files by hand, place them into the local cache, then access them? Can you think of any way to recover these files? -assuming that nothing is wrong with any part of this except that the CloudDrive software just cannot access the files (again, API change, or whatever). I'm assuming that if I use WHOLE drive encryption the files are lost. I would like to have this MIRROR files that are in specific folders on my existing DrivePool server. I do not want them moved to a "local cache" - except if necessary for uploading/downloading. All of the accelerator / precaching etc. is not necessary for me and would really just waste hard drive space. Is this capable of that? Thanks for all of your support and MANY years of great service and use! --Dan