Jump to content
Covecube Inc.
  • Announcements

    • Christopher (Drashna)

      Login issues   11/07/17

      If you have issues with logging in, make sure you use your display name and not the "username" or email.  Or head here for more info.   http://community.covecube.com/index.php?/topic/3252-login-issues/  
    • Christopher (Drashna)

      Getting Help   11/07/17

      If you're experiencing problems with the software, the best way to get ahold of us is to head to https://stablebit.com/Contact, especially if this is a licensing issue.    Issues submitted there are checked first, and handled more aggressively. So, especially if the problem is urgent, please head over there first. 
  • 0
ironhead65

Writing Enabled?

Question

Hello all,

 

In my continued efforts to get the first SEED out, with the combination of all the drives I have as well as my lack of upload speed, first, thanks to Chris (config/settings) and the rest of the community (settings) and I think the latest release (algorithm changes?), at this time, instead of getting ALL of my Google Drives needing to be reauthorized, I get one every say...week or so.  That is VERY happy!

 

I was again looking through the options of the CloudDrive software, as I am still waiting on ~100GB of data to be uploaded (a LOT of it in cache at this time).  

As I wait for the feature request to be implemented (I'm being patient, really!) - where you can set a GLOBAL upload thread limit or at least potentially get a ROUND robin style upload (work one provider / drive, process some amount of data, then move on to the next - which I would HOPE will provide more targeted bandwidth instead of fracturing my meager upload speed (5mbps/50mbps cable modem) into nothingness as I get overtaken by overhead).

 

Regardless, I noticed that under Technical Details, located next to the drive selection pulldown there is a checkbox for allowing "writing to this drive".  As this has been running for over a week and I do not want to bugger anything up, I thought I would ask what this does.

 

If I go through ALL my drives and UNSELECT this, will this put the processing of the drives in a paused state?  This would allow me to go through one at a time enabling this.  Let that one drive finish, move on to the next, etc.  Then, once the seeding is done, enable all drives and let it manage things as it needs.  Backups shall not be very large as it will be differences (~20-50 MB at a time, vs the 177GB I'm seeding with).

 

Does this sound like a reasonable action?  Or is this an abuse of a troubleshooting tool that will hose things?

 

Just as a reminder, I have 20 Cloud Drives.  Using BOX, Google Drive (not Enterprise Drive), Dropbox.  Before the latest release I also had Mediafire in there.

Those drives are pooled together using Drivepool.

 

As I understand it, DrivePool is happy as the cache is taking up the slack.  From CloudDrive's perspective, it is trying to upload as fast as it can across EVERY drive, which I think at this point, my drives are tolerant enough (again, thanks to the settings from the community), but I think the overhead of hitting EVERY drive is slowing things to a crawl.  If we had the global threading, I would just set that to 1 - 3 to start with.  As we do not, I'm trying to see if that checkbox will give me this option at least to finish up this initial seeding.

 

Thanks again to everyone for their help!

Share this post


Link to post
Share on other sites

10 answers to this question

Recommended Posts

  • 0

I think, the setting you are looking for can be found under Manage Drive -> Performance -> I/O Performance.
Unselect the "Upload threads" checkbox. This will stop any data upload for that drive.
(The "Writing enabled" box seems to be an informational item, which cannot be unselected directly.)

Share this post


Link to post
Share on other sites
  • 0

Viktor,

 

That did not actually seem to do anything.  I'm guessing by unchecking it, the drive was allowed to make many threads instead of being limited to 2?  I was not able to enter a 0 into that box.  I can also confirm that the Write Enabled checkbox can not seemingly be toggled.

Share this post


Link to post
Share on other sites
  • 0

Unchecking the "Upload threads" via this link (http://stablebit.com/Support/CloudDrive/Manual?Section=I/O%20Performance) should prevent it form uploading at all. At least for that drive.

 

However, if it is still uploading, then ... that's not supposed to happen, and is a bug.   But check the "technical details" window and make sure that it's not actually uploading. 

Share this post


Link to post
Share on other sites
  • 0

OK, I just remoted back into that Virtual Machine - having left most of the Upload Threads Unchecked.  Anything that was unchecked was NOT uploading any longer.

The few that were, were certainly splitting up my bandwidth MUCH better (I'm guessing), since scrolling through (left/right arrows showing the different drives) showed only those two with the Uploads selected as actually uploading.  They also showed about 5Mbps between the two of them.  Normally, I only see about 300-500kbps on maybe 7 drives - which is a bit low for my link of 5Mbps Total.  Hence the assumption that I'm being killed in overhead traffic at the moment.

 

I suppose that I was too impatient and since they are THREADS, they just completed before they actually were removed?  That makes sense to me at least!

 

I also will say that this seems to have made that Virtual Machine happier as well.  I can now remote into it without a TON of lag.  The machine seems peppier and looking at the Task Monitor, the CloudDrive service is not pegging the processor to 100% any longer.  Maybe I had multiple issues?  Not enough System resources as well as low link speed?  I could see the AES engine needing a LOT of effort due to the MASSIVE overhead I was probably accumulating?  I do realize that this was probably more intended to link 1-2 drives / services...and I am abusing that a BIT as I pooled together to many services to get a large drive pool.

 

Thanks for confirming the behavior.  I guess I would have seen that next time I remoted into the machine.

Share this post


Link to post
Share on other sites
  • 0

Wow, thanks again to both of you!  I am not making some SERIOUS headway in getting my Cloud Seeded!  I had 5 drives with about 80-100MB of data waiting to be uploaded.  By targeting those first, they finished within MINUTES!!  Each being given 5+Mbps and they just RIPPED through that data!

Now on to the other drives one at a time!  This is a beautiful setup now!!

Share this post


Link to post
Share on other sites
  • 0

Another update, I can't believe how FAST this is going now!  I waited at least 1 week, I am thinking closer to TWO weeks and got a total of maybe 50GB uploaded to the cloud services I'm tied to.  At this time, I've been at this with the "upload threads" unchecked for 6 hours and I've dumped well over 3GB!  That is more progress in so many hours than I could have HOPED for in 1/2 a week before!!  

 

--This is assuming:

50GB / 14 days = 3.57GB/day  BEFORE

3GB/6hour * (24 hour / day) = 12GB/day NOW!  This is AMAZING!!!  I can now actually WATCH the progress update in the Drive PIE diagrams!!

 

Now I REALLY want have to figure out how to maintain my sanity as I wait for the "Global Threading" or "Global Round Robin" feature to be added!  So exciting!!  I feel that I am mamually simulating what the feature would end up doing!

Share this post


Link to post
Share on other sites
  • 0

well, I'm glad to hear that everything is working great now!

 

And that's a significant increase in performance!

 

And alex is reworking some of the code, and that may help things further, actually.  

 

 

But global limits would be a good idea. Or at least *per provider*. 

https://stablebit.com/Admin/IssueAnalysis/27562

Share this post


Link to post
Share on other sites
  • 0

Hello all,

 

I had an idea.  I'm going to try to use the Drivepool plugin "Ordered File Placment", which as I understand it will let me run the disks more like JBOD.  If I disable all optimizations for the Drivepool and just let it JBOD...that might resolve my need for the "single install" macro at this time.  I still would like the feature however maybe this will work by keeping most of my updates on one or two drives at a time!

Share this post


Link to post
Share on other sites
  • 0

Hello all,A 

 

little feedback..This MOSTLY works.  The issue seems to be that whenever a do my compare (for mirroring), there is ~768KB of data written to EVERY drive.  As this is the case, the ordered placement does not really help.  If I leave all the drives enabled for upload, as soon as I do the compare, ALL the drives start to hammer the internet to try to push their data out.  The BULK is still in the SINGLE drive (using ordered placment).  However, for the first few hours, that 768KB kills my drives and loops back through the above described undesired process (drives requiring re-authorization).

 

Guess I'm back to using my script mechanism for controlling the upload stream.

 

I will say that as long as I leave 1 drive running at a time, I can pull FULL bandwidth for that 1 drive!  This really is an awesome addon for my network!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×