Jump to content

  • Log in with Twitter Log in with Windows Live Log In with Google      Sign In   
  • Create Account

Photo

Writing Enabled?


  • Please log in to reply
10 replies to this topic

#1 ironhead65

ironhead65

    Advanced Member

  • Members
  • PipPipPip
  • 38 posts

Posted 17 July 2017 - 04:07 PM

Hello all,

 

In my continued efforts to get the first SEED out, with the combination of all the drives I have as well as my lack of upload speed, first, thanks to Chris (config/settings) and the rest of the community (settings) and I think the latest release (algorithm changes?), at this time, instead of getting ALL of my Google Drives needing to be reauthorized, I get one every say...week or so.  That is VERY happy!

 

I was again looking through the options of the CloudDrive software, as I am still waiting on ~100GB of data to be uploaded (a LOT of it in cache at this time).  

As I wait for the feature request to be implemented (I'm being patient, really!) - where you can set a GLOBAL upload thread limit or at least potentially get a ROUND robin style upload (work one provider / drive, process some amount of data, then move on to the next - which I would HOPE will provide more targeted bandwidth instead of fracturing my meager upload speed (5mbps/50mbps cable modem) into nothingness as I get overtaken by overhead).

 

Regardless, I noticed that under Technical Details, located next to the drive selection pulldown there is a checkbox for allowing "writing to this drive".  As this has been running for over a week and I do not want to bugger anything up, I thought I would ask what this does.

 

If I go through ALL my drives and UNSELECT this, will this put the processing of the drives in a paused state?  This would allow me to go through one at a time enabling this.  Let that one drive finish, move on to the next, etc.  Then, once the seeding is done, enable all drives and let it manage things as it needs.  Backups shall not be very large as it will be differences (~20-50 MB at a time, vs the 177GB I'm seeding with).

 

Does this sound like a reasonable action?  Or is this an abuse of a troubleshooting tool that will hose things?

 

Just as a reminder, I have 20 Cloud Drives.  Using BOX, Google Drive (not Enterprise Drive), Dropbox.  Before the latest release I also had Mediafire in there.

Those drives are pooled together using Drivepool.

 

As I understand it, DrivePool is happy as the cache is taking up the slack.  From CloudDrive's perspective, it is trying to upload as fast as it can across EVERY drive, which I think at this point, my drives are tolerant enough (again, thanks to the settings from the community), but I think the overhead of hitting EVERY drive is slowing things to a crawl.  If we had the global threading, I would just set that to 1 - 3 to start with.  As we do not, I'm trying to see if that checkbox will give me this option at least to finish up this initial seeding.

 

Thanks again to everyone for their help!



#2 Viktor

Viktor

    Newbie

  • Members
  • Pip
  • 7 posts

Posted 17 July 2017 - 06:15 PM

I think, the setting you are looking for can be found under Manage Drive -> Performance -> I/O Performance.
Unselect the "Upload threads" checkbox. This will stop any data upload for that drive.
(The "Writing enabled" box seems to be an informational item, which cannot be unselected directly.)



#3 ironhead65

ironhead65

    Advanced Member

  • Members
  • PipPipPip
  • 38 posts

Posted 17 July 2017 - 06:21 PM

Viktor,

 

That is great!  Thanks.  I was cautious about playing in the secondary window.  However, as this is RIGHT in the main drive pie screen, I'll give it a try.  Thanks!



#4 ironhead65

ironhead65

    Advanced Member

  • Members
  • PipPipPip
  • 38 posts

Posted 17 July 2017 - 06:36 PM

Viktor,

 

That did not actually seem to do anything.  I'm guessing by unchecking it, the drive was allowed to make many threads instead of being limited to 2?  I was not able to enter a 0 into that box.  I can also confirm that the Write Enabled checkbox can not seemingly be toggled.



#5 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,208 posts
  • LocationSan Diego, CA, USA

Posted 17 July 2017 - 08:25 PM

Unchecking the "Upload threads" via this link (http://stablebit.com...I/O Performance) should prevent it form uploading at all. At least for that drive.

 

However, if it is still uploading, then ... that's not supposed to happen, and is a bug.   But check the "technical details" window and make sure that it's not actually uploading. 


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#6 ironhead65

ironhead65

    Advanced Member

  • Members
  • PipPipPip
  • 38 posts

Posted 17 July 2017 - 08:46 PM

OK, I just remoted back into that Virtual Machine - having left most of the Upload Threads Unchecked.  Anything that was unchecked was NOT uploading any longer.

The few that were, were certainly splitting up my bandwidth MUCH better (I'm guessing), since scrolling through (left/right arrows showing the different drives) showed only those two with the Uploads selected as actually uploading.  They also showed about 5Mbps between the two of them.  Normally, I only see about 300-500kbps on maybe 7 drives - which is a bit low for my link of 5Mbps Total.  Hence the assumption that I'm being killed in overhead traffic at the moment.

 

I suppose that I was too impatient and since they are THREADS, they just completed before they actually were removed?  That makes sense to me at least!

 

I also will say that this seems to have made that Virtual Machine happier as well.  I can now remote into it without a TON of lag.  The machine seems peppier and looking at the Task Monitor, the CloudDrive service is not pegging the processor to 100% any longer.  Maybe I had multiple issues?  Not enough System resources as well as low link speed?  I could see the AES engine needing a LOT of effort due to the MASSIVE overhead I was probably accumulating?  I do realize that this was probably more intended to link 1-2 drives / services...and I am abusing that a BIT as I pooled together to many services to get a large drive pool.

 

Thanks for confirming the behavior.  I guess I would have seen that next time I remoted into the machine.



#7 ironhead65

ironhead65

    Advanced Member

  • Members
  • PipPipPip
  • 38 posts

Posted 17 July 2017 - 09:11 PM

Wow, thanks again to both of you!  I am not making some SERIOUS headway in getting my Cloud Seeded!  I had 5 drives with about 80-100MB of data waiting to be uploaded.  By targeting those first, they finished within MINUTES!!  Each being given 5+Mbps and they just RIPPED through that data!

Now on to the other drives one at a time!  This is a beautiful setup now!!



#8 ironhead65

ironhead65

    Advanced Member

  • Members
  • PipPipPip
  • 38 posts

Posted 18 July 2017 - 12:25 AM

Another update, I can't believe how FAST this is going now!  I waited at least 1 week, I am thinking closer to TWO weeks and got a total of maybe 50GB uploaded to the cloud services I'm tied to.  At this time, I've been at this with the "upload threads" unchecked for 6 hours and I've dumped well over 3GB!  That is more progress in so many hours than I could have HOPED for in 1/2 a week before!!  

 

--This is assuming:

50GB / 14 days = 3.57GB/day  BEFORE

3GB/6hour * (24 hour / day) = 12GB/day NOW!  This is AMAZING!!!  I can now actually WATCH the progress update in the Drive PIE diagrams!!

 

Now I REALLY want have to figure out how to maintain my sanity as I wait for the "Global Threading" or "Global Round Robin" feature to be added!  So exciting!!  I feel that I am mamually simulating what the feature would end up doing!



#9 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,208 posts
  • LocationSan Diego, CA, USA

Posted 19 July 2017 - 09:49 PM

well, I'm glad to hear that everything is working great now!

 

And that's a significant increase in performance!

 

And alex is reworking some of the code, and that may help things further, actually.  

 

 

But global limits would be a good idea. Or at least *per provider*. 

https://stablebit.co...eAnalysis/27562


  • ironhead65 likes this

Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#10 ironhead65

ironhead65

    Advanced Member

  • Members
  • PipPipPip
  • 38 posts

Posted 25 July 2017 - 08:13 PM

Hello all,

 

I had an idea.  I'm going to try to use the Drivepool plugin "Ordered File Placment", which as I understand it will let me run the disks more like JBOD.  If I disable all optimizations for the Drivepool and just let it JBOD...that might resolve my need for the "single install" macro at this time.  I still would like the feature however maybe this will work by keeping most of my updates on one or two drives at a time!


  • Christopher (Drashna) likes this

#11 ironhead65

ironhead65

    Advanced Member

  • Members
  • PipPipPip
  • 38 posts

Posted 14 August 2017 - 02:37 PM

Hello all,A 

 

little feedback..This MOSTLY works.  The issue seems to be that whenever a do my compare (for mirroring), there is ~768KB of data written to EVERY drive.  As this is the case, the ordered placement does not really help.  If I leave all the drives enabled for upload, as soon as I do the compare, ALL the drives start to hammer the internet to try to push their data out.  The BULK is still in the SINGLE drive (using ordered placment).  However, for the first few hours, that 768KB kills my drives and loops back through the above described undesired process (drives requiring re-authorization).

 

Guess I'm back to using my script mechanism for controlling the upload stream.

 

I will say that as long as I leave 1 drive running at a time, I can pull FULL bandwidth for that 1 drive!  This really is an awesome addon for my network!


  • Christopher (Drashna) likes this




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users