Jump to content
Covecube Inc.
  • 0
msq

A few questions about measuring.....

Question

There is a couple of issues and questions about measuring.

 

Why it is not optimized / cached / smarter?

Case 1:

If there is a pool of 2 or more drives and one is disconnected for whatever reason - once the drive is back entire pool is measured. If that is large, cloud based pool with a lot of small files - a short interruption in connectivity triggers hours of measuring.

Would that be possible to cache results of previous measure run and measure only the drive that was reconnected?

Case 2:

Four cloud based drives CLD1, CLD2, CLD3 and CLD4 are coupled in two pools: CLD1+2 and CLD3+4. Those two are paired again as a pool CLD1+2&CLD3+4. And this pool is coupled with physical drive in the highest level pool: HDD1&CLD1+2&3+4. Apart from CLD1+2 and CLD3+4 - every other pool is set to duplicate all files.

Now - when measure is running it is starting from the highest level, then is measuring middle tier and two on the lowest level. So files on the cloud based drives are actually measured three times.

Would that be possible to start from the lowest level, then cache and re-use data it on higher levels?

Why it is so sensitive?

Case 3:

Let's say there is more than one pool running measure. As far as I can see this is slightly misleading - as only one pool can be measure at a time - other pools are actually awaiting. If for whatever reason one drive in the pool that is NOT currently effectively being measured disconnected and reconnected - measure on the other drive is restarted. Why..?

And is it possible by change in configuration to allow more than one measure thread to run?

Would that be possible to set specific measure order for pools?

Case 4 (feat. CloudDrive):

As mentioned above - short interruption in internet connectivity can disconnect all cloud drives, enforce manual intervention (re-connect) and massive re-measure.

Would that be possible to make CloudDrive drives a bit more offline-friendly and disconnection resistant ? In cases when there is no pool activity from users?

And can we receive some feature that will automatically re-connect cloud drives, please? :)

Thanks :)

 

EDIT: typos, wording

Share this post


Link to post
Share on other sites

5 answers to this question

Recommended Posts

  • 0
23 hours ago, msq said:

Case 1:

If there is a pool of 2 or more drives and one is disconnected for whatever reason - once the drive is back entire pool is measured. If that is large, cloud based pool with a lot of small files - a short interruption in connectivity triggers hours of measuring.

Would that be possible to cache results of previous measure run and measure only the drive that was reconnected?

The problem is that the contents of the disk may have changed.  Measuring makes sure that the statistics are accurate. 

For the CloudDrives, yeah, it can take a while.  

On 5/13/2018 at 11:09 AM, msq said:

Case 2:

Four cloud based drives CLD1, CLD2, CLD3 and CLD4 are coupled in two pools: CLD1+2 and CLD3+4. Those two are paired again as a pool CLD1+2&CLD3+4. And this pool is coupled with physical drive in the highest level pool: HDD1&CLD1+2&3+4. Apart from CLD1+2 and CLD3+4 - every other pool is set to duplicate all files.

 Now - when measure is running it is starting from the highest level, then is measuring middle tier and two on the lowest level. So files on the cloud based drives are actually measured three times.

Would that be possible to start from the lowest level, then cache and re-use data it on higher levels?

 Why it is so sensitive?

Yes and no.  The pooled data is in PoolPart folders.  So the information may not be completely correct for higher level pools, since all of the data may not be pooled data.

But I've flagged this: https://stablebit.com/Admin/IssueAnalysis/27821

On 5/13/2018 at 11:09 AM, msq said:

Case 3:

Let's say there is more than one pool running measure. As far as I can see this is slightly misleading - as only one pool can be measure at a time - other pools are actually awaiting. If for whatever reason one drive in the pool that is NOT currently effectively being measured disconnected and reconnected - measure on the other drive is restarted. Why..?

And is it possible by change in configuration to allow more than one measure thread to run?

Would that be possible to set specific measure order for pools?

Hmm, not sure about this, actually. 

Though, probably a performance/driver related reason. 

https://stablebit.com/Admin/IssueAnalysis/27822

On 5/13/2018 at 11:09 AM, msq said:

 Case 4 (feat. CloudDrive):

As mentioned above - short interruption in internet connectivity can disconnect all cloud drives, enforce manual intervention (re-connect) and massive re-measure.

 Would that be possible to make CloudDrive drives a bit more offline-friendly and disconnection resistant ? In cases when there is no pool activity from users?

This is by design.  Loss of internet connectivity translates into I/O issues.  And Windows is very sensitive to I/O issues.  So, we disconnect the drive, rather than allowing the system to lock up and potentially lose data.   

There are some settings that may help:
http://wiki.covecube.com/StableBit_CloudDrive_Advanced_Settings

"CloudFsDisk_MaximumConsecutiveIoFailures", for instance. But some of the IoManager may help. 

 

As for "offline friend".... sure, if you don't mind potentially corrupt data..... Seriously.  
There is a "mount disconnected option: https://stablebit.com/Support/CloudDrive/Manual?Section=Mounting%20a%20Disconnected%20Drive

 

 

On 5/13/2018 at 11:09 AM, msq said:

And can we receive some feature that will automatically re-connect cloud drives, please? :)

No.  You wouldn't want this.  This can cause issues, it could cause serious performance/reliability issues?  
Like would you use a drive long term that would randomly and frequently disconnect from the USB hub?  

 

Share this post


Link to post
Share on other sites
  • 0
22 hours ago, Christopher (Drashna) said:
On 5/13/2018 at 7:09 PM, msq said:

And can we receive some feature that will automatically re-connect cloud drives, please? :)

No.  You wouldn't want this.  This can cause issues, it could cause serious performance/reliability issues?  
Like would you use a drive long term that would randomly and frequently disconnect from the USB hub?  

No, I wouldn't.

But what I meant by "feature that will automatically re-connect cloud drives" is some functionality that will perform exactly the same action as user from the front end by clicking 'Retry' - that's all. Every x minutes if drive gets disconnected. If there will be no connectivity - nothing will happen. But if the connectivity will be back - drive will be just reconnected.

I'm on a rather crappy internet here and sometimes cloud drives get disconnected. All I have to do is click 'Retry' and it is connected again. I'm asking to get this 'Retry' action automated. IUt could be turned off by default and hidden deep into advanced json setup - that's fine.

If that is gui action someone will finally write a script that will mimic such click :)

Share this post


Link to post
Share on other sites
  • 0

I'm sorry to hear about your ISP quality... :(

 

That said, you want to modify the "CloudFsDisk_MaximumConsecutiveIoFailures" option then.  That should help to prevent it from disconnecting in the first place.  Set it to 5 or 8, and see if that helps. 

Share this post


Link to post
Share on other sites
  • 0
On 5/15/2018 at 9:07 PM, Christopher (Drashna) said:

That said, you want to modify the "CloudFsDisk_MaximumConsecutiveIoFailures" option then.  That should help to prevent it from disconnecting in the first place.  Set it to 5 or 8, and see if that helps. 

Thank you - good point. I had it set to something non-default already but most likely during some re-installation this has been lost.

But hey - I just noticed something in change log :)

Thank you once again and will test that next few days!

 

.988
* Added an option to automatically retry mounting drives. See "Automatically retry mounting" under Options.
	- Drives that were unmounted due to an incorrect or missing encryption key, an unknown storage provider, or 
	  damaged metadata will not retry mounting automatically.

 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...