Jump to content
  • 0

problem rebalancing


Vittorio Zamparella

Question

I've beeing owning a license for Drivepool for some time now and I'm very satisfied. A few junctions here and there and my new ssd is practically transparent; some folders reside on the ssd, some on the hd and c:\ looks exactly the same as it did before.

However I'm having some problem rebalancing.
(I think everything was working as expected (by me) at the beginning, but I can't tell for sure.)
 

A few days ago my ssd (c:\ boot windows drive) filled up unexpectedly but I found out the cause:
First: I was loading a big chunk of data; second: on the mechanical drive I had forgotten to expand the partition to the full capacity. Therefore Drivepool had to fill up also the ssd drive. No big deal. I expanded the "big" drive partition to its full capacity and told Drivepool to move some big folders to it.
I could have done it manually but I was eager to see a big rebalance in action.
Nothing happend.

Drivepool version is 2.2.2.934
Now a list of thing I tryed (in various order and mixes) and of unexpected things.
I added some rules in the file placement checking only the big drive.
"Allow files to be placed on other disks if all selected disks are this full" is enabled at 90%
I disabled all the balancers.
I installed the SSD optimizer, enabled it alone, disabled it.
I uncheck "file placement rulse respect real-time file placemente limits set by the balancing plug-ins".
I unchecked "unless the drive is beein emptied".
I triggered "remeasure".
I triggered "rebalance" (it flashes for a moment then ends).
I scheduled a one-time rebalance.
I disabled automatic balancing.
I got a verbose log of a manual re-balance, and here things are a bit weird because the strings  that look like timestamps at the beginning of lines are off by a weird number of hours AND minutes in respect to the machine clock (which is quite right by itself).
 
21:03:58.4: Information: 0 : [Rebalance] Setting Background I/O priority
21:03:58.4: Information: 0 : [FsControl] Set overall pool mode: PoolModeNoMeasure (lastKey=Rebalance, pool=9cbf91d2-40b0-45b5-b326-35429cc490bf)
21:03:58.4: Information: 0 : [Rebalance] Balance states (START):
21:03:58.4: Information: 0 : [Rebalance]   6ce3ec8d-c620-4034-b599-126c09029a9b (ColorfulStone (C:\)): Fill = 626,074,648,576 B / 640,029,118,464 B (97.82%)
21:03:58.4: Information: 0 : [Rebalance]     Unprotected: 481,816,012,691 B / 640,029,118,464 B (75.28%), Delta = 0 B, Limit = 1.0
21:03:58.4: Information: 0 : [Rebalance]     Protected: 2,405,517,651 B / 640,029,118,464 B (0.38%), Delta = 0 B, Limit = 1.0
21:03:58.4: Information: 0 : [Rebalance]   e260dc30-3718-4de5-8d0d-61b59b07298f (GreenishStone (F:\)): Fill = 981,641,318,400 B / 2,000,289,812,480 B (49.07%)
21:03:58.4: Information: 0 : [Rebalance]     Unprotected: 962,070,509,969 B / 2,000,289,812,480 B (48.10%), Delta = 0 B, Limit = 1.0
21:03:58.4: Information: 0 : [Rebalance]     Protected: 2,405,517,651 B / 2,000,289,812,480 B (0.12%), Delta = 0 B, Limit = 1.0
21:03:58.4: Information: 0 : [Rebalance] Balancing file placement...
21:03:58.9: Warning: 0 : [Rebalance] Error balancing. The system cannot find the path specified
21:03:58.9: Information: 0 : [FsControl] Set overall pool mode: PoolModeNormal (lastKey=Rebalance, pool=9cbf91d2-40b0-45b5-b326-35429cc490bf)
21:03:58.9: Information: 0 : [Rebalance]   Calculating optimal slack...
21:03:58.9: Information: 0 : [Rebalance]     Largest PoolSize (UIDs: e260dc30-3718-4de5-8d0d-61b59b07298f, Largest=1,983,124,521,700 B)
21:03:58.9: Information: 0 : [Rebalance]     Other PoolSize (UIDs: 6ce3ec8d-c620-4034-b599-126c09029a9b, Other=498,175,934,694 B)
21:03:58.9: Information: 0 : [Rebalance]       (Largest=1,484,948,587,006)
21:03:58.9: Information: 0 : [Rebalance]   Optimal slack: 1,484,948,587,006 B
21:03:58.9: Information: 0 : [Rebalance]   Calculating duplication obstruction...
21:03:58.9: Information: 0 : [Rebalance]   Calculating current slack...
21:03:58.9: Information: 0 : [Rebalance]     Largest FreeSpace (UIDs: e260dc30-3718-4de5-8d0d-61b59b07298f, Largest=1,018,648,494,080 B)
21:03:58.9: Information: 0 : [Rebalance]     Other FreeSpace (UIDs: 6ce3ec8d-c620-4034-b599-126c09029a9b, Other=13,954,404,352 B)
21:03:58.9: Information: 0 : [Rebalance]       (Largest=1,004,694,089,728)
21:03:58.9: Information: 0 : [Rebalance]   Current slack: 1,004,694,089,728 B
21:03:58.9: Information: 0 : [Rebalance]   Obstructed: 1,004,694,089,728 B
21:03:58.9: Information: 0 : [Rebalance]   Other pooled files: 626,074,714,112 B
21:03:58.9: Information: 0 : [Rebalance]   (DuplicationObstructionSlack=1,004,694,089,728 B)
21:03:58.9: Information: 0 : [Rebalance]   (DuplicationObstructionSlack / 2=502,347,044,864 B)
21:03:58.9: Information: 0 : [Rebalance]   (TotalPoolSize=2,481,300,456,394 B)
21:03:58.9: Information: 0 : [Rebalance]   (DuplicationObstructionLimitRatio=0.3)
21:03:58.9: Information: 0 : [Rebalance]   (DuplicationObstructionRatio=0.2)
21:03:58.9: Information: 0 : [Rebalance]   (TotalPooledFiles=964,476,027,620 B)
21:03:58.9: Information: 0 : [Rebalance]   (DuplicationObstructionStateTotalSize=2,000,289,812,480 B)
21:03:58.9: Information: 0 : [Rebalance]   (DuplicationObstructionStatePooledFiles=964,476,027,620 B)
21:03:58.9: Information: 0 : [Rebalance]   (ToMoveRatio=0.7333)
 
Snapshot taken around 16:20

I solved the immediate problem of full disc by moving some stuff manually. Drivepool didn't complained, just found one duplicate file (I was too lazy to stop the service) and fixed it.

Can you advice?
What other informations can I give?
 
Thank you!
Link to comment
Share on other sites

10 answers to this question

Recommended Posts

  • 0

It's hard to know what to recommend without seeing your drive sizes and your pool configuration.  If you could post those (screens of the DrivePool UI would be great) it would make it easier.  Additionally, any rules you've set up would help determine what's going on.

Also - when you resized the "big" drive partition to it's full size, did it *actually* change in Explorer, or just in the Drive Management snap-in?  Often Windows will change it in the Drive Management GUI, but Explorer will still report the old space incorrectly.  You may need to open an elevated command prompt, run diskpart, use "list volume" to find the drive you're extending, then "select volume x" where x is the number of that drive, then "extend".  It helps Windows finish and recognize what the Drive Management didn't quite finish.  I have to do it anytime I resize my C: drive and the volume(s) after it.

 

 

Link to comment
Share on other sites

  • 0
7 hours ago, Vittorio Zamparella said:
21:03:58.9: Warning: 0 : [Rebalance] Error balancing. The system cannot find the path specified
 

Disk issue most likely. 

Run CHKDSK on all of your pooled disks, remeasure the pool, and see if those help. 

If not, then try resetting the settings. 
http://wiki.covecube.com/StableBit_DrivePool_Q2299585B

If that doesn't help, then enable file system logging and reproduce the issue (rebalance the pool)
http://wiki.covecube.com/StableBit_DrivePool_2.x_Log_Collection

And then open a ticket at https://stablebit.com/contact, and run the StableBit Troubleshooter:
http://wiki.covecube.com/StableBit_Troubleshooter

Link to comment
Share on other sites

  • 0
Quote

It's hard to know what to recommend without seeing your drive sizes and your pool configuration. 

SSD 600Gb, HD 2TB

Quote

If you could post those (screens of the DrivePool UI would be great) it would make it easier. 

Sure!

image.png.7484dccab8637186581413bb64388040.png

 

Quote

Additionally, any rules you've set up would help determine what's going on.

Very simple rules: just prefernce to the second drive for some folders:

image.png.47cb5abcdf6232e8d05e58bb19482729.png

 

Quote

Also - when you resized the "big" drive partition to it's full size, did it *actually* change in Explorer,

Yes

Quote

or just in the Drive Management snap-in? 

I actually used EaseUS Partition Master 12.10 in restart mode and got no errors whatsoever.

Quote

Often Windows will change it in the Drive Management GUI, but Explorer will still report the old space incorrectly.  You may need to open an elevated command prompt, run diskpart, use "list volume" to find the drive you're extending, then "select volume x" where x is the number of that drive, then "extend".  It helps Windows finish and recognize what the Drive Management didn't quite finish.  I have to do it anytime I resize my C: drive and the volume(s) after it.

I'm gonna check it.

Thank you!

Link to comment
Share on other sites

  • 0
23 minutes ago, Christopher (Drashna) said:

Disk issue most likely. 

Run CHKDSK on all of your pooled disks, remeasure the pool, and see if those help. 

If not, then try resetting the settings. 
http://wiki.covecube.com/StableBit_DrivePool_Q2299585B

If that doesn't help, then enable file system logging and reproduce the issue (rebalance the pool)
http://wiki.covecube.com/StableBit_DrivePool_2.x_Log_Collection

And then open a ticket at https://stablebit.com/contact, and run the StableBit Troubleshooter:
http://wiki.covecube.com/StableBit_Troubleshooter

<This editor doesn't fit me>

Quote

Run CHKDSK on all of your pooled disks, remeasure the pool, and see if those help. 


Yes that makes sense. I was trying not to be to hysterical about errors in the logs since I've been warning that most of them means nothing bad.
Still it would have been useful to read in the log *which* file was not found.


I'll check disks overnight.

Link to comment
Share on other sites

  • 0
4 minutes ago, Vittorio Zamparella said:

I'm gonna check it.

Thank you!

Christopher caught the line in the log about a disk not being available - he's probably right in this regard.  I've seen it happen sometimes with USB drives sleeping and not spinning up when tried to access.  I was curious though about your junction points and how you had it configured.

If he's suggesting you submit a ticket, take advantage of it.  :) 

Link to comment
Share on other sites

  • 0
16 minutes ago, Jaga said:

Christopher caught the line in the log about a disk not being available - he's probably right in this regard.  I've seen it happen sometimes with USB drives sleeping and not spinning up when tried to access.  I was curious though about your junction points and how you had it configured.

If he's suggesting you submit a ticket, take advantage of it.  :) 

This forum editor is much harder to master than Drivepool :-)

Yes, with so many files sometimes a logical failure happens, but I'm perplexed that this stops so harsly drivepool.

I'll file a ticket If the checkdisk doesn't solve.

My Junctions are as following. I'll skip many details and try to distill the fun part.
1) I cloned my mechanical onto the SSD

2) I planned to leave undisturbed Windows, Users folders and all program folders; their overall footprint is small except for the caches which I want onto the ssd of course.

3) I left many big "data" folders on the mechanical. You have to know that I have many pieces of software that point to theese "data" folders: sync softwares, builders, semiautomatic scripts, net shares, I don't even recall all the interlinks. You know, stuff you build year over year making your house confortable.

4) I established a pool and assigned the letter D:

6) I moved all the "data" folders into the pooled folder on the same drive (very cheap operation)

5) I put back on c: junctions to the missing folders pointing to D: (drivepool managed) f: (mechanical) or c: (ssd)

This way the layout I want for my folders doesnt interfere with the physical location.
I always hated systems with more drives than necessary, but also I don't like the *nix habit of mounting everything under the same root.
One drive for all local is what my mind needs :)

Unless you tell me I can boot from a pooled drive :)

OK I confess I also do the other thing around: some folders are shared and mounted with their own letter; this way it doesn't matter if they come from the local machine, my home server or a remote location: again the layout is purpose-driven and not physical location driven.

(sorry for any typo, but this editor hates me) 

Link to comment
Share on other sites

  • 0
3 minutes ago, Vittorio Zamparella said:

3) I left many big "data" folders on the mechanical. You have to know that I have many pieces of software that point to theese "data" folders: sync softwares, builders, semiautomatic scripts, net shares, I don't even recall all the interlinks. You know, stuff you build year over year making your house confortable.

Hopefully none of those (or any of the paths/files that the software uses) are hard links.  Due to the way they work, Drivepool can't support hard links.

https://docs.microsoft.com/en-us/windows/desktop/fileio/hard-links-and-junctions

Junctions (soft links) are okay however.

Link to comment
Share on other sites

  • 0
Just now, Jaga said:

Hopefully none of those (or any of the paths/files that the software uses) are hard links.  Due to the way they work, Drivepool can't support hard links.

https://docs.microsoft.com/en-us/windows/desktop/fileio/hard-links-and-junctions

Junctions (soft links) are okay however.

Woa. This is not peanuts. Yes, no hardlinks in the pool. I hope. I surely have hardlinks in a program. Ehr. Maybe. Now that I think... there ARE hardlinks in the pool :(
Oooooh nooo!!!
What shall I do now? :(
They were useful!

Link to comment
Share on other sites

  • 0
3 hours ago, Vittorio Zamparella said:

Woa. This is not peanuts. Yes, no hardlinks in the pool. I hope. I surely have hardlinks in a program. Ehr. Maybe. Now that I think... there ARE hardlinks in the pool :(
Oooooh nooo!!!
What shall I do now? :(
They were useful!

I don't think I can advise you in that case very well - it's a question for someone with more knowledge of how they impact the pool to answer.  @Christopher (Drashna) or @Alex might know some tricks or workarounds.

Edit:  Okay - so that's not entirely true, I do know one workaround for using hardlinks and the pool.  You can install a program to the drive's hidden Poolpart-xxxxx folder directly, and it will continue to utilize hard links on that drive just fine.  Re-measuring the pool will then show the folders/files in there okay too.  Just don't allow that set of folders/files to be moved off the drive by a balancing - create a file placement rule so they stay put.  The program will access through the Poolpart-xxxxx folder instead of the Pool drive, and should have no issues with it's hard links.

 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...