Jump to content
Covecube Inc.
  • 0

Change allocation unit size of all drives in DrivePool: Best way?


Question

Hi folks,

 

I've got 4x8TB and 2x6TB drives in a 6 drive DrivePool with 2x duplication on all files, and 3x duplication for a specific folder.

 

Attached is a photo of current drive stats:

4Vc6s1P.png

 

I formatted these drives with 4098 byte unit allocation size. I want to change all 6 drives to 64KB.

 

I'm looking to find out the best way to perform this change on all 6 drives with minimal risk. Basically looking for a recommended set of exact procedure.

 

In my head I'd be doing this:

  • Remove a drive from drivepool
  • Wait for the 2x/3x stuff to propagate to the remaining drives
  • Format drive with new allocation size
  • Add back to DrivePool
  • (Presumably it won't be moving any data back to this drive)
  • Repeat for the remaining drives

However there must be an easier way. Is there a way to change it without formatting? Without removing it from the DrivePool?

 

I'm sure there's 20 ways to get this done, hopefully I can get some staff or user input on the issue. I'm tech savvy but never dealt with a situation like this before. Not to mention the added complexity of the drives being in a pool.

 

Thanks!

Link to post
Share on other sites

10 answers to this question

Recommended Posts

  • 0

LOL.  I just recently did this on my own pool, actually. 

 

 

There isn't a good way. You need to format the drives to do this (or use a 3rd party tool, and hope it doesn't corrupt your disk).

 

The simplest way is to use the balancing system.  Use the "Disk Space Limiter" balancer to clear out one or more disks at a time. Once it's done that (it make take a few hours or longer), remove the disk from the pool and reformat it. Re-add it and repeat until you've cycled out ALL of the disks.

 

 

Specifically, the reason that I mention the Disk Space Limiter balancer is that it runs in the background and doesn't set the pool to be read only. And is mostly automated.

Link to post
Share on other sites
  • 0

Thanks for your input Christopher, I'll be sure to utilize that balancer and have it move all files from that drive, then pull it out of the pool and format, then add it back.

 

I figured it'd have to be something like that but your idea sounds like the right approach. My main concern is I wanted to ensure that while I was doing all of this moving and formatting, I always had my 2x and 3x duplication intact (aka I wasn't manipulating my drives in a "degraded" state with a single instance of a file in the case of 2x, or 2x in the case of 3x).

 

I'll most likely do that, and as you said it'll take time so I'll have to find an optimal time to do it because I'd like to monitor it while I'm doing it all.

 

Thanks!

Link to post
Share on other sites
  • 0

I figured it'd have to be something like that but your idea sounds like the right approach. My main concern is I wanted to ensure that while I was doing all of this moving and formatting, I always had my 2x and 3x duplication intact (aka I wasn't manipulating my drives in a "degraded" state with a single instance of a file in the case of 2x, or 2x in the case of 3x).

 

Well, removing the disk from the pool normally would do that as well (as long as you're not using the "duplicate data later" option). 

 

But using the balancer is best way for this, IMO. and it clears out disks without having to remove them first (and the hassle with that).

 

And if you have enough space (and it looks like you do), you could clear out a couple disks at a time. Also, when this is going on, there should be an "X" and a ">>" button on the left of the pool status bar. Clicking the ">>" button increases the balancing priority, which should allow it to run a bit faster.

Link to post
Share on other sites
  • 0

Why would you want to change it to 64 kb?   Do you have large media files and looking fro better streaming performance?  

For a number of reasons. 

 

By default, Windows wants to use 4k clusters. This means that each allocation unit for files is this size. If you ahve a 10MB file, it's broken up into 4k chunks and placed on the disk.  The flip side is that each file uses 4k period. That includes folders, which have very little data.  

 

Additionally, even if the file (or the remaining chunk) uses a couple of bytes, it will take up the entire allocation unit. 

 

 

Now, why is this important? Well, a 64kb chunk is 16x the size of the 4kb one.  

This means that more data is guaranteed to be sequential.  This is of one main benefit: Fragmentation. 

 

Sinch each chunk is larger, that means that you're less likely to see fragmentation, Additionally, since more of that data is in once section of the disk, it means that there is less seeking being performanced on the disk, and may increase performance.

 

THis is the same reason some people recommend that you defragment your disk. 

 

 

 

The flip side, as I mentioned, that if you have a lot of small files, you're losing disk space. A cluster or allocation unit can only be used by one file. 

For an OS disk, or one with a lot of very small files, this can cause an issue, as you may run out of allocation units before you run out of disk space.   

However, for storage, a majority of the files are going to be much larger, and will benefit from the allocation unit size increase. 

 

 

 

 

So what does this equate to for you? Well, on my Seagate Archive drives, I saw fewer performance drop offs, and I saw about a 20MB/s speed increase on reads. 

Link to post
Share on other sites
  • 0

What's about SSDs clusters size if they are used with SSD plugin on a pool with 64k hard disks?

Should SSDs clusters match HD's one? or is it better to leave them at rhe default size?

 

It's better to not use the larger allocation unit size for SSDs.  

 

Specifically, because SSDs don't benefit from them. The main reason to use the larger size is performance and preventing fragmentation. Since neither affect an SSD, there is no benefit.

Additioanlly, since SSDs are smaller in capacity, you're more likely to lose a good chunk of space by doing so.

 

 

However, there is absolutely no harm in using the different sizes on the SSDs vs the spinning hard drives. 

 

 

So 64k allocation unit size for "spinners" and default for SSDs is good.

Link to post
Share on other sites
  • 0

For a number of reasons. 

 

By default, Windows wants to use 4k clusters. This means that each allocation unit for files is this size. If you ahve a 10MB file, it's broken up into 4k chunks and placed on the disk.  The flip side is that each file uses 4k period. That includes folders, which have very little data.  

 

Additionally, even if the file (or the remaining chunk) uses a couple of bytes, it will take up the entire allocation unit. 

 

 

Now, why is this important? Well, a 64kb chunk is 16x the size of the 4kb one.  

This means that more data is guaranteed to be sequential.  This is of one main benefit: Fragmentation. 

 

Sinch each chunk is larger, that means that you're less likely to see fragmentation, Additionally, since more of that data is in once section of the disk, it means that there is less seeking being performanced on the disk, and may increase performance.

 

THis is the same reason some people recommend that you defragment your disk. 

 

 

 

The flip side, as I mentioned, that if you have a lot of small files, you're losing disk space. A cluster or allocation unit can only be used by one file. 

For an OS disk, or one with a lot of very small files, this can cause an issue, as you may run out of allocation units before you run out of disk space.   

However, for storage, a majority of the files are going to be much larger, and will benefit from the allocation unit size increase. 

 

 

 

 

So what does this equate to for you? Well, on my Seagate Archive drives, I saw fewer performance drop offs, and I saw about a 20MB/s speed increase on reads. 

 

 

 

 

 

Sorry to bump an old post but I just wanted to say thanks for this. I just added another 8tb Seagate today and decided to format at 64k on the strength of this thread.

Without the speed boost button I'm seeing 60-90 MB/s constant. (It didn't drop below 60MB/s in a twenty minute period.)

 

With the speed boost enabled I do see drop offs, but much less frequently than on the previous 8tb (formatted at 4k) and by the same token I'm hitting 180-210 MB/s at times. Happy days. :)

Anyway this latest drive addition will give me available space to empty and reformat the other drives over the next few weeks so I'm gonna do it. .

 

Going back to your comment about fragmentation, I understand what you mean about larger clusters should lead to less fragmentation.. Am I also correct in thinking that if over time fragmentation occurs, the larger clusters will ALSO make it easier to defrag?

I will add that all the Seagate archive drives will hopefully be write once as intended, so this question mainly applies to some of the 4TB drives that are much more read/write intensive.

 

Either way thanks so much for your in depth reply to the guys in this thread. It all makes plenty of sense and definitely seems worth doing. :P

Link to post
Share on other sites
  • 0

Glad to hear that it's helped you (and no problem with bumping the thread). 

 

As for the fragmentation, yes and no. Chances are, there will be less fragmentation, and when defragging it, it will move around larger chunks of data.   But it won't make defrag-ing easier really. 

Link to post
Share on other sites
  • 0

Glad to hear that it's helped you (and no problem with bumping the thread). 

 

As for the fragmentation, yes and no. Chances are, there will be less fragmentation, and when defragging it, it will move around larger chunks of data.   But it won't make defrag-ing easier really. 

 

No worries Christopher, thanks very much.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...