Migrating from Windows Drive Pool to StableBit Drive Pool

[rogue_9]

Hi,

Before I make a purchase of StableBit's DrivePool, I have a couple of questions that on how to migrate from the Windows Drive Pool to StableBit's DrivePool since I hit the lovely 63 TB limit based on the bytes per inode.

I currently have four drives hooked up to the drive pool. What's going to be the best way to do this migration without breaking apart the Windows drive pool while maintaining at the very least read access to the files (the drives are my Plex Media Server). Write access would be nice since I also record gameplay and streams to the same drive, but that's not a necessary thing I need. The drive pool is set up as just a single giant disk JBOD setup since I use BackBlaze to keep all my data backed up. That leads me to question 2. Will BackBlaze play nicely with StableBit?

Thanks in advance.

Ellia

[Shane]

Could you explain what you mean by "Windows Drive Pool"? Do you mean a pool created via Windows Storage Spaces? Something else?

What drives (e.g. 3x12TB + 1x10TB) do you currently have and how are they currently allocated (e.g. system, storage spaces, other, empty)? How many additional drives do you have room for? How fussy is your media server about how it sees your files (e.g. you could tell it "everything is stored on drive P and drive Q" with those being your Stablebit and Windows pools, and then just gradually move files between pools)?

The method I saw folks use re BackBlaze was to backup each drive within the (Stablebit) pool rather than the pool as a whole, so that if a drive within the pool fails that whole drive can be restored from BackBlaze rather than trying to figure out which files were on it; I don't know if BackBlaze's restore process has improved since then.

[rogue_9]

Yeah, I meant the Windows Storage Spaces drive pool.

I currently have 3x 20 TB and 1x 18 TB allocated to the drive pool as storage space and only using 32 TB at the moment. I don't have any spare drives of capacity to move files between; the remaining drives are NVMe drives ranging between 1 and 4 TB with one of them as the system drive. Due to the space I have, I could remove one drive at a time to recreate the pool in Stablebit and transfer files overnight. Plex is rather fussy about the drive letters if I don't go in manually into the registry to change the location. But if I do the move overnight (or, well, 24 hours) on a day I don't need it, that wouldn't be a huge hit.

Backblaze hasn't changed on restoration process if a drive fails.

[Christopher (Drashna)]

Well, StableBit DrivePool does support adding a Storage Spaces array to the pool.  So until you have more disks and are able to migrate the data away, you could add both to a pool.

[rogue_9]

2 hours ago, Christopher (Drashna) said:

Well, StableBit DrivePool does support adding a Storage Spaces array to the pool.  So until you have more disks and are able to migrate the data away, you could add both to a pool.

To make sure I understand, I can just add my current drive pool to StableBit, and will that let me push the capped capacity I'm stuck at to the actual capacity, or will I still be stuck at the capped 63 TB capacity because it is a Windows-based storage pool?

[Shane]

Well that makes things easier! Thanks Christopher! Can confirm, just grabbed some external drives and made a Storage Spaces pool with them then added that pool to be part of a DrivePool pool without any problems.

However: I've been playing with Storage Spaces in JBOD mode to see what you'd need to do, and it was NOT happy after removing the second-last disk - Storage Spaces removed it without any warnings but then immediately afterward complained that there was an issue, its pool ceased to be accessible and the drives became stuck in a limbo of being removed but not actually removed until I physically pulled them and manually wiped them. Ouch.

I would strongly recommend that you have enough spare capacity to empty the last two drives, not just one, if/when you dismantle the old Storage Spaces pool, and the order is apparently "delete storage pool then remove last two physical drives".

So all that said, the steps for migrating from Storage Spaces to DrivePool as the pool that Plex, etc, sees should be:

1. Create a DrivePool pool (let's say it's P: drive). Doesn't actually have to have any other physical drives in it yet.
    
2. Add your Storage Spaces pool (let's say it's S: drive) to the above.
    
3. Turn off any services/programs looking for S: drive (e.g. your Plex software). Also close the Stablebit DrivePool program and turn off the StableBit DrivePool service.
    
4. Manually move all your user content in S: drive - so excluding system folders like System Volume Information, $RECYCLE.BIN, etc - into the hidden PoolPart folder that's now in the root of S: drive. For example S:\MyStuff\MyFile.txt would become S:\PoolPart.guidstring\MyStuff\MyFile.txt
    
5. Swap the drive letters of the two pools via Windows Disk Management (remove S: from Storage Spaces pool, remove P: from DrivePool pool, add S: to DrivePool pool, add P: to Storage Spaces pool).
    
6. Turn the Stablebit DrivePool service back on and open the StableBit DrivePool program. If it doesn't proceed to do so automatically, tell DrivePool to re-measure the pool via Manage Pool -> Re-measure... so that it accurately reports the disk usage to you.
    
7. Turn your services/programs that use S: drive back on.

Presuming everything is now humming along with the DrivePool pool as your "front end", you could then gradually remove your drives from the Storage Spaces pool and add them directly to the DrivePool pool instead (keeping in mind what I discovered about trying to remove the last two drives - if need be you could just leave the last two drives alone).

If/when you're ready to delete the old Storage Spaces pool (i.e. only two drives left inside it and you've got enough spare capacity on your other drives in the DrivePool pool), remove it from the DrivePool pool first and then once that's successful delete the old Storage Spaces pool and only then remove the physical drives from the Storage Spaces pool so they can be added directly to the DrivePool pool.

1 hour ago, rogue_9 said:

will that let me push the capped capacity I'm stuck at to the actual capacity,

DrivePool's theoretical limit is at least 8 PB (yes, PB, as in petabytes) and only because the Windows OS itself doesn't currently support larger than that (note that some older versions of Windows only support up to 256 TB).

[Christopher (Drashna)]

On 5/31/2023 at 6:19 PM, rogue_9 said:

To make sure I understand, I can just add my current drive pool to StableBit, and will that let me push the capped capacity I'm stuck at to the actual capacity, or will I still be stuck at the capped 63 TB capacity because it is a Windows-based storage pool?

The limit is actually based on the allocation unit size (aka cluster size).  the normal 4kb cluster has that limit.    So each volume will still have this limit, regardless if it is in a pool or not.  However, the pool itself is an emulated drive, and has no clusters/blocks.  So technically, it's limited by what Windows will report, at that point.

Changing the allocation unit size requires a refomatting, so you can't do it "on the fly".  

Another fun caveat is that NTFS doesn't support VSS snapshots on volumes over 64TBs.  Which means no CHKDSK pass on volumes larger than that, as well. 

 

And shane's comments are spot on. As usual.