Jump to content
  • 0

Possible to remove drive including its files from pool?


esoul

Question

Hello

My goal is to fill up a lot of drives with data and remove full disk including their data from the pool and add empty disks constantly.

Requirements

  • A drive pool with several drives, where I can add and remove drives at any time without interrupting the file copying process. The pool must be accessible all the time.
  • A file copied into this pool, must be stored on one of the available drives, without being scattered across several drives.
  • If I remove one drive from the pool, the files on that drive will be removed to.
  • If I remove a drive, there is a failsafe, that will first write the full file to the disk, before attempting to remove the drive.
  • If I attach the drive to any system in the world, I need to be able to still read the files. Again.. files must be written to one disk only


Is this possible? I read here, that it is "kinda" possible. But It smells like data corruption. :)

Link to comment
Share on other sites

3 answers to this question

Recommended Posts

  • 0

In order:

  • Pools become read-only while a drive is missing (or if a trial version expires) until that drive is returned or formally removed. This may be a problem for your plan (see bullet point four).
  • Drivepool never splits files. It may split folders.
  • If you try to remove a drive from the pool via the GUI, it will attempt to first move the files on that drive to a different drive (if any) in the pool. If duplication is turned on, duplicated folders will be read-only for the duration.
  • If you remove a drive from the pool any other way, the pool becomes read-only until this is resolved. If a files are being copied to the pool when this happens, the result will depend on the copy method (e.g. a standard Windows Explorer copy window will request permission to continue and will only be able to do so once the pool is writable again).
  • DrivePool uses standard Windows NTFS format (or ReFS if you want) so removed drives can be read by any matching Windows system.

Based on my testing, "dpcmd ignore-poolpart" does not wait for open file handles to be closed. The results are as per bullet point four above.

Renaming a poolpart will also 'disconnect' that drive (volume) from the pool, but poolparts cannot be renamed if they contain open file handles, so this may be a partial workaround (partial, because the pool will still become read-only until you formally remove the 'missing' drive via the GUI (or via dpcmd).

Can you pause the copying at all? Alternatively, can you at least throttle the copying speed from the source? Then you could have the source filling the pool while you empty the pool to destination drives that you swap out as the latter fill (i.e. the drives in the pool act as a buffer so the source never needs to stop copying, so long as you can empty the pool faster than the source can fill it)?

Edit: A third option might be to make a feature request for StableBit to add an advanced setting (advanced because it would be a 'use at own risk' setting) to DrivePool that enables forcing pools to remain writable even when a drive is missing. I don't know how feasible that would be.

 

Link to comment
Share on other sites

  • 0
On 5/16/2021 at 7:20 AM, esoul said:

Hello

My goal is to fill up a lot of drives with data and remove full disk including their data from the pool and add empty disks constantly.

Requirements

  • [1] A drive pool with several drives, where I can add and remove drives at any time without interrupting the file copying process. The pool must be accessible all the time.
  • [2] A file copied into this pool, must be stored on one of the available drives, without being scattered across several drives.
  • [3] If I remove one drive from the pool, the files on that drive will be removed to.
  • [4] If I remove a drive, there is a failsafe, that will first write the full file to the disk, before attempting to remove the drive.
  • [5] If I attach the drive to any system in the world, I need to be able to still read the files. Again.. files must be written to one disk only


Is this possible? I read here, that it is "kinda" possible. But It smells like data corruption. :)

I had a similar goal, but use a different method than what you suggest trying to do with DrivePool HDDs. I added some numbers to your requirements for discussion.

First of all, I don't think DrivePool is capable of performing #1-#4 like you want.

[1] As soon as you remove a drive from DrivePool, the pool goes into a missing disk routine and the pool becomes read-only.

[2] DrivePool will scatter files over various HDDs in the pool. So, if you have an album of 15 tracks, you might have 10 tracks on HDD01 and maybe 5 tracks on HDD03. DrivePool does not split a file, but it will split folders. There are some plugins that, in theory, that will write all files to one HDD until full, then move on to the next HDD in the ordered list. But there are a number of overrides that can happen and I would not completely trust a plugin to keep folders complete and not split files in a folder to other HDDs in the pool.

[3] Yes, if you remove a HDD from the pool, the files on that drive will be removed, but only so far as DrivePool reporting that the HDD is missing and locking down the pool into a read-only mode. I don't think you want that.

[4] DrivePool does not split a file. So any files written to a HDD will be the complete file. But that does not mean that your folder(s) will be complete as the files may be split over any number of HDDs in the pool.

As to [5], you can take any DrivePool HDD and attach it to any other computer and still be able to read the files in the hidden PoolPart directory. Since DrivePool writes complete files to the HDD, the files on that HDD are complete and can be read by any computer. However, the folders might not be complete, as mentioned before, so you might only have 10 tracks of the 15 tracks of the complete album folder. But, those 10 tracks would be complete. Again, I don't think this is the solution you are looking for.

MY METHOD: Here is how I manage my DrivePool data. I have my DrivePool data up and running all the time. When I want to backup or archive any data for offline storage, I pop a HDD into a disk caddy and transfer files from DrivePool to the HDD in the disk caddy using TeraCopy with verification. In that way, DrivePool will read and consolidate all the data from the pool and transfer the complete files and folders to the HDD in the disk caddy. When the HDD in the disk caddy gets full, I label it for storage and put it in my closet. This method ensures [1] the pool is accessible all the time. [2] DrivePool will consolidate and transfer all selected files/folders to the HDD in the disk caddy. [3] If you choose to move the files from DrivePool to the HDD in the disk caddy, then those files will no longer be in the pool freeing up space in DrivePool for other data. [4] I use Teracopy to verify all archive moves. So DrivePool will write the file to the HDD in the caddy, and if the newly written file passes the verification checksum, then, and only then, is the file removed from DrivePool. [5] The HDD in the disk caddy can be removed and placed in any other of my computer systems and it reads just fine as a standard NTFS file formatted HDD.

I think my method achieves your main goal without the problems inherent in removing HDDs from DrivePool. I currently buy bare HDDs which is why I use the hard disk caddy. In the past, I have also used USB HDDs for backup/archive purposes in the same manner. I know my method works for me, but I suspect your proposed approach to removing and adding drives to DrivePool will not work as you want.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...