Migrate from Drive Bender to Drive Pool?

thepregnantgod · March 2, 2014

When both products were in beta, I picked Drive Bender and have been using it since. However, recently, I'm getting a lot of dropped drives which is greatly frustrating. The idea of running Stablebit Scanner along side Drive Pool has me considering the switch.

However, I have 20 drives totaling 60TB in storage with everything duplicated (a hidden duplicate on various drives) so manually recombining stuff is going to be a pain.

That being said, can someone familiar with both programs answer these questions for me:

1. If a drive drops, is there a manual/automatic tool that will identify which primary files were located on the dropped drive and then find the duplicates remaining in the pool and mark them primary then duplicate them?

2. Is there a caching mechanism? What type of read/write should I expect if a file is being copied twice in parallel? All my drives are green 5400rpm drives yet on tests, I'm still getting around 60megs read/100megs write in my current config.

3. If Stablebit scanner finds errors on a drive - what does it do? Some of my reading says it pulls the drive automatically - but then would it initiate (the answer the question 1?)

4. In the same vein, is there a tool to check the integrity of the duplication? Can I scan for every file in a duplicated directory to confirm that it has a duplicate somewhere else in the pool?

Thanks.

Edit:

Larsp has posted an "unofficial" guide on how to migrate the Drive Bender pool to StableBit DrivePool, include a script to handle reorganizing the duplication into a format that StableBit DrivePool will properly handle.

http://community.covecube.com/index.php?/topic/511-migrate-from-drive-bender-to-drive-pool/?p=4554

-Christopher, aka Drashna

Edited July 16, 2014 by Christopher (Drashna)
adding migration guide

larsp · July 16, 2014

I have just migrated from DriveBender and back to DrivePool. I tried DriveBender for a period, but it was just slower than DrivePool. When copying many files to DriveBender (ie starting FileHistory from Windows 8), it just stopped working and I had to restart the server. TV-recordings stored in DriveBender was also very slow and uresponsive in VMC. I never had that kind of problems with DrivePool. I do think that the interface in DriveBender is a little more "fancy" but stability is just more important.

Anyway. To migrate back to DrivePool i did the following (on a window server 2012 essential):

1. Installed DrivePool alongside DriveBender.

2. Release your DriveBender license before uninstalling

3. Set up a duplicate DrivePool on the same disks that DriveBender used.

4. Stopped DriveBender and DrivePool services from services.msc

5. Moved alle folders on physical disks from DriveBender root folder to DrivePool root folder.

6. Used the script below to move the duplicate files from DriveBenders location til DrivePools location (DriveBender stores the duplicate in a subfolder named FOLDER.DUPLICATE.$DRIVEBENDER, so you will have to move all the content in that folder to the folder below).

7. Started DrivePool and uninstalled DriveBender.

8. Used computer management to give DrivePool the same disksletter that DriveBender had.

Bat file to move files stored in FOLDER.DUPLICATE.$DRIVEBENDER to the directory below (hidden files will no longer be hidden as move doesn't move hidden files). This script has to be run on all the physical disks. Stand in the \poolpart.* folder. To try it out, stand in a subfolder where you can't do much harm.

Please note that I give ABSOLUTELY NO GUARANTEE that the script will work for you. Try it out first, and if it seem to work, remove the echo

@echo off
for /r %%f in (FOLDER.DUPLICATE.$DRIVEBENDER) do if exist %%f (
  echo %%f
  cd "%%f"
echo  attrib -h *.*
  if exist *.* (
echo    move *.* ..
  )
  cd ..
echo  rd "%%f"
)

All in all it took just 1 hour to set up and migrate from DriveBender to DrivePool and I did not have to duplicate 6 tb.

Edited June 20, 2018 by Christopher (Drashna)
Make script more obvious

Christopher (Drashna) · March 3, 2014

Welcome, and thank you for considering our products!

To answer your questions:

We do not have a concept of primary or second files when it comes to duplication.
At best, we will read from the faster disk, or from both, depending
This means that recovery from a lost drive is trivial at best. The existing files are checked for their duplication status, and then copied to another drive, as needed
http://stablebit.com/Support/DrivePool/2.X/Manual?Section=File%20Protection
Yes, "read Striping", and as I mentioned, it will read from the faster drive if one of the drives is slower, or it will cache the files to memory and read it from there. More details about it can be found here:
http://stablebit.com/Support/DrivePool/2.X/Manual?Section=Performance%20Options
There is an "Archive Optimizer" balancer plugin that you can install. This balancer allows you to create a "feeder disk" or "landing zone disk", which files are written to and then later moved off of. Add a RAID0 array (or two) or a couple of SSDs, and you'll get much better writes.
If Scanner finds damaged sections on the disk (as in bad sectors), by default it will start "evacuating" the drive. You can change this behavior, if wished (such as move out only unduplicated data).
Additionally, you can (optionally) configured DrivePool to evacuate the drive on SMART errors. And you can set DrivePool to avoid placing new files on drives that have exceeded their rated temperature limit. (as these drives are statistically more likely to fail).
DrivePool does this automatically, and daily. If it detects issues, it will let you know. Or if it's missing a duplicate, it will automatically copy it to another disk. And here is the link again:
http://stablebit.com/Support/DrivePool/2.X/Manual?Section=File%20Protection

I think I've answered everything and adequately. If not, then please ask away. Or if you have any more questions.

As for migrating... somebody else asked this recently. And the best answer I have (and this isn't a good one) is to disable duplication on Drive Bender, completely. And then migrate the pool.

thepregnantgod · March 5, 2014

Thanks. I'm migrating now and though it's tedious with 60TB of data, I enjoy your product. The interface is much cleaner and simpler. I do have some questions regarding standy on disks, but I posted that question to a more relevant thread.

Christopher (Drashna) · March 5, 2014

Yeah, I can definitely understand that.

However, for migrating... for the most part, it will be this:

(optional) unduplicate your pool in Drive Bender (I don't like this, but .... it's the simpliest method)
uninstall Drive Bender
Install DrivePool
Add each disk to a Pool in drivePool
Enable "Show Hidden Files"
Stop the StableBit DrivePool Service (in Services.msc)
Open each disk. There should be a {GUID}, and a "PoolPart.GUID" folder on each disk.
Move the contents of the GUID folder (Drive Bender's pool) into the PoolPart.GUID folder
Repeat this for each disk.
Reset DrivePool's settings:
http://wiki.covecube.com/StableBit_DrivePool_Q2299585B
Start the StableBit DrivePool Service back up
Change duplication status to what you wish.

As you do this, you should notice more and more files appear in the pool.

Additionally, you should notice a "duplication" sub-folder in each folder that DriveBender had duplicated. You'll need to either delete these folders, or move the contents into their parent folders.

The reason for this is that Drive Bender has a clear definition of "original" and "duplicate" files that it maintains. StableBit DrivePool does not. Each disk is identical in structure and folder layout.

So you can understand why I said "just disable duplication first". You may have to go through HUNDREDS of folders to fix this. Or write a script to do so. But I'm not comfortable with scripts that mass delete or move files like this, bad thinks can happen.

alkelly · March 7, 2014

A word of caution, I too have a large pool, 27 drives, and recently moved from Drive Bender to DrivePool. The final result was not great, it seems that DrivePool does not handle large pools as well as Drive Bender, for me performance was very much reduced. I'm now in the process of working out if I should go from one large pool to a number of smaller pools, or switch back to Drive Bender.

Christopher (Drashna) · March 8, 2014

@alkelly,

I'm sorry to hear about the performance hit. That shouldn't be happening.

If you could open a ticket at http://stablebit.com/Contact, I'd love to get this issue resolved for you.

If you're using Read Caching, try disabling that temporarily, and see if that helps.

Also, is Network IO Boost enabled?

And could you do this:

DrivePool v1: http://wiki.covecube.com/StableBit_DrivePool_Log_Collection

DrivePool v2: http://wiki.covecube.com/StableBit_DrivePool_2.x_Log_Collection

Let that run that way for about an hour, and then upload it.

Christopher (Drashna) · July 16, 2014

Larsp,

That looks pretty good, and like it should definitely work. Though, like you said, it may take a while depending on the size of your pool.

joeh · January 30, 2015

I've just performed the migration, however I have to agree with alkelly, my pool is much slower under DrivePool. I have 22 drives, and there is clearly a performance difference. I was concerned that I may have done something wrong, but all appears to be ok, just seems DrivePool does not handle a large pools as well as Drive Bender. Now need to workout what to do, argh, split into smaller pools, or back to Drive Bender??

otispresley · January 30, 2015

There are some options you will want to check in DrivePool. I think these are mentioned somewhere in this thread, but make sure the following are enabled and then try again:

Go to Pool Options > Performance and make sure Read Striping and Network I/O Boost are enabled.

I have 20 drives in my pool, and network transfer rates are near max gigabit speeds (112 MB/s); The max is 125 MB/s.

Christopher (Drashna) · February 3, 2015

I've just performed the migration, however I have to agree with alkelly, my pool is much slower under DrivePool. I have 22 drives, and there is clearly a performance difference. I was concerned that I may have done something wrong, but all appears to be ok, just seems DrivePool does not handle a large pools as well as Drive Bender. Now need to workout what to do, argh, split into smaller pools, or back to Drive Bender??

That is very unusual that it doesn't get good speed.

While I only have 12 drives in my pool, I do usually saturate my gigabit network (~100MB/s).

Read Striping can definitely help, but that should be enabled by default, and only works with duplicated data.

However, Network IO Boost may definitely help out with network performance, but it may use a bit more CPU and memory. And by a "bit" it depends on the hardware in usage.

If the performance issues continue, please open a ticket at "https://stablebit.com/Contact" so we can help you more directly.

tbyt2000 · June 14, 2015

There seems to be a very important piece of information missing here. I just purchased The 3 product package so I can move my 2012 R2 server from DriveBender to Drivepool (having read this thread).

On unistalling DriveBender I was pretty horrified to find that Drivepool doesn't recognise any of my disks.

DriveBender has been supporting ReFS fo so long is it unthinkable for Drivepool not to support ReFS right? Right?

Hmmm.....

I have had lots of problems with NTFS over the years so I'm going to have to switch back now. Money well wasted!

Christopher (Drashna) · June 14, 2015

There seems to be a very important piece of information missing here. I just purchased The 3 product package so I can move my 2012 R2 server from DriveBender to Drivepool (having read this thread).

On unistalling DriveBender I was pretty horrified to find that Drivepool doesn't recognise any of my disks.

DriveBender has been supporting ReFS fo so long is it unthinkable for Drivepool not to support ReFS right? Right?

Hmmm.....

I have had lots of problems with NTFS over the years so I'm going to have to switch back now. Money well wasted!

I'm very sorry to hear about that.

However, there is a technical reason for this.

In part, because we do use some of the file system features that are not universally available in ReFS (File IDs and Alternate Data Streams).

Specifically, we store some information on the pool as Alternate Data Streams (including the duplication status of a folder), which isn't supported in the Server 2012 version of ReFS.

Additionally, there is a performance hit for using ReFS over NTFS.

Regardless, please open a ticket at https://stablebit.com/Contactand we can see about issuing you a full refund of your purchase.

kenwshmt · August 3, 2015

I did 2 drives with drive benders 'remove' drive.. a week later, changed the whole set in one go and am currently moving from the drivebender subdirectories on each drive into the drivepool in a propper file copy.

It will still take a week to do, but importing the full disks into drivebender had many file protection problems, and probably long name problems, the copying, so far, hasnt. i'm getting about 2 TB a day this way.

I am impressed how robust drivepool is, in my fidling with usb lines I've knocked drives off a few times and it put itself back in working shape very quickly.

win 7, 7 drives, 12 TB.

PhilA · December 22, 2020

On 7/16/2014 at 5:23 AM, larsp said:
I have just migrated from DriveBender and back to DrivePool. I tried DriveBender for a period, but it was just slower than DrivePool. When copying many files to DriveBender (ie starting FileHistory from Windows 8), it just stopped working and I had to restart the server. TV-recordings stored in DriveBender was also very slow and uresponsive in VMC. I never had that kind of problems with DrivePool. I do think that the interface in DriveBender is a little more "fancy" but stability is just more important.

Anyway. To migrate back to DrivePool i did the following (on a window server 2012 essential):

1. Installed DrivePool alongside DriveBender.

2. Release your DriveBender license before uninstalling

3. Set up a duplicate DrivePool on the same disks that DriveBender used.

4. Stopped DriveBender and DrivePool services from services.msc

5. Moved alle folders on physical disks from DriveBender root folder to DrivePool root folder.

6. Used the script below to move the duplicate files from DriveBenders location til DrivePools location (DriveBender stores the duplicate in a subfolder named FOLDER.DUPLICATE.$DRIVEBENDER, so you will have to move all the content in that folder to the folder below).

7. Started DrivePool and uninstalled DriveBender.

8. Used computer management to give DrivePool the same disksletter that DriveBender had.

Bat file to move files stored in FOLDER.DUPLICATE.$DRIVEBENDER to the directory below (hidden files will no longer be hidden as move doesn't move hidden files). This script has to be run on all the physical disks. Stand in the \poolpart.* folder. To try it out, stand in a subfolder where you can't do much harm.

Please note that I give ABSOLUTELY NO GUARANTEE that the script will work for you. Try it out first, and if it seem to work, remove the echo
@echo off
for /r %%f in (FOLDER.DUPLICATE.$DRIVEBENDER) do if exist %%f (
  echo %%f
  cd "%%f"
echo  attrib -h -s *.*
  if exist *.* (
echo    move *.* ..
  )
  cd ..
echo  rd "%%f"
)
All in all it took just 1 hour to set up and migrate from DriveBender to DrivePool and I did not have to duplicate 6 tb.

I made one change to the batch file text above -> I added a "-s" to the attrib line. I was getting several files in my duplicate folders that had the system attribute flag set, and when that was set, I could not complete the move and deletion of the directory. Adding that simple "-s" to that line solved the problem.

Jay1988 · August 4, 2021

Do you remove the echo from everywhere, or just the first echo?

On 7/16/2014 at 6:23 AM, larsp said:
I have just migrated from DriveBender and back to DrivePool. I tried DriveBender for a period, but it was just slower than DrivePool. When copying many files to DriveBender (ie starting FileHistory from Windows 8), it just stopped working and I had to restart the server. TV-recordings stored in DriveBender was also very slow and uresponsive in VMC. I never had that kind of problems with DrivePool. I do think that the interface in DriveBender is a little more "fancy" but stability is just more important.

Anyway. To migrate back to DrivePool i did the following (on a window server 2012 essential):

1. Installed DrivePool alongside DriveBender.

2. Release your DriveBender license before uninstalling

3. Set up a duplicate DrivePool on the same disks that DriveBender used.

4. Stopped DriveBender and DrivePool services from services.msc

5. Moved alle folders on physical disks from DriveBender root folder to DrivePool root folder.

6. Used the script below to move the duplicate files from DriveBenders location til DrivePools location (DriveBender stores the duplicate in a subfolder named FOLDER.DUPLICATE.$DRIVEBENDER, so you will have to move all the content in that folder to the folder below).

7. Started DrivePool and uninstalled DriveBender.

8. Used computer management to give DrivePool the same disksletter that DriveBender had.

Bat file to move files stored in FOLDER.DUPLICATE.$DRIVEBENDER to the directory below (hidden files will no longer be hidden as move doesn't move hidden files). This script has to be run on all the physical disks. Stand in the \poolpart.* folder. To try it out, stand in a subfolder where you can't do much harm.

Please note that I give ABSOLUTELY NO GUARANTEE that the script will work for you. Try it out first, and if it seem to work, remove the echo
@echo off
for /r %%f in (FOLDER.DUPLICATE.$DRIVEBENDER) do if exist %%f (
  echo %%f
  cd "%%f"
echo  attrib -h *.*
  if exist *.* (
echo    move *.* ..
  )
  cd ..
echo  rd "%%f"
)
All in all it took just 1 hour to set up and migrate from DriveBender to DrivePool and I did not have to duplicate 6 tb.

This is a good tip!

On 12/21/2020 at 10:22 PM, PhilA said:

I made one change to the batch file text above -> I added a "-s" to the attrib line. I was getting several files in my duplicate folders that had the system attribute flag set, and when that was set, I could not complete the move and deletion of the directory. Adding that simple "-s" to that line solved the problem.

My question is, is the script end up looking like this? #1

for /r %%f in (FOLDER.DUPLICATE.$DRIVEBENDER) do if exist %%f (
  echo %%f
  cd "%%f"
echo  attrib -h -s *.*
  if exist *.* (
echo    move *.* ..
  )
  cd ..
echo  rd "%%f"
)

or like this? #2?

for /r %%f in (FOLDER.DUPLICATE.$DRIVEBENDER) do if exist %%f (
   %%f
  cd "%%f"
attrib -h -s *.*
  if exist *.* (
    move *.* ..
  )
  cd ..
  rd "%%f"
)

RudgerWolvram · July 26, 2023

I ran into a small issue with this. That issue is that I had dynamic disks instead of basic. Luckily I had about 45% space used on my DriveBender pool.

If you happen to fall into this category (less than half space used. I used windows Robocopy with some Pwershell to clear the duplicates.

Idea: Break down the DriveBender pool by stopping the service, (note this will cause the LanManServer or Server process to die because some brilliant Dev decided to make DriveBender dependent on the Server Service in windows.) I didn't uninstall DriveBender, I just killed the mount point, stopped the service and then set it to manual instead of automatic (this is all done in services.msc and the properties of the serivce). Fixing the dependencies is below.

To Fix the dependancy problem, on an admin CMD run:

sc qc LanmanServer

This will return something like this:

SERVICE_NAME: LanmanServer
        TYPE               : 20  WIN32_SHARE_PROCESS
        START_TYPE         : 2   AUTO_START
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : C:\WINDOWS\system32\svchost.exe -k netsvcs -p
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : Server
        DEPENDENCIES       : SamSS
                           : Srv2
                           : DriveBender
        SERVICE_START_NAME : LocalSystem

Note the names of the Dependencies and copy them. SC doesn't allow single service dependency manipulation so you have to overwrite the whole thing, luckily it's fairly easy if you know, the syntax. In my instance I had 2 other services that depend on Server that are part of windows natively. It's just a matter of adding them in with the / syntax below.

sc config LanmanServer depend="SamSS/Srv2"

Then run the query to see that it worked.

C:\WINDOWS\system32>sc qc LanmanServer
[SC] QueryServiceConfig SUCCESS

SERVICE_NAME: LanmanServer
        TYPE               : 20  WIN32_SHARE_PROCESS
        START_TYPE         : 2   AUTO_START
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : C:\WINDOWS\system32\svchost.exe -k netsvcs -p
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : Server
        DEPENDENCIES       : SamSS
                           : Srv2
        SERVICE_START_NAME : LocalSystem

note DriveBender is not there anymore.

Now on to the date moving. I assigned drive letters to each of my Drives. Then I used the following robocopy to move all the data off one drive to another (being at 45% or less makes this easy as there is enough space to do the copy) if you also have disks that are all the same size or have enough free space to fit it.

In my instance I had 4 2tb drives. For sanity I moved data off the odd numbered drives to even numbered drives.

robocopy N:\ O:\ /e /j /COPY:DTSO /A-:HS /MT /r:1 /w:1 /reg /log:C:\temp\DP1-DP2_copy.log

/E copies everything including empty
/J users unbuffered IO, good for large files.
/COPY: (D)ata(T)imestamps(S)ecurity(O)wner A is left off because of the System and Hidden file problem mentioned in this original post.
/A-:HS tells it to remove the Hidden and System attributes on the files and folders in the copy.
/MT for multithread, use /log to increase performance
/R and /W is just setting Retry and Wait times, if this is used it retries forever.
/REG writes the retry and wait times to the registry as the new default for robocopy. I do this just because no one wants to wait forever for a file to be unlocked and I don't want to do the /r /w every time
/LOG is any path you want to store the log at, recommended as /MT will produce really weird output that doesn't any queuing on console output, so it's a jumbled mess.

I did this for both drives, just copying everything, including duplicate data. However, you can run the powershell command below to remove all duplicates if you aren't afraid of a disk failure mid-migration and will free up space. You can run it against all of your drives similar to the bat file listed above.

$Path = "N:\"
# This path should be the root of whatever drive you want to clear. Set it each time for each drive.
# Command to delete all duplicate data
Get-ChildItem $path -Recurse -Force -Directory -Include 'FOLDER.DUPLICATE.$DRIVEBENDER' | Remove-Item -Recurse -Confirm:$false -Force -WhatIf
# Remove the WhatIf to actually do the Delete but test it first to make sure it looks OK

Then, Once the data has been moved, I used compmgmt.msc to kills the migrated drives, and re-build them as simple/basic volumes.
Once they were simple/basic I went to the DrivePool console and created the pool with those 2 drives. You should have a D:\ named DrivePool, or whatever Drive letter was the next in line for use in windows.

Then, it's just a copy back. In this instance my O: is one of the drives that contain 2 drives worth of data that has also been cleaned by the powershell command. D:\ is the new DrivePool. Note /r /w and /reg aren't there as they were set in the registry in the previous commands. But it is the same command as before. However, we will be copying from the GUID named folder from DriveBender to the root of the DrivePool. You will need to copy the folder name from your own DriveBender as it is unique for each install. Below is just an example that needs to be altered for your instance.

robocopy O:\{A71B3760-F8BE-4BD8-965D-A819032005BC} D:\ /e /j /COPY:DTSO /A-:HS /MT /log:C:\temp\DP2-Storage_copy.log

Once all the data has been copied into the pool, you can then rebuild the 2 remaining disks and add them to the pool as described above.

Sign In

Migrate from Drive Bender to Drive Pool?

Question

thepregnantgod

16 answers to this question

Recommended Posts

larsp

Christopher (Drashna)

thepregnantgod

Christopher (Drashna)

alkelly

Christopher (Drashna)

Christopher (Drashna)

joeh

otispresley

Christopher (Drashna)

tbyt2000

Christopher (Drashna)

kenwshmt

PhilA

Jay1988

RudgerWolvram

Join the conversation

Browse

Activity