Jump to content
Covecube Inc.
  • 0

Migrating to DP / where is my backdoor?


Question

Hello,

I moved most of my hard drives from my main PC to my Storage / HTPC server.

 

Now currently I am running a

- Windows Raid 1 (duplication) two 3 TB drives (personal Media up to 2011)

- 1 Single 3 TB harddrive (personal other files)

- 1 single 2 TB files which seems to be slowly failing (personal Media 2012+)

- adding two new 3 TB drives next week

 

all of those are backed up to the cloud with crashplan. Crashplan is re-synchronizing right now.

 

I heard much good about drivepool. BUT I am really confused why I hardly cant find any YOUtube videos like reviews or hands one videos (is it not that popular)

 

I just installed a trial and what i read was that it will always write data on the harddrive that is the least full.

Now I always like a backdoor. And it seems drivepool gives you that backdoor. BUT and that seems to be a BIG BUT...

files might be randomly on different harddrives...!!! With 5TB of Photos (lets say 200.000 photos) I see a NIGHTMARE ever wanting to change back....!

 

I DO like the idea that drivepool could duplicate everything, but I am really scared off making a HUGH Mess!!!

Why do I want to use drivepool... for example i like to have all Media in one drive... since i have more than 3 TB data i cannot physically store it in only one HD.

Also currently my Raid1 cant be used from drivepool. I would have to "unraid" and go from there... (but how)

 

any thoughts would really be appreciated!!!

 

Let say I want to Migrate... what would be the best way to do so?

 

Alex

 

 

Link to post
Share on other sites

11 answers to this question

Recommended Posts

  • 0

Well, first, thank you for your interest in our product! 
We are defintiely popular, but we haven't gotten to uploading youtube videos. And there are definitely videos out that do display our products. But they do tend to be parts of longer podcasts. 
But thank you for reminding us that we need to produce a bunch of videos showcasing our products!
 
 
Let me address each of these points:
 
 

it will always write data on the harddrive that is the least full.

Yes and no. By default, it writes new data to the drive with the most free space. So if you have a 3TB, a 2TB, and a 1TB, and duplication enabled for everything. It will fill the 3TB and 2TB first. Once there is only 1TB free, it will alternate between each of the drives. 

 

 

Now I always like a backdoor

I'm not sure what you mean here. At all.

But DrivePool stores the files on normal NTFS volumes. So you can access the data from any system that can access NTFS. 

However, it does store them in hidden "PoolPart" folders on each drive. And yes, it does "spread the data out" by default.

 

 

 

I see a NIGHTMARE ever wanting to change back

Yes, this can be problematic. There isn't a way around it. But .... the simple solution, don't change back. In fact, once you've started using it, I don't think you will want to, actually.

However, there is a solution here. One of the optional balancers, "Ordered File Placement" changes the default file placement strategy. It fills up one disk at a time (or two, with duplication enabled). And it tries to keep the contents of each folder on the same disk.

https://stablebit.com/DrivePool/Plugins

Additionally, the File Placement Rules can allow you to specify which disks that folders, file types, etc end up on, based on rules you set up.

 

This would be good for management, performance and if you decide to move away from DrivePool.

 

 

 

 

 i like to have all Media in one drive... since i have more than 3 TB data i cannot physically store it in only one HD.

Well, once you move to DrivePool and if you decide to stick with it, well, it would all be one one drive, or at least appear that way.

Also, I've posted a bunch of info already about how you could control that file placement to your hearts' desire. :)

 

Also, just FYI, the pool is "portable". As in, if you move the drives over to another system, and install DrivePool, it will immediately recognize the pooled disks and recreate the pool. Same with reinstalling windows. 

 

 

 

 

Also currently my Raid1 cant be used from drivepool

Why not? The only reason that would be true is if you're using Dynamic Disks, as Windows handles that differently.

If you're using a controller card to do this, then you can add that array to the pool, without any problems. Same thing with storage spaces (if that's what you're doing)

 

And once the disk is added, you could "seed" the pool with it's contents, remove the disk from the pool (which will move the contents to the other drives in the pool), break the array, and then add the disks to the pool.

 

Here is the guild for seeding the pool (aka, migrating to StableBit DrivePool):

http://wiki.covecube.com/StableBit_DrivePool_Q4142489

 

 

 

I think I have answered everything you've asked about. If I have missed anything or you have any additional questions, don't hesitate to ask.

 

 

 

Additionally, if you install and purchase StableBit Scanner, StableBit DrivePool will pull information from Scanner. 

And in the case of damage to the disk (bad sectors), DrivePool will automatically move data off of the drive. You can configure the same with SMART errors as well. Additionally, you can set DrivePool to avoid placing data on disks that have exceeded their max temperature threshold as well (as these disks are more likely to experience problems).

Link to post
Share on other sites
  • 0

ok, can someone please help me out?

I have the first raid1 disk imported under drive pool.... should i just add the second one, and DP will see that i have 2 files which are the same and wont start a new "duplication" ?

 

How does the duplication of DP work... ? 

If i have folder duplication active... will DP scan all files in the pool and check if duplicates exist and if they do it will not take action, and if no duplicate is found it will create one.

 

Would it delete files that are present i.e. 5 times if i only have duplication active? If so which files will it delete?

 

Please help... I am eager to get this done... and am scared to mess this up!

Link to post
Share on other sites
  • 0

If the first disk is imported... the other disk may still be a Dynamic disk. If it is... you need to convert it still (and lose the contents on it).

 

If it's a basic disk, then you can just add it, and move the contents into the hidden "PoolPart" folder, remeasure the pool, and you're "done".

 

 

 

 

As for duplication, make sure that is enabled for the folders you want. I'd recommend setting it after adding the second drive, though.

And as for how it works, when you change the status, it will scan the contents of the drives, check on the duplication status of all the files, compare that to what it expects, and then copy or delete files as needed to get to the proper duplication status.

 

However, this will only process files in the "PoolPart" folders, and will only compare files in the same location on the different drives.

Link to post
Share on other sites
  • 0

I converted the second one to basic disk... but I found a tool that lets me keep the contend.

 

 

 

And as for how it works, when you change the status, it will scan the contents of the drives, check on the duplication status of all the files, compare that to what it expects, and then copy or delete files as needed to get to the proper duplication status.

 

how does it know what to delete?  Lets say i have a file 10 times on my disks... how does it know that i want it deleted of drive A for example.

 

 

 

However, this will only process files in the "PoolPart" folders, and will only compare files in the same location on the different drives.

 

 

 

SO only if a file is in the same location it counts as a duplicate...?!?

Link to post
Share on other sites
  • 0

I converted the second one to basic disk... but I found a tool that lets me keep the contend.

 

I would recommend that you be careful with such utilities. They can error out, and ... well lose everything anyways.

 

how does it know what to delete?  Lets say i have a file 10 times on my disks... how does it know that i want it deleted of drive A for example.

 

 

SO only if a file is in the same location it counts as a duplicate...?!?

Just to be clear, when it checks for duplicates, they must be in the EXACT same location on each disk. At least the same location under "PoolPart.xxxxx". 

 

For instance, lets say you have:

D:\PoolPart.123..890\Folder1\textfile1.txt

E:\PoolPart.abc..xyz\Folder1\textfile1.txt

D:\PoolPart.123..890\Folder2\textfile1.txt

E:\PoolPart.abc..xyz\Folder2\textfile2.txt

 

 

When it checks for duplication, it will look at the "Folder1" and "Folder2" folders, and the files in them. If they match, like Folder1\textfile1.txt, it will mark that the file is duplicated, then check to see if it is supposed to be. If it is, then nothing happens. If it isn't, it will delete one copy, leaving only one.

However, it will not mark Folder1\textfile1.txt and Folder2\textfile1.txt as duplicate files. They are in different folders and will be processed separately. The same with textfile1.txt and textfile2.txt in the Folder2 folder, even if both files contain the same contents.

 

 

So yes, only if it's in the same location in the PoolPart folder is it counted as a duplicate, as that is how we store the files. 

 

If that is problem there are programs that will sniff out duplicate files and report them. You should be able to use that on the pool itself, and consolidate the files rather quickly.

If you're using a WHS2011 based system, then "Snoop De-Dupe" is a good add-in that does that (we are not affiliated, nor get anything for recommending them).

Link to post
Share on other sites
  • 0

Just to be clear, when it checks for duplicates, they must be in the EXACT same location on each disk. At least the same location under "PoolPart.xxxxx". 

 

For instance, lets say you have:

D:\PoolPart.123..890\Folder1\textfile1.txt

E:\PoolPart.abc..xyz\Folder1\textfile1.txt

D:\PoolPart.123..890\Folder2\textfile1.txt

E:\PoolPart.abc..xyz\Folder2\textfile2.txt

 

 

When it checks for duplication, it will look at the "Folder1" and "Folder2" folders, and the files in them. If they match, like Folder1\textfile1.txt, it will mark that the file is duplicated, then check to see if it is supposed to be. If it is, then nothing happens. If it isn't, it will delete one copy, leaving only one.

However, it will not mark Folder1\textfile1.txt and Folder2\textfile1.txt as duplicate files. They are in different folders and will be processed separately. The same with textfile1.txt and textfile2.txt in the Folder2 folder, even if both files contain the same contents.

 

 

So yes, only if it's in the same location in the PoolPart folder is it counted as a duplicate, as that is how we store the files. 

 

 

I think this is very IMPORTANT INFO to understand the concept, and makes me feel much better...  It should be better explained in the product description!

 

so everything will be stored in the SAME FOLDER STRUCTURE.... I didnt know this until you explained it... and would have felt much safer to migrate... if explained in the product description.

 

That being said, I understand that the whole concept isnt to easy to break down....

Link to post
Share on other sites
  • 0

Alex,

 

To be honest, most people don't really "need" to know this. For most, that it pools the data, can and does duplicate it, is enough for them.

For those that need or want to know more, we late them ask. It may not be ideal, but as I said, a majority of the people buying it don't care.

 

And yeah, it can be a rather complex concept, and that's not even including the balancing feature (which can get very complicated)

Link to post
Share on other sites
  • 0

ok.... so.... here an update...

 

I synced as much as i could Raid0 from main computer with Photos drive old (slowley failing) to photo drive new.... (with the intention of later removing the old failing one and add a second new one for duplication)

Now i could sync most of it, but i keptr getting fails becasue of I/O errors... So I decided to let 95% sync be good enough (I still have the good original on the Raid0 on main PC to complete the last 5%)

 

disconnected the old drive and put in the new drive. ok... added it to the pool... and well it seemed to start to copy.... My intention was that it should "duplicate" to the new drive since i have the balancer (only one active) to use this drive as the first HD to write on...

 

Now after a few hgours... I found out 2 things that suprised me...

 

eventhough the balancer is set NOT to start by itself...

 

The duplication started!!!

 

and 2nd It didnt use the HD i expected to be use... but the other 2 HD (Photos/1990-2011) where I had a a few GB left, but intended to leave empty... (it was a also a raid1 before... so they were identical) since drivepool used there space to duplicate Photo 2012+ i guess i have part of it on the first (Photos/1990-2011) and another part on the second (Photos/1990-2011)...

 

can u explain why duplication started on its own... and also why it didnt use the HDD I selected?

 

 

Would like some clarification.... since anything that is unexpected or I don't understand i really dislike!

 

 

 

If that is problem there are programs that will sniff out duplicate files and report them. You should be able to use that on the pool itself, and consolidate the files rather quickly.

If you're using a WHS2011 based system, then "Snoop De-Dupe" is a good add-in that does that (we are not affiliated, nor get anything for recommending them).

 

 

 

 

on another note... I am not using a server platform but WIN 8.   Any other software suggestions (freeware preferred)

Link to post
Share on other sites
  • 0

Duplication is handled independantly of balancing. They are two completely different components.

 

Duplication handles the ... well, duplication status of the files (and "fixes" them as needed).

Balancing handles file placement. Where the files should be on the drives.

 

And speaking of which, the default balancing strategy is to place the files on the drive with the most free space available. Only the "Ordered File Placement" balancer (or option in SSD Optimizer) changes that behavior.  

From the sounds of it, you're using the SSD Optimizer (I apologize if you've stated that, but I'm very tired ATM, and wanted to answer you sooner rather than later).   If you did not use the Ordered placement option in the balancer, then when it moves the files off the SSD drive, it will place them on the disk(s) with the most free space still. 

 

 

However, you can use the "File Placement Rules" to control and limit where these files end up, though.

Link to post
Share on other sites
  • 0

Do you mean for finding duplicate files? If so, then no, I don't really have a recommendation. 

I'm ... mildy OCD, so I do all this stuff by hand, and generally don't have an issue with it.

 

Though, I do remember (vaguely) people mentioning Auslogic's Duplicate File Finder software, in the past. 

http://www.auslogics.com/en/software/duplicate-file-finder/

But I haven't used it personally. 

 

 

Though, if you want a huge list of similar utilities:http://alternativeto.net/software/auslogics-duplicate-file-finder/

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...