Jump to content
  • 0

Is it possible to tell where duplicates are stored?


Elijah_Baley

Question

I have my new DrivePool setup to duplicate most of the data I have that I consider important and the duplication process seems to be progressing fine.

 

The process is going pretty slowly but there is over 14 TB of data involved so taking a while is to be expected.

 

What I want to know is if there is any way to find where, in the pool, duplicates are stored for each file and/or which files have actually been duplicated?

 

I know I should trust the software but I am of the school that says "trust" but "verify" so I would just like to know when/if certain files have been duplicated in the process.

 

Right now I only have space, with about 20% extra, for 2X duplication but I hope to, over the next few months, add sufficient storage to go to 3x and maybe then I will feel safe.

 

DrivePool is working well and has proved to be easy to maintain and add or change storage so it really looks like it will be my solution. I have noticed that access to files in the pool seem to be a little slower BUT the files I am storing are almost all audio and video files and speed of access is not a primary concern for those.

 

If I can just get over my small concerns about the duplicating and verifying then I can worry a lot less than before getting DrivePool.

 

Oh, one more question that I think I know the answer to: Is it correct that DrivePool will assure that duplicate files are not stored on the same physical drive. That is if File1.MKV is stored on physical drive 1 then its duplicate will be stored on a different drive in the pool?

 

Sorry if those questions are answered in the documentation but I could not be sure. It seems like much of the documentation was written by a programmer.  :D

 

BTW: I was a programmer for over 30 years starting back when the medium was paper tape and/or punch cards. Hence my documentation read the same way.

Link to comment
Share on other sites

5 answers to this question

Recommended Posts

  • 1

You can use the command  dpcmd get-duplication object  as an administrator, on the machine on which DrivePool is installed, where object is the full path of a folder or file in the pool (e.g. "p:\testfolder" or "p:\testfolder\testfile.txt") to check that the actual duplication matches* the expected duplication of the path, and it will also return which poolparts contain that object.

*Note that due to the way DrivePool works, the actual duplication may exceed the expected duplication for folder objects; this is normal.

Link to comment
Share on other sites

  • 0

Hi

 

Currently there is no way to verify the duplicates other than what the software tells you however this has been a much asked feature request, I belive they call it filesafe.

Alex the developer has been working on the next product some sort of cloud based software I haven't followed it much since I am not interested in it however like you I have been very worried about losing data and Iam eagarly waiting for filesafe and maybe parity there is a feature request section you may want to check it out and leave your own comments.

 

I can confirm drivepool is supposed to place a duplicate file on a different drive again tho there is no way to verify this there is also no way to find out what is stored on which drive they have been some post with homemade scripts that will scan the drives and produce a list but again I am hopefull once the new software is done Alex can maybe and very hopefully Add some of the more requested features.

Link to comment
Share on other sites

  • 0

If you are a programmer then you could perhaps write a bit of code yourself in, say VBA (what I would do as I only know VBA and SQL) that would check the folder/file structure of all underlying drives. Come to think of it, if there would be a demand for it I might give it a go myself (although, it would require Excel as the VBA shell and Excel typically will not be installed on servers I would think...).

 

FileSafe, AFAIK, is a possible new product that will check whether duplicates actually contains the exact same data (to spot possible corruption of one or more of the duplicates).

 

They make a point of it to ensure duplicates are not stored on the same physical HDD, not even if a HDD has more than one partition.

Link to comment
Share on other sites

  • 0

As it has been pointed out here already, no, we don't have any sort of auditing tool. This has been highly requested, and we will see about doing this soon. 

In fact, we plan some significant overhauls to both StableBit Scanner and StableBit DrivePool soon, once StableBit CloudDrive has been finished and released. 

 

As Umfriend has indicated, it should be "trivial" to write a script to identify and compare the disk contents.

Specifically, we store everything in hidden PoolPart.xxxxx folders (where the "xxxxx" is a GUID), and we don't store the files in any unusual formats. So you can read from these directories without any problems.

 

 

Also, DrivePool makes sure that the duplicates are not on the same physical drive. So even if you have multiple partitions on the same disk, it will not place the duplicates on the same physical disk (this is also why we don't support Dynamic disks, as this makes it significantly more complex).

 

As for the documentation..... you'd be spot on. :)

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...