Jump to content
  • 0

Duplication inconsistent


JazzMan

Question

Now both my pools are showing "Duplication Inconsistent" message in the UI.

 

I think that was true on both the 2.2.0.651 I was running before I upgraded for this thread;

http://community.covecube.com/index.php?/topic/3088-disk-reporting-incorrect-size-after-replacebalance/

as well as 2.2.0.823 now.  The pool automatically re-measured when I rebooted post-install.

 

The drive sizes look ok now, but both (even the pool I did not work on) show duplication inconsistent.  They showed green/ok before I replaced the first drive under 2.2.0.651.  DI message has not been an issue for me before.

 

I ran chkdsk on all drives in both pools and no errors were found.

 

I ran the troubleshooter program and sent you all the data.

Link to comment
Share on other sites

15 answers to this question

Recommended Posts

  • 0

If it's giving the warning that I think it is, it should also give an option to re-duplicate, or at least resolve the inconsistency.

 

Run that (if you don't see it, look for the "^" next to the pool condition bar at the bottom of the UI), and see if it helps.

 

If not, then let me know

Link to comment
Share on other sites

  • 0

Sorry, yeah, the "recheck duplication" option is the one.

 

If that's not fixing the issue, then enable file system logging and re-run the "recheck" option:

http://wiki.covecube.com/StableBit_DrivePool_2.x_Log_Collection

 

And if you could, run this:

http://wiki.covecube.com/StableBit_Troubleshooter

Link to comment
Share on other sites

  • 0

I did run the troubleshooter back on 8/27 and I think it uploaded about 125MB of data.  Hopefully you have that.

 

Does the file system logging need to be active for the whole duplication check, or can I start it a short time after the check starts?

 

My issue is this;  I got a BSOD the first time I ticked that option and then started the re-check duplication.

 

So the system restarted, and the duplication check starts before I can go into the UI and tick the option.  Its running now with the option ticked but a short while after the duplication check had started.  Not sure if there is a way to set this in a config file so it will be active at boot time without needing UI interaction.

 

Is there not a way as an enhancement for the program to be a little more explicit about the inconsistencies it is seeing in the pool?

 

I guess I could try going back to the 2.1 version, but I know it has its own bugs.

 

I did look at the pool I have that is just 2 drives in RAID 1 and the total file size and # of files/folders matched.  What other things make a pool inconsistent .. file attributes and security settings?

 

Thanks,

David

Link to comment
Share on other sites

  • 0

Unfortunately, I'm not really seeing the error or anything related to it here.

 

Just in case, could you try resetting the settings and see if that helps?

http://wiki.covecube.com/StableBit_DrivePool_Q2299585B

 

If not, try rerunning the troubleshooter to get updated/fresh logs (at least for comparison).

And if you could, use "3093" as the contact ID when running it.

Link to comment
Share on other sites

  • 0

OK. Can you give me some idea of the things that cause this "Duplication Inconsistent" message?  And if it could be a false positive?  As I mentioned, when I checked before, I saw the same number of files, folders and total file size on the 2-drive Raid 1 pool.

 

Is there any way to tell the program to not measure the pool, or to select when it measures, or something?

 

At the moment every time i reboot to try to diagnose this  it goes through the hours-long process of re-measuring my 100+ TB pool just to report again that the duplication.is inconsistent.

 

Would the "BitLocker_PoolPartUnlockDetect" have any effect on this?  I don't use bitlocker but did muck about with  that recently as a workaround to the disks-not-spinning-down issue;

http://community.covecube.com/index.php?/topic/2036-disks-never-spin-down/

Link to comment
Share on other sites

  • 0

OK, I have seemed to fix this for now.  Here is what I did.

 

Downgraded to 2.1.

 

Pool D

 

On the pool that is a 2-drive RAID 1 I saw when looking at the Poolpart subdirectories on each drive that there was a subdirectory with a lock overlay icon.  I don't have any fancy permissions on this machine, and I saw from here (https://www.howtogeek.com/howto/17117/remove-the-lock-icon-from-a-folder-in-windows-7/) this can be caused by not having the Users group as part of the permissions.  This was in fact missing from the security list, so I added it to both folders (windows recursed the files and subfolders) and then re-measured the pool and it is now green.

 

I did not look at how that folder was showing in the pool drive, just the poolparts, so perhaps I could have found and fixed it from the pool.  I also do not know if there was an actual difference in the security permissions for the folder or subitems between the two poolpart copies, I just know adding Users seems to have fixed it. 

 

Pool G

 

With so many drives, folders, and files it was impossible to try to find something manually like I stumbled across above for Pool D.  I found this software and loaded it to try to help me (http://cjwdev.co.uk/Software/NtfsReports/Info.html) and pointed it to mount points for each of the individual pooled drives.  The only difference I was seeing for permissions was related to the Recycle Bin.  There were both objects named "Recycle Bin" in the root of the drive, and variously "Recycle Bin" and $RECYCLE.BIN in the "root" of some of the poolpart directories.  The program reported these objects as having different permissions than their parent.  Although I'm guessing that in itself is normal for Windows, since it was the only type of permissions difference I was seeing I went through and deleted all the "Recycle Bin" and $RECYCLE.BIN folders/objects from each pooled drive root and poolpart folder they were in.  I then re-measured the pool and it is also now green.  Again I am not sure exactly waht this was showing in the combined pooled drive.  I don't think the pooled drive showed either a "Recycle Bin" or $RECYCLE.BIN in its root, but I'm not 100% sure.

 

I don't know if it was the permissions, or something else about the Recycle Bin such as the contents, or what causes the two different folder names, but for this pool there seemed to be something Drivepool thought was inconsistent about the replication of the recycle bin.

Link to comment
Share on other sites

  • 0

I just want to add that after fixing this as above I tried to re-upgrade to various beta versions, as 2.2.0.651 had been working, but that version, along with *.823 and *.830 all don't seem stable now on my system.  Even after deleting the recycle bin on each pool it is created on, I get a BSOD usually the first time I go to re-measure a pool after the post-install measure chugs away and finishes.

 

It is really confusing/frustrating as the only change I made (intentionally at least) was to swap 2 4TB drives to 10TB in one pool.The second pool shouldn't be going inconsistent.  I seem to be the only one reporting this, but there seems to maybe be some problem with the Recycle Bin replication and/or handling of 10TB drives in the 2.2 betas.

Link to comment
Share on other sites

  • 0

The permissions issue could/would cause this issue, but only if the SYSTEM account was removed/denied from the files/folder.

 

This is because the service itself handles the duplication issues and resolution (as well as duplication passes ... and anything that isn't real time duplication).  Normally, this account should ALWAYS have full control on everything. But there are cases that this can end up not being the case.

 

So for why, some software does this, Windows will do this in some cases (such as user folders, and folder redirection).

 

 

As for the "$RECYCLE.BIN", this is the actual folder used for the recycle bin. It's hidden by default though. 

If that got corrupted, it could cause issues, but not usually this.

Link to comment
Share on other sites

  • 0

Thanks Chris,

 

I did some more looking in NTFS Permissions Reporter (on the native NTFS PoolPart mount points, not the combined pool) and found that there were some (maybe a few hundred) files and folders where it reported something to the effect that it could not translate the SID for the Owner to a user account.  (sorry I don't have the exact error any more).  I used the Windows TakeOwn command to correct that.  I set the owner to the Administrators group as that seems to be the default of how all the other files/folders were owned.

 

This was done while DP 2.1 is the installed version.  I have not tried to re-upgrade to see if 2.2.0.x behaves better now.  .

 

Again though, DP needs to report better on what inconsistencies it sees in the pool, not just that there are inconsistencies. 

Link to comment
Share on other sites

  • 0

If you moved the pool from one system to another, and you'd used specific user accounts rather than the built in ones, then this would be normal.   

 

Also, there are some cases where this could happen anyways (yay NTFS permissions).

 

As for the takeown stuff, that does work, but we also have a guide on how to "brute force" fix this issue, actually:

http://wiki.covecube.com/StableBit_DrivePool_Q5510455

 

And yes, "Administrators" is the default owner (especially on a newly formatted drive).

 

The big thing here, is that the "SYSTEM" account does need to have full control.  Not just because of our software, but there is a bunch of other Microsoft processes that run in the system account. And if they don't have access, it can cause issues. (For instance, it breaks Windows Search, as well).

 

 

As for the reporting, yeah. But for the NTFS permissions.... "nightmare" is a nice way of describing them.  If you get them working properly, they're great.... but otherwise .... case in point, they can be a nightmare to deal with otherwise. 

Link to comment
Share on other sites

  • 0

SYSTEM should have had access to all the folders/files.  I never had problems accessing the files, and i haven't noticed it missing.  "Users" was missing from some folders/files. 

 

I'm less concerned about how the ACLs got out of whack as I understand things happen.  I never moved the whole pool, but it is possible some of those files were moved or copied into the pool from a portable NTFS drive in such a way as the ACLs were also carried over from the source system.  Or some such thing.

 

Not to be a broken record, because you are always very helpful Chris, but if this is the root cause of the 'duplication inconsistent' or BSODs then DP should log something like "SYSTEM cannot access file \\pool\path\file.ext" not just say "Duplication Inconsistent".  Nothing in the thread above, or any other thread I have seen on the forum, or article I have found in the wiki, confirms for me that file permissions are even a thing that can cause the "Duplication Inconsistent" warning/error.  This tack is something I started on my own trying to find a root cause.

 

I might take the pooled disks offline, try the beta upgrade one more time, bring the small pool back online, and see how it goes.  Overall I'm pleased with the DP product (I've had a few licenses for 4+ years), just not so much with this "Duplication Inconsistent" message and behavior.

 

Cheers.

Link to comment
Share on other sites

  • 0

OK, it took some effort but I seem now to have both pools' duplication consistent.

 

I still think there is some bug in the beta around the triggering for when to remeasure the pool and/or the actual logic that the duplication is not consistent.

 

Here is what I did after fixing all the owner and ACL issues I could find as described above.

 

- set all the drives of pool G offline
- Did not touch Pool D and it was Green/OK as far as duplication consistency
- stopped the drivepool service
- uninstalled 2.1.1.561 from control panel
- rebooted

- installed 2.2.0.833 Beta
- Received the "unable to connect to service" dialog (although the service is started)
- stopped the DrivePool service
- cleaned out the ProgramData and AppData directories for the program as described somewhere in this forum or Wiki as this seems to be a know/documented issue with 2.2 and/or upgrading
- Restarted the drivepool service
- Opened the DrivePool UI and Activated my Key
- Pool D showed as Duplication inconsistent  [WHY?]

- Re-measured pool D
- Pool D now shows Green/OK [if the pool was inconsistent why did I have to manually re-measure it? and why did it miraculously become consistent with no change on disk?]

- set all the drives of pool G online
- Reopened the drivepool UI.  Drive G showed as "Measuring"
- When done measuring, pool was "duplication inconsistent"
- Re-measured pool G
 - Pool G then showed Green/OK

 

- Rebooted cleanly.  System shut down and restarted without forcing any programs to close or BSODs or whatever

 

- Upon reboot, logged in and opened the DP UI.

- Drive D was OK/Green

- However, Pool G was "Measuring..." again [WHY????]

-  After being Measured for the Umteenth time Pool G shows Green again.

 

- Rebooted

- This time Pool G seemed to stay consistent across the reboot and was OK/Green when I logged in to the DP UI after the reboot.

 

Thanks,

David

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...