Jump to content

  • Log in with Twitter Log in with Windows Live Log In with Google      Sign In   
  • Create Account

Photo

Can DP Be Instructed to Not Put UN-Duplicated Files on Specified Drives?

UN-Duplicated Unduplicated

  • Please log in to reply
16 replies to this topic

#1 GDog (Gary)

GDog (Gary)

    Newbie

  • Members
  • Pip
  • 9 posts
  • LocationOrange County, California

Posted 01 September 2017 - 02:43 AM

I don't know how to apply the correct terminology for this issue, so I'm not certain if it has been discussed before or not. Please forgive me if it has. I have looked for almost an hour and I can't find anything relevant.

 

Please refer to my Signature for my System Description. It is up to date.

 

About 80-85% of my 35TB Pool is duplicated, but there are some files that just don't warrant the space allocation. That doesn't mean I don't care if those files are lost however, the're just not critical files and I'm trying to be somewhat reasonable. A few of my Drives are starting to get some age on them, like 5+ years. HD Sentinel and Scanner are not reporting any problems at all, but I would like to minimize my risk with these drives if I can. What I would like to do is tell DrivePool to not put any NON-Duplicated files on drives I so designate.

 

Is this possible? If so, how would I do it. I have looked DP over and I can't see how.

 

Thanks for your help,

-Gary

 

 


WHS2011, DrivePool 1.x, Scanner 2.x, SuperMicro X7SBE Mobo, Core2 Duo E8500 CPU, 8GB ECC RAM, 2x SuperMicro AOC-SAT2-MV8 PCI-X Controllers with Sonnet Drivers (>2TB HDD's), 13x 3.5" 2TB HDDs + 2x 4TB HDDs in 3x SuperMicro CSE-M35T 5-Slot Cages in modified Antec 900 Case, PLUS 3x 3.5" 2TB HDDs in modified AMS DS-2350J External 5-Slot NAS Box (gutted electronically - used just for storage & power) attached to a SuperMicro AOC-SASLP-MV8 SATA PCIe-4x Controller via Internal SFF-8087 -> to External SFF-8088 PCI Mini SAS 26P Adapter Card. Total HDDs currently @18 w/ 20 Max. VERY reliable system thanks mostly to Christopher Courtney, whose support is simply incredible!

#2 Umfriend

Umfriend

    Advanced Member

  • Members
  • PipPipPip
  • 372 posts

Posted 01 September 2017 - 07:56 AM

Hmmm, I am not sure how exactly as I haven't used DP 1.x for ages (you *can* run DP 2.x on WHS2011, I do too) but it should be possible. Somehow you should be able to access settigns for Balancers and look up the Drive Usage LImiter. Here you get a list of HDDs that are used by the Pool and you can check /uncheck duplicated/unduplicated files. If you uncheck Unduplicated for the HDDs where you don't want them, you are all set. I guess you need to enforce a run of the balancer though.



#3 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,203 posts
  • LocationSan Diego, CA, USA

Posted 01 September 2017 - 09:07 PM

Yes, you can do this.

 

Specifically, it is the "Drive Usage Limiter" balancer.  

 

Click on "Pool Options" -> "Balancing", and open the "Balancers" tab.  Click on the balancer, and then configure and save.  This will rebalance the data as necessary. 

 

Additionally, the StableBit Scanner balancer can be configured to do the same for any disk with SMART warnings. (it does so for unduplicated and duplicated data, in the case of unreadable/damaged sectors)


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#4 GDog (Gary)

GDog (Gary)

    Newbie

  • Members
  • Pip
  • 9 posts
  • LocationOrange County, California

Posted 01 September 2017 - 10:26 PM

Thanks Christopher! You too Umfriend! I will give this a try right away and report back here how it turns out.

 

BTW Christopher: Is it time now I should go ahead and start using DP 2.X? Don't know if you remember, but I already paid for the upgrade when it came out, but switched back to 1.x because at the time, I felt more comfortable with the 1.x integration with WHS2011. Now however, I understand the process a whole lot better than I did then.

 

Thanks again, appreciate your help as always!

Gary


WHS2011, DrivePool 1.x, Scanner 2.x, SuperMicro X7SBE Mobo, Core2 Duo E8500 CPU, 8GB ECC RAM, 2x SuperMicro AOC-SAT2-MV8 PCI-X Controllers with Sonnet Drivers (>2TB HDD's), 13x 3.5" 2TB HDDs + 2x 4TB HDDs in 3x SuperMicro CSE-M35T 5-Slot Cages in modified Antec 900 Case, PLUS 3x 3.5" 2TB HDDs in modified AMS DS-2350J External 5-Slot NAS Box (gutted electronically - used just for storage & power) attached to a SuperMicro AOC-SASLP-MV8 SATA PCIe-4x Controller via Internal SFF-8087 -> to External SFF-8088 PCI Mini SAS 26P Adapter Card. Total HDDs currently @18 w/ 20 Max. VERY reliable system thanks mostly to Christopher Courtney, whose support is simply incredible!

#5 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,203 posts
  • LocationSan Diego, CA, USA

Posted 02 September 2017 - 02:35 AM

I would definitely recommend the 2.X version of the 1.X.   There are more features and bug fixes and the like in the newer version.  And it's much more actively developed for.  

 

Just uninstall the 1.X version, reboot and install the 2.X version, and reboot again.  you'll be all set after that. 


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#6 GDog (Gary)

GDog (Gary)

    Newbie

  • Members
  • Pip
  • 9 posts
  • LocationOrange County, California

Posted 18 September 2017 - 02:18 AM

Thanks Christopher re the 2.X version, and your help with the duplicated file issue! I am happy to report that your instruction re the "Drive Usage Limiter" balancer seems to have worked perfectly! It took a while to get what I was after, since DP had to move a bunch of files around, but once it was all over, it looked just like I wanted. Super! 

 

I have a couple of concerns re upgrading to the 2.x version:

  1. You make the UG sound sooooo simple it scares me a little. Are you saying that if I UN-install 1.x that all my settings and preferences will be retained and picked back up by the 2.x install? 
  2. Take a look at the attached screen shot of the Dashboard. For some reason, the SAME Serial Numbers are being shown for more than one physical drive. It doesn't seem to be causing any problems with file retrieval or anything with v1.x, but I am concerned if i do the upgrade, it might cause a thrashing of all my data when UG to v2.x. What to you think about this? 
    ***ALSO, do you know how to fix this? I have been told that it is the Sonnet Driver I am using for the SuperMicro PCI-X SATA cards. This driver allows HDD's larger than 2TB, so I can't stop using it unless you have a better solution.

I have since ran into a couple more issues on different topics, so I will start a new thread for those.

Thanks Christopher!

Gary

 

 

PS: Last night, I didn't see how to attach a file to this post, but this morning I found it, so the screen shot is here now.

Attached Thumbnails

  • Serial Number Dups-WHS2011 Dashboard-170917-SERVER3.png

WHS2011, DrivePool 1.x, Scanner 2.x, SuperMicro X7SBE Mobo, Core2 Duo E8500 CPU, 8GB ECC RAM, 2x SuperMicro AOC-SAT2-MV8 PCI-X Controllers with Sonnet Drivers (>2TB HDD's), 13x 3.5" 2TB HDDs + 2x 4TB HDDs in 3x SuperMicro CSE-M35T 5-Slot Cages in modified Antec 900 Case, PLUS 3x 3.5" 2TB HDDs in modified AMS DS-2350J External 5-Slot NAS Box (gutted electronically - used just for storage & power) attached to a SuperMicro AOC-SASLP-MV8 SATA PCIe-4x Controller via Internal SFF-8087 -> to External SFF-8088 PCI Mini SAS 26P Adapter Card. Total HDDs currently @18 w/ 20 Max. VERY reliable system thanks mostly to Christopher Courtney, whose support is simply incredible!

#7 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,203 posts
  • LocationSan Diego, CA, USA

Posted 18 September 2017 - 08:54 PM

I'm glad to hear it! 

 

  1. Well, the licensing and duplication settings will be retained.  I'm not sure about the balancing settings.  But these should be relatively simple to reproduce on the new version, if they aren't retained. 
    And duplication settings are actually stored on the pool, so they always are retained, even on a new system or installation.
     
  2. For the serial number, it looks like the enclosure that you're using (the AMS one) is using a JMicron chipset.  In this case, IIRC, this is entirely normal/expected because of how the chipset handles the serial numbers (poorly).  Sort of telling you to buy a new one, that doesn't us JMicron... I'm not sure what we can do here. 

Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#8 GDog (Gary)

GDog (Gary)

    Newbie

  • Members
  • Pip
  • 9 posts
  • LocationOrange County, California

Posted 18 September 2017 - 10:12 PM

Hi Christopher! 

Re the AMS enclosure, I don't know where you are getting the JMicron chipset thing. True, the box did come with that originally, but I ripped all that out (See BTW: below) and I am just using the box for a place to store & power 5 more drives. If you look at the end of my Signature, you will see that all the drives in that enclosure are being handled by a "SuperMicro AOC-SASLP-MV8 PCIe-4X Controller (in the main case) connected via an Internal SFF-8087 -> to External SFF8088 PCI Mini SAS 26P Adapter Card". HOWEVER, this problem was there before the AMS box and propagates across ALL my drives including the 15 drives in the Antec 900 case on the 2 x AOC-SAT2-MV8 PCI-X Controller Cards. Does that change anything you might be able to comment on? Especially whether this "problem" of duplicating serial numbers would have any effect on the outcome of the DP 2.x Upgrade or not?

 

BTW: I never used the AMS box in it's stock form because it would not accept drives larger than 1TB. ...  Even before I was just using it as storage & power, I had previously replaced the stock controllers with a SiI 3726 Backplane Controller and a SiI 3132 Host. I eventually abandoned that system however because i was getting too many drives being dropped out & just disappearing. Creates havoc with DrivePool, eh?

 

Thanks Christopher,

Gary


WHS2011, DrivePool 1.x, Scanner 2.x, SuperMicro X7SBE Mobo, Core2 Duo E8500 CPU, 8GB ECC RAM, 2x SuperMicro AOC-SAT2-MV8 PCI-X Controllers with Sonnet Drivers (>2TB HDD's), 13x 3.5" 2TB HDDs + 2x 4TB HDDs in 3x SuperMicro CSE-M35T 5-Slot Cages in modified Antec 900 Case, PLUS 3x 3.5" 2TB HDDs in modified AMS DS-2350J External 5-Slot NAS Box (gutted electronically - used just for storage & power) attached to a SuperMicro AOC-SASLP-MV8 SATA PCIe-4x Controller via Internal SFF-8087 -> to External SFF-8088 PCI Mini SAS 26P Adapter Card. Total HDDs currently @18 w/ 20 Max. VERY reliable system thanks mostly to Christopher Courtney, whose support is simply incredible!

#9 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,203 posts
  • LocationSan Diego, CA, USA

Posted 18 September 2017 - 11:14 PM

Ah, yeah, that would make a difference, actually.

 

But if all of the drives on the same controller are getting the same serial number, it's the controller that is the problem. 

Since we pull the information from the controller.... 

 

 

And no, this won't affect the upgrade, at all. Though, you may not see it as much, because of it. 


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#10 GDog (Gary)

GDog (Gary)

    Newbie

  • Members
  • Pip
  • 9 posts
  • LocationOrange County, California

Posted 19 September 2017 - 04:21 PM

Hi again Christopher!

I sure don't mean to beat this thing to death, but I am still left with some questions. If you remember, I like to know WHY things do what they do  ...

 

"Ah, yeah, that would make a difference, actually."

 

Well, I am assuming you are referring to how you COMMENT and not how the Upgrade goes, right?

 

"But if all of the drives on the same controller are getting the same serial number, it's the controller that is the problem. 

Since we pull the information from the controller.... "

 

I *DO* suspect the controller is the problem, but do you mean DP pulls directly from the hardware, or the driver? Reason I am asking is because I am not using the SuperMicro recommended driver for the 2 x PCI-X cards (which is the only place I have seen this issue so far), because it will not see drives larger than 2TB. I am using the Sonnet driver which has been updated 2 years newer than the SM driver and DOES support drives larger than 2TB. Since I am already using 2 x 4TB drives with 3 more 3TB drives about to go into the pool, I can't really go back on this. I got the recommendation to use this driver from one of the members on this Forum several years ago. 

 

Has anyone else reported this Serial Number duplication thing?

 

"And no, this won't affect the upgrade, at all. Though, you may not see it as much, because of it."

 

I'm curious to know why not. How does DP tell the drives apart from one another if not by the Serial Number? Does it use the Windows assigned "Unique ID"? Knowing this would calm my concerns a LOT. HD Sentinel says it uses ONLY the drive's internal serial number, which it somehow reads from the drive itself, not what gets passed through by the driver. They say that is the only way to FOR SURE tell drives apart since the "Unique ID" is only a windows thing that gets assigned when the drive is initialized in Windows. I know these things from direct conversations I have had with the Developer. See why I am so concerned about this?

 

Thanks again Christopher,

Gary


WHS2011, DrivePool 1.x, Scanner 2.x, SuperMicro X7SBE Mobo, Core2 Duo E8500 CPU, 8GB ECC RAM, 2x SuperMicro AOC-SAT2-MV8 PCI-X Controllers with Sonnet Drivers (>2TB HDD's), 13x 3.5" 2TB HDDs + 2x 4TB HDDs in 3x SuperMicro CSE-M35T 5-Slot Cages in modified Antec 900 Case, PLUS 3x 3.5" 2TB HDDs in modified AMS DS-2350J External 5-Slot NAS Box (gutted electronically - used just for storage & power) attached to a SuperMicro AOC-SASLP-MV8 SATA PCIe-4x Controller via Internal SFF-8087 -> to External SFF-8088 PCI Mini SAS 26P Adapter Card. Total HDDs currently @18 w/ 20 Max. VERY reliable system thanks mostly to Christopher Courtney, whose support is simply incredible!

#11 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,203 posts
  • LocationSan Diego, CA, USA

Posted 19 September 2017 - 05:22 PM

yes, it affects my response, not the upgrade process.   Not all hardware is equal. 

 

 

As for both ....   HD Sentinel uses some methods that we don't support (partially because they're proprietary commands, or in some cases, can cause stability issues).  IIRC, these cards use proprietary methods for querying SMART data. 

 

Unfortunately, we are loath to use proprietary commands, because the ycan cause serious issues if they're not "perfect".   So that's why HDSentinel gets the better info. 

 

So we use the more approved methods, which is why we get back the poor data (because that's what the controller is returning).

 

 

As for the upgrade, the serial numbers aren't show as as visibly, but are still accessible. 

As for how we track the drives, the PoolPart.xxxxx folders. The "xxxxx" is the disk ID, actually.  Additionally, we use some "tags" to identify which pool the drives belong to.   It's also part of why the software will rebuild the pool on a completely new system. 


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#12 GDog (Gary)

GDog (Gary)

    Newbie

  • Members
  • Pip
  • 9 posts
  • LocationOrange County, California

Posted 19 September 2017 - 11:33 PM

Hi Christopher,

I got it now. Thanks for the add'l details. No more on the upgrade thing, but I would still like to find a solution for the duplicating serial numbers issue. I found an old thread that I resurrected where a member named SGRX posted about this issue. His description was so identical to mine that I decided to resurrect it appealing to everyone, yourself included of course.

 

The thread is titled re the SiI 3114, but evolved to the SiI 3132 and then to the Super Micro SAT2-MV8 using the same driver as I am using. I couldn't see a Topic number, but the URL for the thread is here:

 

http://community.covecube.com/index.php?/topic/4-drivepool-showing-incorrect-serial-numbers-using-sil-3114-pci-card/?hl=%20serial%20%20numbers

 

My post is #24. Feel free to RE-visit this topic (hoping you will :) ). Maybe you can find that old support ticket you handled for SGRX. Hopefully you had a solution for him.

 

Thanks again,

Gary


WHS2011, DrivePool 1.x, Scanner 2.x, SuperMicro X7SBE Mobo, Core2 Duo E8500 CPU, 8GB ECC RAM, 2x SuperMicro AOC-SAT2-MV8 PCI-X Controllers with Sonnet Drivers (>2TB HDD's), 13x 3.5" 2TB HDDs + 2x 4TB HDDs in 3x SuperMicro CSE-M35T 5-Slot Cages in modified Antec 900 Case, PLUS 3x 3.5" 2TB HDDs in modified AMS DS-2350J External 5-Slot NAS Box (gutted electronically - used just for storage & power) attached to a SuperMicro AOC-SASLP-MV8 SATA PCIe-4x Controller via Internal SFF-8087 -> to External SFF-8088 PCI Mini SAS 26P Adapter Card. Total HDDs currently @18 w/ 20 Max. VERY reliable system thanks mostly to Christopher Courtney, whose support is simply incredible!

#13 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,203 posts
  • LocationSan Diego, CA, USA

Posted 20 September 2017 - 02:15 AM

Topic number is "4". It's one of the first threads... 

 

 

Could you make sure that you're on the latest version of StableBit Scanner? 

 

http://dl.covecube.c...2.3128_BETA.exe

http://dl.covecube.c....3128_BETA.wssx

 

And if you are, and it's still coming up with the issue, try enabling the "unsafe" direct IO setting, and see if that helps.

http://wiki.covecube...vanced_Settings

 

 

If that doesn't help, then try turning off this:

DirectIO_DoNotCorrectSize

 

(it should be the "DoNotCorrectSize" in the "DirectIO" section).


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#14 GDog (Gary)

GDog (Gary)

    Newbie

  • Members
  • Pip
  • 9 posts
  • LocationOrange County, California

Posted 21 September 2017 - 02:10 AM

Hi Christopher,

Sorry to bother you again. According to the Dashboard, I am using Scanner 2.5.3062 and DrivePool 1.3.7585 (see attached). I looked all over in Scanner and DrivePool for a link to do the upgrades to the latest versions, but I could not find one anywhere. Exactly how does one do these upgrades anyway? It's been so long since i did one I just can't remember. Do I just download an executable from the website and run it or what?

Thanks man. Hopefully I won't have to bother you any more.

Gary

Attached Thumbnails

  • Windows Home Server 2011 Dashboard-170920-SERVER3.png

WHS2011, DrivePool 1.x, Scanner 2.x, SuperMicro X7SBE Mobo, Core2 Duo E8500 CPU, 8GB ECC RAM, 2x SuperMicro AOC-SAT2-MV8 PCI-X Controllers with Sonnet Drivers (>2TB HDD's), 13x 3.5" 2TB HDDs + 2x 4TB HDDs in 3x SuperMicro CSE-M35T 5-Slot Cages in modified Antec 900 Case, PLUS 3x 3.5" 2TB HDDs in modified AMS DS-2350J External 5-Slot NAS Box (gutted electronically - used just for storage & power) attached to a SuperMicro AOC-SASLP-MV8 SATA PCIe-4x Controller via Internal SFF-8087 -> to External SFF-8088 PCI Mini SAS 26P Adapter Card. Total HDDs currently @18 w/ 20 Max. VERY reliable system thanks mostly to Christopher Courtney, whose support is simply incredible!

#15 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,203 posts
  • LocationSan Diego, CA, USA

Posted 21 September 2017 - 02:13 AM

Since you're using the dashboard version, you'd want to grab this installer:

http://dl.covecube.c....3128_BETA.wssx

 

Just run this on any system with the connector installed. 

Otherwise, you can run it on the server directly.


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#16 GDog (Gary)

GDog (Gary)

    Newbie

  • Members
  • Pip
  • 9 posts
  • LocationOrange County, California

Posted 22 September 2017 - 02:11 AM

Hi again Christopher,

Well I **tried** to install the updated files, but both the .wssx and .exe install files failed. Remember, I do not run headless, so these attempts were right from the Server Desktop. Screen shots in parentheses and quotes.

 

I tried the .wssx install first, and right away, I got a popup that the Publisher could not be verified, do I want to proceed? (see "wssx Publisher Not Verified"). I clicked install of course, and after a minute or so, another popup told me that the Install failed (see "wssx Install Failed - Packaged Wrong"). I tried the install again and got the exact same results. 

 

THEN, I decided to try the .exe Setup. Right off, I got a popup that the account already exists (see "exe Account Exists"). I clicked on OK and this popped up 2 or 3 more times. finally, I got a popup that the Setup Failed (see "exe Setup Failed"). There was a link to a LOG in this last popup, so I clicked on that and saved the file (see "StableBit_Scanner_20170921182725.log").

 

Finally, I rebooted the Server and tried them all again. Same results, so I am sending them to you.

 

What now Boss?

Gary

 

Attached Thumbnails

  • wssx Publisher Not Verified.png
  • exe Account Exists.png
  • wssx Install Failed - Packaged Wrong.png
  • exe Setup Failed.png

Attached Files


WHS2011, DrivePool 1.x, Scanner 2.x, SuperMicro X7SBE Mobo, Core2 Duo E8500 CPU, 8GB ECC RAM, 2x SuperMicro AOC-SAT2-MV8 PCI-X Controllers with Sonnet Drivers (>2TB HDD's), 13x 3.5" 2TB HDDs + 2x 4TB HDDs in 3x SuperMicro CSE-M35T 5-Slot Cages in modified Antec 900 Case, PLUS 3x 3.5" 2TB HDDs in modified AMS DS-2350J External 5-Slot NAS Box (gutted electronically - used just for storage & power) attached to a SuperMicro AOC-SASLP-MV8 SATA PCIe-4x Controller via Internal SFF-8087 -> to External SFF-8088 PCI Mini SAS 26P Adapter Card. Total HDDs currently @18 w/ 20 Max. VERY reliable system thanks mostly to Christopher Courtney, whose support is simply incredible!

#17 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,203 posts
  • LocationSan Diego, CA, USA

Posted Yesterday, 08:30 PM

Well, the 70643 error is ... a poorly handled "Pending reboot" error.

 

Try rebooting and try again. 


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users