Jump to content
  • 0

Can DP Be Instructed to Not Put UN-Duplicated Files on Specified Drives?


GDog

Question

I don't know how to apply the correct terminology for this issue, so I'm not certain if it has been discussed before or not. Please forgive me if it has. I have looked for almost an hour and I can't find anything relevant.

 

Please refer to my Signature for my System Description. It is up to date.

 

About 80-85% of my 35TB Pool is duplicated, but there are some files that just don't warrant the space allocation. That doesn't mean I don't care if those files are lost however, the're just not critical files and I'm trying to be somewhat reasonable. A few of my Drives are starting to get some age on them, like 5+ years. HD Sentinel and Scanner are not reporting any problems at all, but I would like to minimize my risk with these drives if I can. What I would like to do is tell DrivePool to not put any NON-Duplicated files on drives I so designate.

 

Is this possible? If so, how would I do it. I have looked DP over and I can't see how.

 

Thanks for your help,

-Gary

 

 

Link to comment
Share on other sites

19 answers to this question

Recommended Posts

  • 0

Hmmm, I am not sure how exactly as I haven't used DP 1.x for ages (you *can* run DP 2.x on WHS2011, I do too) but it should be possible. Somehow you should be able to access settigns for Balancers and look up the Drive Usage LImiter. Here you get a list of HDDs that are used by the Pool and you can check /uncheck duplicated/unduplicated files. If you uncheck Unduplicated for the HDDs where you don't want them, you are all set. I guess you need to enforce a run of the balancer though.

Link to comment
Share on other sites

  • 0

Yes, you can do this.

 

Specifically, it is the "Drive Usage Limiter" balancer.  

 

Click on "Pool Options" -> "Balancing", and open the "Balancers" tab.  Click on the balancer, and then configure and save.  This will rebalance the data as necessary. 

 

Additionally, the StableBit Scanner balancer can be configured to do the same for any disk with SMART warnings. (it does so for unduplicated and duplicated data, in the case of unreadable/damaged sectors)

Link to comment
Share on other sites

  • 0

Thanks Christopher! You too Umfriend! I will give this a try right away and report back here how it turns out.

 

BTW Christopher: Is it time now I should go ahead and start using DP 2.X? Don't know if you remember, but I already paid for the upgrade when it came out, but switched back to 1.x because at the time, I felt more comfortable with the 1.x integration with WHS2011. Now however, I understand the process a whole lot better than I did then.

 

Thanks again, appreciate your help as always!

Gary

Link to comment
Share on other sites

  • 0

I would definitely recommend the 2.X version of the 1.X.   There are more features and bug fixes and the like in the newer version.  And it's much more actively developed for.  

 

Just uninstall the 1.X version, reboot and install the 2.X version, and reboot again.  you'll be all set after that. 

Link to comment
Share on other sites

  • 0

Thanks Christopher re the 2.X version, and your help with the duplicated file issue! I am happy to report that your instruction re the "Drive Usage Limiter" balancer seems to have worked perfectly! It took a while to get what I was after, since DP had to move a bunch of files around, but once it was all over, it looked just like I wanted. Super! 

 

I have a couple of concerns re upgrading to the 2.x version:

  1. You make the UG sound sooooo simple it scares me a little. Are you saying that if I UN-install 1.x that all my settings and preferences will be retained and picked back up by the 2.x install? 
  2. Take a look at the attached screen shot of the Dashboard. For some reason, the SAME Serial Numbers are being shown for more than one physical drive. It doesn't seem to be causing any problems with file retrieval or anything with v1.x, but I am concerned if i do the upgrade, it might cause a thrashing of all my data when UG to v2.x. What to you think about this? 
    ***ALSO, do you know how to fix this? I have been told that it is the Sonnet Driver I am using for the SuperMicro PCI-X SATA cards. This driver allows HDD's larger than 2TB, so I can't stop using it unless you have a better solution.

I have since ran into a couple more issues on different topics, so I will start a new thread for those.

Thanks Christopher!

Gary

 

 

PS: Last night, I didn't see how to attach a file to this post, but this morning I found it, so the screen shot is here now.

post-5460-0-41803000-1505752375_thumb.png

Link to comment
Share on other sites

  • 0

I'm glad to hear it! 

 

  1. Well, the licensing and duplication settings will be retained.  I'm not sure about the balancing settings.  But these should be relatively simple to reproduce on the new version, if they aren't retained. 
    And duplication settings are actually stored on the pool, so they always are retained, even on a new system or installation.
     
  2. For the serial number, it looks like the enclosure that you're using (the AMS one) is using a JMicron chipset.  In this case, IIRC, this is entirely normal/expected because of how the chipset handles the serial numbers (poorly).  Sort of telling you to buy a new one, that doesn't us JMicron... I'm not sure what we can do here. 
Link to comment
Share on other sites

  • 0

Hi Christopher! 

Re the AMS enclosure, I don't know where you are getting the JMicron chipset thing. True, the box did come with that originally, but I ripped all that out (See BTW: below) and I am just using the box for a place to store & power 5 more drives. If you look at the end of my Signature, you will see that all the drives in that enclosure are being handled by a "SuperMicro AOC-SASLP-MV8 PCIe-4X Controller (in the main case) connected via an Internal SFF-8087 -> to External SFF8088 PCI Mini SAS 26P Adapter Card". HOWEVER, this problem was there before the AMS box and propagates across ALL my drives including the 15 drives in the Antec 900 case on the 2 x AOC-SAT2-MV8 PCI-X Controller Cards. Does that change anything you might be able to comment on? Especially whether this "problem" of duplicating serial numbers would have any effect on the outcome of the DP 2.x Upgrade or not?

 

BTW: I never used the AMS box in it's stock form because it would not accept drives larger than 1TB. ...  Even before I was just using it as storage & power, I had previously replaced the stock controllers with a SiI 3726 Backplane Controller and a SiI 3132 Host. I eventually abandoned that system however because i was getting too many drives being dropped out & just disappearing. Creates havoc with DrivePool, eh?

 

Thanks Christopher,

Gary

Link to comment
Share on other sites

  • 0

Ah, yeah, that would make a difference, actually.

 

But if all of the drives on the same controller are getting the same serial number, it's the controller that is the problem. 

Since we pull the information from the controller.... 

 

 

And no, this won't affect the upgrade, at all. Though, you may not see it as much, because of it. 

Link to comment
Share on other sites

  • 0

Hi again Christopher!


I sure don't mean to beat this thing to death, but I am still left with some questions. If you remember, I like to know WHY things do what they do  ...


 


"Ah, yeah, that would make a difference, actually."


 


Well, I am assuming you are referring to how you COMMENT and not how the Upgrade goes, right?


 


"But if all of the drives on the same controller are getting the same serial number, it's the controller that is the problem. 


Since we pull the information from the controller.... "


 


I *DO* suspect the controller is the problem, but do you mean DP pulls directly from the hardware, or the driver? Reason I am asking is because I am not using the SuperMicro recommended driver for the 2 x PCI-X cards (which is the only place I have seen this issue so far), because it will not see drives larger than 2TB. I am using the Sonnet driver which has been updated 2 years newer than the SM driver and DOES support drives larger than 2TB. Since I am already using 2 x 4TB drives with 3 more 3TB drives about to go into the pool, I can't really go back on this. I got the recommendation to use this driver from one of the members on this Forum several years ago. 


 


Has anyone else reported this Serial Number duplication thing?


 


"And no, this won't affect the upgrade, at all. Though, you may not see it as much, because of it."


 


I'm curious to know why not. How does DP tell the drives apart from one another if not by the Serial Number? Does it use the Windows assigned "Unique ID"? Knowing this would calm my concerns a LOT. HD Sentinel says it uses ONLY the drive's internal serial number, which it somehow reads from the drive itself, not what gets passed through by the driver. They say that is the only way to FOR SURE tell drives apart since the "Unique ID" is only a windows thing that gets assigned when the drive is initialized in Windows. I know these things from direct conversations I have had with the Developer. See why I am so concerned about this?


 


Thanks again Christopher,


Gary

Link to comment
Share on other sites

  • 0

yes, it affects my response, not the upgrade process.   Not all hardware is equal. 

 

 

As for both ....   HD Sentinel uses some methods that we don't support (partially because they're proprietary commands, or in some cases, can cause stability issues).  IIRC, these cards use proprietary methods for querying SMART data. 

 

Unfortunately, we are loath to use proprietary commands, because the ycan cause serious issues if they're not "perfect".   So that's why HDSentinel gets the better info. 

 

So we use the more approved methods, which is why we get back the poor data (because that's what the controller is returning).

 

 

As for the upgrade, the serial numbers aren't show as as visibly, but are still accessible. 

As for how we track the drives, the PoolPart.xxxxx folders. The "xxxxx" is the disk ID, actually.  Additionally, we use some "tags" to identify which pool the drives belong to.   It's also part of why the software will rebuild the pool on a completely new system. 

Link to comment
Share on other sites

  • 0

Hi Christopher,

I got it now. Thanks for the add'l details. No more on the upgrade thing, but I would still like to find a solution for the duplicating serial numbers issue. I found an old thread that I resurrected where a member named SGRX posted about this issue. His description was so identical to mine that I decided to resurrect it appealing to everyone, yourself included of course.

 

The thread is titled re the SiI 3114, but evolved to the SiI 3132 and then to the Super Micro SAT2-MV8 using the same driver as I am using. I couldn't see a Topic number, but the URL for the thread is here:

 

http://community.covecube.com/index.php?/topic/4-drivepool-showing-incorrect-serial-numbers-using-sil-3114-pci-card/?hl=%20serial%20%20numbers

 

My post is #24. Feel free to RE-visit this topic (hoping you will :) ). Maybe you can find that old support ticket you handled for SGRX. Hopefully you had a solution for him.

 

Thanks again,

Gary

Link to comment
Share on other sites

  • 0

Topic number is "4". It's one of the first threads... 

 

 

Could you make sure that you're on the latest version of StableBit Scanner? 

 

http://dl.covecube.com/ScannerWindows/beta/download/StableBit.Scanner_2.5.2.3128_BETA.exe

http://dl.covecube.com/ScannerWhs2/beta/download/StableBit.Scanner_2.5.2.3128_BETA.wssx

 

And if you are, and it's still coming up with the issue, try enabling the "unsafe" direct IO setting, and see if that helps.

http://wiki.covecube.com/StableBit_Scanner_Advanced_Settings

 

 

If that doesn't help, then try turning off this:

DirectIO_DoNotCorrectSize

 

(it should be the "DoNotCorrectSize" in the "DirectIO" section).

Link to comment
Share on other sites

  • 0

Hi Christopher,

Sorry to bother you again. According to the Dashboard, I am using Scanner 2.5.3062 and DrivePool 1.3.7585 (see attached). I looked all over in Scanner and DrivePool for a link to do the upgrades to the latest versions, but I could not find one anywhere. Exactly how does one do these upgrades anyway? It's been so long since i did one I just can't remember. Do I just download an executable from the website and run it or what?

Thanks man. Hopefully I won't have to bother you any more.

Gary

post-5460-0-73971600-1505959805_thumb.png

Link to comment
Share on other sites

  • 0

Since you're using the dashboard version, you'd want to grab this installer:

http://dl.covecube.com/ScannerWhs2/beta/download/StableBit.Scanner_2.5.2.3128_BETA.wssx

 

Just run this on any system with the connector installed. 

Otherwise, you can run it on the server directly.

Link to comment
Share on other sites

  • 0

Hi again Christopher,

Well I **tried** to install the updated files, but both the .wssx and .exe install files failed. Remember, I do not run headless, so these attempts were right from the Server Desktop. Screen shots in parentheses and quotes.

 

I tried the .wssx install first, and right away, I got a popup that the Publisher could not be verified, do I want to proceed? (see "wssx Publisher Not Verified"). I clicked install of course, and after a minute or so, another popup told me that the Install failed (see "wssx Install Failed - Packaged Wrong"). I tried the install again and got the exact same results. 

 

THEN, I decided to try the .exe Setup. Right off, I got a popup that the account already exists (see "exe Account Exists"). I clicked on OK and this popped up 2 or 3 more times. finally, I got a popup that the Setup Failed (see "exe Setup Failed"). There was a link to a LOG in this last popup, so I clicked on that and saved the file (see "StableBit_Scanner_20170921182725.log").

 

Finally, I rebooted the Server and tried them all again. Same results, so I am sending them to you.

 

What now Boss?

Gary

 

post-5460-0-41463200-1506046033_thumb.png

post-5460-0-60884200-1506046051_thumb.png

post-5460-0-16530900-1506046070_thumb.png

post-5460-0-95914600-1506046081_thumb.png

StableBit_Scanner_20170921182725.log

Link to comment
Share on other sites

  • 0

Well, the 70643 error is ... a poorly handled "Pending reboot" error.

 

Try rebooting and try again. 

 

Hi Christopher,

I tried rebooting again. Only this time, I power cycled to get a RAM Diag. System booted fine, but not a single thing changed from my other attempts to install. The .EXE Setup generated another LOG file, which I will attach here. StableBit_Scanner_20170925123726.log

 

So what do I do now? Any ideas?

Thanks for helping,

Gary

 

PS: Would you prefer that I open a Support Ticket on this issue? You seem to be getting a little tired of this thread. 

Link to comment
Share on other sites

  • 0

Sorry for the delay.

 

Opening a ticket is always the best option, to be blunt.

 

 

And no, I'm not "tired" of the thread. I don't check the forums as often as I check tickets.  Tickets get checked every day (usually), but the forum gets checked "when I feel like it". While I try to get to that every day, if I'm not feeling great or I'm feeling tired, I may end up skipping it.  

 

So if it's urgent, DEFINITELY open a ticket. 

 

 

 

That said, the "publisher cannot be verified" issue is non-critical and can be ignored.

The "specified user already exists' error, uninstall the current version and then retry.  If that has issues, let me know.

And like I said above, the 70643 error is a "pending reboot" notification that is poorly handled.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...