Umfriend

So what are the chances I'll be able to run 10G over my Cat5e cables with max length of, say, 25 meters?

You're welcome. I was not actually aware of USB-SATA bridges messing up > 4TB as well, just > 2TB. And >2TB I can understand, >4TB makes no sense to me.

Is the external USB HDD dock documented to support HDDs and volumes in excess of 2TB? Could you try and connect it to anything somehow that is only SATA, whithout a USB-SATA conversion? Perhaps installing it in the machine itself?

Wednesday, logging in and opening explorer I found I had way to much space free and on opening DPUI indeed balancing had dropped and was rebuilding. One consequence of this that my server backups are getting screwed (but, AFAICS, no risk of dataloss). Can I upload anything for you? WHS 2011 - DP 2.2.0.659 BETA.

But if a file is stored on two partitions of the same HDD, will it shows x1 or x2?

Hi, any updates? Still working on Clouddrive I guess? Thing is, my setup is a bit screwed up right now due to this issue and I am looking forward to straighten it out. I guess it is also a *must* for Grouping/Storage Strings.

Until we get Grouping / Strings...

Well, aside from the file-placement issue, I now again have a small amount Unduplicated while I have x2 duplication on everything... Where can I find good documentation on those file-placement limiters? I have one for instance that says: "Un-duplicated target for re-balancing (168KB)" and another that says the same but for -168KB.

As I backup my Server to those Archive drives, I can tell you that having been written to once does not mean you get hit by the write penalty. Somehow, the HDD does track what is occupied and what is not and if you can get sequential writes to clean tracks then it writes like crazy as well (for a spinner). On the other hand, if it gets filled up and needs to replace a lot, then it really kills. I have had Server Backups that took somewhat over 24 hours for little over 4TB of data.

It is probably an outrageous ridiculous question but have you added the new partition to the Pool?

Uhm, I am not sure what happened. I incidently logged into the Server (to change backup HDD) and checked DP. All of a sudden it is doing a huge duplication pass. No idea waht caused it. I will check the backup files like I did a few posts back when it is done. Edit: I did not know this but there is a pop-up when mousing over the duplication bar and it says: "Working on the entire Pool {CRLF} Now processing \ServerFolders\Client Computer Backups. Edit2: So while it was duplicating I ran a dpcmd again and it reported over 14K Inconsistent Files. It almost seems as if DP discarded _all_ duplicates and started to re-duplicate. Mind you, this was not user-initiated. Is this a strategy that can be employed if it does not find a way to achieve (multi-HDD) duplication in a simple manner? For instance, perhaps DP saw both duplicates on the 4TB HDD but could not find the space to move both duplicates to the two 2TB HDDs (but did know that it should have enough space to place all)? What is really strange is that something triggered after having been inconsisten for over a month of time... Edit3: So once it was done (and I did do a Server Reboot as it was taking ages to build a bucket list), the end result is that that files are still duplicated on the 4TB HDD, even though quite a bit was moved around in the rebalancing. Moreover, even though Pool Organisation is 100%, there is now 179 KB unduplicated data. And dpcmd (I save all the runs seperately) does not show a [x1/x1] file and does not report duplication inconsistencies. Mind you, the reporting of Unduplicated was what started this thread. Anything I can do or send over, let me know.

Can I see what he writes (as it is on _my_ ticket ;-) )? Anyway, I do hope it is not something silly I did that is causing you guys to work on this. Thanks for the update. I am not changing anything so if I can provide more info from the system or similar just let me know. Edit: But yeah, the behaviour certainly, to me, seems out-of-spec, right?

No, I do not use File Placement Rules. Always thought of them as evil ;-). I assume that can be seen from the service logs I submitted as well, no?

Yes, that is probably another way of doing this but then how would I be certain the issue never occurs again? Moreover, moving files like this will create havoc with my Server Backup (I do not think it de-duplicates across volumes) and I think I would prefer to have a definite diagnose and solution first, implement once and initialise backups anew then. Somehow I have this feeling it may be an issue that was introduced with the File Placement Rules functionality (but then, what do I know, I am biased here). It may also not be a bug but rather according to spec, which I would really like to know as well. So I guess I will await Alex' analysis.

The 2TB volume limit in WHS2011 you refer to does not apply here. You can use 8TB HDDs and have them in the Pool without any issue. Ever checked Event Viewer for suspect messages? Tried replacing the SATA cable? How is it connected exactly?

I am just wondering, suppose I seeded the G:\ and H:\ poolpart folders by copying duplicates to two volumes on the same HDD. Should DP then see that the file is duplicated but on the same device and correct this? I hope so but I can imagine this to be a bit out of spec. To be sure, I _never_ seed poolparts, I always go through the regular Pool (exactly becuase I want to be sure the files are handled as I think DP does/should).

Well the 100%/at least 1GB trick did not do anything. Remeasuring I have tried. I could do x3 duplication (or x1 duplication which I am pretty sure I did early January). I have considered adding 2x2TB HDDs to the Pool, removing the 4TBB HDD, re-adding it and then removing the 2x2TB etc. However even if the result may look better, there is no certainty now that the integrity of duplication is guaranteed. I would like a version of dpcmd checkpool-fileparts where duplicates stored on the same device is flagged as an error as well. And hopefully you guys find a bug that can be solved so that we at least understand how what I see can be. Meanwhile, I am now backing up all Pool volumes so basically twice the files I want to backup.. Can't risk data loss. Edit: If you look at the earlier DP picture (2nd picture of post #12), the is a dark blue pointer/triangle at the beginning of volume H:\. The text says: :"Un-duplicated target for re-balancing (14.3 GB)". I find it a cryptic message but I am sure you understand what it means.

Nothing changed by the move of DSO...I am really a bit stuck with this. Not only will I actually lose data if HDD 5 fails, my Server Backups do not include that HDD either so it would be really gone! I can think of various things to do but, well, without a current backup I'd rather not try just anything.

- Duplication Space Optimiser (DSO) is enabled, fifth in list (Scanner, Volume Equalisation, Drive Usage Limiter, Prevent Drive Overfill come first). - Pool is at 100% (green, complete width, not sure why it does not also give a percentage, I like numbers better than graphicals). Service logs are uploaded and the dpcmd output from yesterday is in the same zip file. One of the issues I now have with dpcmd is that even though x2 duplicates exist, it should (IMHO) flag when duplicates are stored on the same device as an error. Would like to have confirmed that it will do that in a future release I have moved DSO to 2nd place and started a remeasure, just to see if it will do anythin but I doubt it. Will let you know.

With me it runs off of the standard intel B75 chipset.

I would have had a 100TB Pool if I could just think what to store on it that made at least a little bit of sense. Boys, toys etc.

OK, I am sorry but I think I have a real issue/problem.bug here. The short of it is, duplication does not appear to guarantee storing duplicates on different physical HDDs. I am sure aI am doing something terribly wrong but I would really like some help here. Basically, my current Server Backups are worthless. So let'start with the first picture. It is Diskmgmt and it shows Disk 1 (E:\), Disk 2 (F:\) and Disk 5 (G:\ and H:\). The first two are two 2TB HDDs, the latter a single 4 TB HDD split into two even partitions (save for a MB or 10 max). The second pic shows DP. These four (E:\ to H:\) comprise the Pool, it has x2 duplication. Already you see the duplicated bars on G:\ and H:\ being somewhat longer than on E:\ and F:\. So I decided, finally, to look at the properties of the volumes and found: HDD Volume Files 1 E:\ 145,255.00 2 F:\ 112,656.00 3 G:\ 18,484.00 3 H:\ 37,137.00 That can't be right. The third pics show what explorer shows on Client Backups, which are part of the Pool for the four poolpart folders. Look at the file Commit.dat. It is on G:\ and H:\ and that is the same physical HDD. Same for Data.4096.1.dat, number 3 and 4 as well. Also, dpcmd shows this for Commit.dat: - [2x/2x] P:\ServerFolders\Client Computer Backups\Commit.dat (5.50 KB - 5,632 -> \Device\HarddiskVolume10\PoolPart.ac08c794-b981-42eb-b05b-3994ade7f64b\ServerFolders\Client Computer Backups\Commit.dat [Device 5] -> \Device\HarddiskVolume11\PoolPart.d603fe42-65b1-4dae-89b0-ff9e12b2293c\ServerFolders\Client Computer Backups\Commit.dat [Device 5] I have no File Placement Rules, the default balancers with default settings, pool organisation is 100%. I am pretty sure that when I changed from a 2x2x2TB setup (2 pools of 2x2TB HDDs) to a 4TB+2x2TB HDD I had duplication off and turned it on again during the transition. This transition basically was adding the 4TB HDD to the Pool that consisted of E:\ and F:\ and then moving the Client Backups from the other Pool to this one. It may have been the case that, in order to duplicate these during the transfer, DP would have had to move existing files from E:\ and F:\ to G:\ and H:\ to make room but apparantly that did not happen. HELP! How can I fix this? I _need_ at least one duplicate of each file to be on either E:\ or F:\ (and expected default behaviour to do that for me). I am running WHS2011 and DP 2.2.0.659 BETA.

Just be aware that drive pool, AFAIK, will not allow you to have a database file that is larger than the largest individual HDD in the Pool. It has to be stored on one individual HDD and will not be spread over HDDs. Unless, of course, SQL Server allows you to spread data over multiple files (which I think it does).

Good question and I had checked prior to posting post #4.. I always use default allocation unit size and apparantly they are 4KB for all HDD/partitions. That is what I get as Bytes Per Cluster from FSUTIL.

File is uploaded.