Umfriend

@riahc3: How much data are we talking about and what kind of backup solution(s) do you (want) to use? As an example, I can not imagine image restore or BMR to work. Personally, I use Windows Server Backup (which uses VSS) to backup individual drives (and yes, I have x2 duplication but manage to backup the files only once by using hierarchical Pools).

Now I am rather ignorant on these kind of things but... how do other disk driver creators deal with this? Or is MS the only one?

I understand the concern. I use DP with x2 duplication and Windows Server Backup and that indeed relies on VSS. So what I have done is I created two Pool (Q:\ and R:\) with x1 (so no) duplication and then created another Pool P:\ that consists of Pools Q:\ and R:\ and Pool P:\ does have x2 duplication. This way I have the best of both worlds I guess, I got x2 duplication and x1 backup. It would work similarly with x3 duplication, i.e., create three Pools without and then add them to a Pool with x3 duplication etc. It is not without its compromises, you would want, for instance, each underlying Pool to be roughly the same size and as if and when a disk in the Pool that is being backed up fails, you may need to review the backup settings (as it may need to point to a replacement disk).

By making calls to the VSS service through the API.

I am thinking it is a permissions issue. You might try http://wiki.covecube.com/StableBit_DrivePool_Q5510455 on that drive.

I am thinking it may be a permissions issue. You could try http://wiki.covecube.com/StableBit_DrivePool_Q5510455 But if DP is complaining about missing disks then you may need to remove them in the GUI.

It is possible but I would advise to copy it to the intended target disk but outside of the PoolPart.* folder. Then, if/once succesfull, delete from Pool and then simply move on target disk from location to intended location within PoolPart.* folder.

I strongly suspect that when Scanner found issues with the drive, it told DP to evacuatie it., thus filling the other drives. You're welcome.

Awesome! Yes, install that drive, add it to the Pool, click on remove for the faulty drive and let DP do its magic. It may take quite some time but, especially as I think you do not have duplication, you want to get the Pool healthy ASAP. The faulty drive can be tested, there are various ways and tools to do so. If you can RMA, then that is the way to go, if not you can try that. In and by itselfm a few damaged sectors is no issue (I have them one one HDD due to a power failure). However, if you can get the number of damaged sectors to increase then I would not use it at all other than to experiment with disaster recovery On permissions, yes, that is rather common I think. Do this http://wiki.covecube.com/StableBit_DrivePool_Q5510455. Personally, I would do this after restoring the Pool to health.

How about moving it through the Pool itself? It will save two copies immediately without having to do a check (I would think). I am assuming you currently move through the underlying drives. Or is it that you do the moves within the same drive (which is very fast)?

Oh, and if you run scanner, then possibly it is also a high priority balancer in DP (Open DP -> Manage Pool -> Balancing -> Balancers - StableBit Scanner -> What are the settings of File evacuation? It may be the other HDDs are running full because DP is trying to evacuate the drive. Any chance you have older smaller HDDs lying around and spare connectors?

In Scanner, right-click the row with the column names and select it to show the drive letter. I am assuming you know which drive letter is which actual HDD. If not, well, it becomes a bit messy with, for instance, power down - disconnect all but one, power up. DP will complain about missing disks but leave that for now. Is the disk that is online faulty? You got it. No? Then shut down, add one other disk, power up, rinse and repeat. It is a nuisance but I know of no better way. While you are at it, consider actually labeling them

Saw your PM. I am not official support, I just help out now and then. In your case, if I understand correctly, the only thing that is currently giving issues is that one HDD, right? I would get a new HDD, probably a bigger one. Connect, put in Pool, then remove the faulty HDD. You could still use it but I would perform a burst-test or somesuch and if the number of faulty sectors increases then I would say it is a goner.

Sure. That is one of the reasons I have my data drives connect to a SAS-HBA card, just two cables, in my case, to disconnect from the SAS card ;). But yeah, I can see how it would be nice to be able to complete the full installation prior to DP starting to do its magic. I guess once DP starts to run, it will first remeasure and you can stop that through the GUI but it would have to be attended, you'd have to be quick about it and be aware of this.

One thing, if you use balancers that are not available by default but rather as a plug-in, then there may be an issue and I think you would need to install the plug-in first before reconnecting the Pool-HDDs (see https://community.covecube.com/index.php?/topic/4261-reinstall-wtf/#comment-28626.

Yes, you did state that. I just had not taken into account that that plug-in might not be available at some point. However, it seems to me that if you physically disconnect the Pool prior to re-install, then do the re-install, install DP AND install the plug-in and _then_ attach the Pool, it should work fine. Can't be sure but that is what I would think.

Hmm, I had not thought about plug-ins that are not installed by default. I can't be sure but in the screenprint, is that right after re-install and prior to the installation of the SSD Optimzer plug-in? If so, then I notice that the plug-in is in fact listed so DP did pick up something at least. Perhaps the unavailability caused default balancing setting then. Did you also use File Placement Rules and, if so, were they intact or not?

No need to first activiate the trial version. If you have a Pool, you can actually physically disconnect and move over to another PC. If DP is installed there, it will recognise the Pool and the settings and you are done! So yes. Personally, I would physically disconnect all drives before installing Windows (just to ensure you do not select the wrong disk and to avoid any funny placement of boot partitions), install windows, reconnect the Pooled HDDs and install DP. That's all.

If you are using Seagate Archive HDDs, then this may well be because you use Seagate Archive HDDs. I have had cases where I would get high write speads for a long long time on these but that basically required the HDD to be rather empty and sequential writes. If the HDD has some data on it then this occurs indeed. Having said that, I have never seen writes to the Archives be fast, slow and then fast (as in > 40MB/s) again, so something else may be going on as well.

As I said, afaik the settings are stored and applied at connect time. OP has not reacted but if there is a thread or experience that refutes that then I would be very interested.

It must be me but afaik, the setting are stored in the pools itself and will apply as soon as you connect the pool. Have you a thread where it says otherwise?

The thing is, the question was not to keep files together on a disk, it was to not have them on disk D to the extent possible. File Placement Rules may do that for you but I am not sure once disks B & C are full. And keeping files on the same HDD using FPR will not work with a *.* rule. You would actually need to work out which folders to place where etc. and it could cause folder X, targeted for C, being split between Disk B and D when C is full.

Yeah, but read the notes. Depending on the setting, as I read it, it will either work only at the time of file creation or actually move existing files. That would be nice, I think, for you as, if and when you delete files from HDD E and/or F so that it can empty G.

Yes, that is the idea. I don;t think that file placement rules are the best way deal with this for you. Rather, I would try the Ordered File Placement balancer. It is not installed by default, you'll need to download and install a plug-in (https://stablebit.com/DrivePool/Plugins). Read the notes carefully as the default behaviour is not what you want but it has the options to get it suited for you I think. Caveat: I have not used this plug-in. Oh, and on: I would configure duplication first, so set-up, set duplication, stop service, move files, start service, wait. I have to say, I am rather curious about the exact HDDs you are using. And, to be frank, if a disk is suspect I would at the least ensure I have enough disks in any Pool to ensure that, should it fail, DP has the space available to reduplicate to other disks (and as such I would have a second large fast HDD in the Pool with the 1 big disk). Sure, you would already have duplication but in case of a disk failure, say the big fast one, you would still suffer downtime as the Pool will be in read-only mode and you would not actually have duplication until the matter is resolved. Also, got scanner? I would advise it.

None of the drives are in a Pool yet? Then it is simple: No. In fact, DP will not delete any data on any HDD when you add it to the Pool and such data will not show up in the Pool(s). Imagine a disk D:\ with data on it in the root and a folder D:\Data. When you add that disk to a Pool, DP will create a hidden PoolPart.* folder. So you would have: D:\ D:\Data\ D:\PoolPart.*\ Only what is in the PoolPart.* folder is part of the Pool and everything that has to do with duplication and balancing only applies to the contents of that folder. Root and D:\Data will not be affected in any way. So let's assume you have disks D:\ to G:\ with D:\ being the big fast disk. What I would do is: 1. Create Pool Q, add only disk D:\ 2. Through explorer, _move_ the data on D:\ that you want to be in the Pool to the hidden D:\PoolPart.* folder. 3. Create Pool R, add disks E:\ F:\ and G:\ And here it becomes tricky. If you now create a Pool P by adding "disks" Q and R, with x2 duplication, DP will think that R does not have a duplicate (because the PoolPart.* folders are empty) and copy everything from Q to R. You *could*, I guess, first move the data on E:\ to G:\ (disk by disk) to the PoolPart.* folders. However, I am not sure how DP will react to potential differences in time stamps on the files (date created/modified). So, personally, I would copy all data from E to G to some other disk (external perhaps) and then format them first. Then do step 3 and then 4. Create Pool P using Q and R and set duplication to x2. 5. Let DP do its magic (which may run for quite a bit of time). If there is no possibility to backup the data but the data on Q is less in size then the free space on R, then you could still do steps 4 and 5 and then, after you are satisfied that everything is in order, delete the data on E to G that is not in the PoolPart.* folder (but was present on D:\ or Q:\, don;'t delete data that you have not elsewhere). Failing that space, you could consider simply formatting and do steps 4 and 5 with the idea that you still have a copy on D:\ (=Q:\) and will have duplicates again soon. Alternatively, you could move on those disks but again, I am not sure what issues you might run into. As an example, say there is a file called "MyFile.doc". If on Q:\ it is in folder "MyFiles" and on R:\ it is in folder "MyBackupFiles", then DP will not know that these are the same files. Rather, it will create duplicates of both causing 4 copies to exist. One last thing: If you ever use the move files trick to "seed" a Pool, remember tostop the DrivePoolService before you do that and restart it when finished. Hope this helps.