Shane

In simple terms, some of the methods used by copiers to improve performance push the edge of what is safe. To use a driving analogy, the issue with taking shortcuts via back roads is that sometimes you hit loose gravel during a turn or the map doesn't match the terrain. Great when it works, bad when it doesn't. It might be worth running a memory diagnostic on your computer, though, just in case.

That doesn't account for the possibility of any given pool using a mix of duplication levels. Much better I think to stay with your original suggestions of not treating child pool duplication as Other (which I'd think would be doable via fetching the child pool's own measurements) and checking at interval(s) during re-balancing to see if it's satisfied the goalpost conditions.

Thanks for your patience! Following your steps recreated the issue. So... yeah, that's a bug. I'd hazard a guess that because the ability to pool drivepool volumes was added after the ability to pool physical volumes, the pool measuring code hasn't been updated to take that new ability into account. I've reported this issue to Stablebit in a ticket, however you may still wish to report the constantly oscillating balancing as that may be a separate issue (I wasn't able to reproduce that; I did notice some oscillation with my test data set but it gradually decreased and reached equilibrium after some iterations; your suggestion of having it check partway might be useful there too).

... Something's not right. I'm just going to collect all the relevant screenshots here. .. Okay, there's no "Other" data showing up on the four physical disks. ... But it says there's "Other" data on the two respective pool volumes. ... Even though the only other folders apparently contain <1MB, which is a negligible amount when we're looking for 5.59 TB of "Other" data? Wait, do those explorer views still have protected (system) files hidden (Explorer window, View menu, select Options, select View tab, look under Advanced settings, "Hide protected operating system files (Recommended)", defaults to ticked)? Because unless pool measuring has been borked (which would confuse the balancing), you've got 5.59 TB of "Other" data somewhere and one of the possibilities is shadow files in the System Volume Information folder, and I'm not seeing that folder in those screenshots. On that note, just in case it is borked, can you please open the DrivePool GUI and for each of pools E, H and Z - in that order and waiting for each to finish before starting the next - select Manage Pool -> Re-measure...

Okay, I think I can see what you're trying to accomplish and why it's having issues with balancing. The default balancer settings expect that a pool is not competing for its parts' free space with external applications. Thus, if a user is placing data into each of the pools A, B and C, but C also happens to use pools A and B as parts, then unless the balancer settings used for C are adjusted to take this into account you can end up with the oscillation you're seeing - because from the point of view of pool C, data being added directly to pool A or B outside of C's own purview/control is the action of an external application, or "Other". Your options are pretty much: Continue to micro-manage C's balancing yourself. Turn off automatic balancing for C, or have it balance only every so often. Adjust balancing for C until it can tolerate A and B being used independently. Place data into A or B via C instead of directly, by making use of the File Placement feature of balancing. Personally I'd pick #4, but your use-case may vary.

It's possible that the disconnects are the result of the provider (Dropbox in this case). You might have run afoul of their firewalls for any of a number of innocent reasons. You could try throttling your CloudDrive upload/download speed/threads, etc. Otherwise you might need to contact Dropbox to see whether you can get them to investigate. If you feel it's on CloudDrive's end (or Dropbox goes "nope, not us"), you could open a support ticket with StableBit and they could work with you to examine the connection logs in detail to see where exactly the ball is being dropped. Alternatively, if you can still manually download the Stablebit folders from your provider to a local disk, it should be possible to create a local instance from which to extract your backup... @Christopher (Drashna) I've spent some time looking, but I was unable to find anything official in the user manual or wiki or forums about how to manually copy/setup a clouddrive as an instance on a local disk in the event a cloud provider becomes unreliable, only comments that it is indeed doable. It seems like the sort of issue that should have its process formally described somewhere easily found? Or have I just flubbed my search roll?

Incorrect. There may be no data that is not within _a_ pool, but data that is within a pool is not necessarily also within _the_ pool made using that pool. For example: You have disks A, B, C and D. You have pools E (A+B) and F (C+D), and pool G (E+F). A and B will each contain a root hidden folder PoolPart.guidstring (together containing the contents of Pool E). C and D will each contain a root hidden folder PoolPart.guidstring (together containing the contents of Pool F). E and F will each contain a root hidden folder PoolPart.guidstring (together containing the contents of Pool G). (note that "guidstring" is a variable that may differ between poolparts) Thus you will have a hidden folder A:\PoolPart.guidstring\PoolPart.guidstring where data placed in A:\PoolPart.guidstring outside of A:\PoolPart.guidstring\PoolPart.guidstring will show up as Data in E and Other in G, while data placed in A:\ outside of A:\PoolPart.guidstring will show up as Other in both E and G. Can you provide more information on your balancer/placement settings? I would suggest it's possible that your chosen rules and your specific data sets have created a situation where DrivePool can only fulfill a triggered requirement in a way that sets up a different requirement to be triggered on the next pass that will in its own fulfillment in turn trigger the previous requirement. Alternatively, if you want DrivePool's developer to inspect your situation personally (well, remotely) to see if it is due to a bug and/or whether your suggested fix would be feasible, you could use https://stablebit.com/contact to open a support ticket.

Your pooled files should still be on the physical disks (in hidden PoolPart.* folders) even if something has removed the virtual pool "drive". If the pool drive (S:\) is still in Disk Management but just missing its drive letter, you could try right-clicking it and Change Drive Letter to Add a new letter. If not, you could try uninstalling then reinstalling DrivePool. If that too doesn't work, you could try going to the Cog icon in the DrivePool GUI and selecting it then Troubleshooting then Reset all settings...

"Other" data in a pool usually indicates non-pool data taking up space in one of the volumes that make up the pool. E.g. if you had a pool that had 1 TB of data in it, and you added that pool as a "disk" for another pool, then that 1 TB would show up as "Other" in the latter pool. I don't know why that would affecting the balance however.

I imagine you'd need at least the Cove file system drivers, yes. I don't know what else you'd need. Perhaps you could request a PE build (something like how the trial becomes read-only after expiring) via the Stablebit contact form?

A1) Once you've selected "Fill up the disks in the order specified below", you can then tick the Prioritize checkbox in the Unduplicated and/or Duplicated tabs to select the desired order (note: if you first ticked in the former tab, it will also tick it the corresponding checkbox in the lattertab as well for some reason, but if you're not actually using duplication then the latter tab is irrelevant). A2) On the triangles for each drive in the pool (as indicated by the tool tips when the mouse cursor is resting on them): Dark blue downwards triangle indicates balancing target for existing un-duplicated files Light blue downwards triangle indicates balancing target for existing duplicated files Dark red upwards triangle indicates placement limit for new un-duplicated files Light red upwards triangle indicates placement limit for new duplicated files where "balancing target" is the amount of existing files DrivePool calculated should (under ideal conditions) be on that drive based on your settings, and "placement limit" is the amount of new files DrivePool will put on the drive before it tries to put them somewhere else based on your settings. It can't always achieve these goals due to files having different sizes.

Welcome to DrivePool! If "Or this much free space" is ticked, it does override the percentage slider. Prevent Drive Overfill is different to Drive Usage Limiter. The first is "what to do if a drive gets too full" the second is "where should files go". If the former has a higher priority (is higher in the list), it will override the latter. If you're using the All In One plugin, then both the Drive Usage Limiter and Ordered File Placement balancers should be unticked as they are redundant in such case. The StableBit Scanner plugin balancer for DrivePool gets its data from StableBit Scanner, and is available because you've installed the latter software. It sounds like you might want only StableBit Scanner and All In One ticked.

My Windows 7 test PC had no issue opening your website as "secure" in the URL via both Firefox and Chrome, so perhaps you've already fixed it. On the other hand it also blocked a lot of third-party objects (probably advertising trackers?).

Usually there'll be an "X" button to the right of the animated yellow bar that can be used to abort any active operation. Then you could restart the duplicating from the Cog icon -> Troubleshooting -> Recheck duplication. If you hover the mouse cursor over the "Duplicating... (17.2%)" does it display anything informative?

Yes, it'd be the Ordered File Placement balancer (or cjmanca's All In One balancer, which is what I'm currently using instead of multiple others). My programming ability (such as it is) would be enough that I could write a brute-force shell script to organise the pool as needed on a manual/scheduled basis, but an actual balancer plugin would obviously be far superior.

You should keep the parity drive completely separate. Using it for anything else risks the space usable for parity data becoming less than the space used for content data on one of the protected drives. Don't add it to the pool. EDIT: Though if you mean "can I mount the parity drive as a folder rather than using a drive letter", if SnapRAID lets you do that then sure. You just don't want to use the parity drive for anything except parity data.

I can see the advantages. It's not an available option in the current DrivePool version, though I'd imagine it should be possible to add as (or to) a balancer plugin. How does mergerfs/epmfs handle the situation where the file being written doesn't fit in the remaining space of the chosen drive (because you can't guarantee in advance the size of an incoming file)?

It might be possible, but it would get complicated fast if you made use of different levels of duplication in a pool (e.g. if you had a pool where you set duplication x3 for critical folders, x2 for most folders, and x1 (off) for trash folders).

According to your screenshot, there aren't any duplicated files. So yeah, that's odd? If duplication is actually turned off, then DrivePool GUI -> Manage Pool -> File Protection -> Pool file duplication... should be offering to enable it. If it is instead offering to Disable Duplication, then you should click that that to turn off duplication.

Hi, if you're not using drive letters I'd imagine you'd point SnapRAID at the individual hidden drive folders that make up the pool, and you'd want to dedicate (one of) the biggest drive(s) entirely to parity. You shouldn't need to shrink the partitions on the other drives. Remember to turn off automatic balancing on DrivePool to reduce SnapRAID's workload.

I'd imagine it'd depend on there being enough interest? Some of the existing language support has been done via community translations, so if you're willing to help out with that you could use the contact form to offer such to Stablebit?

A note of caution: if you're using caching software to defer writes, then in the event of a crash or power failure you WILL lose any data that hasn't yet been written to the disk. It's not likely to be a problem, at least if you've got a UPS and/or a stable setup, but it's still something you should take into consideration.

As you mentioned, the simplest method is to physically disconnect the drive or take it offline via Windows Disk Management; both will result in DrivePool noticing the disk is missing, allowing you to Remove it without affecting the files - however if you don't rename or remove the poolpart folder (a "hidden" folder on the drive) before reconnecting/onlining the drive again it may be rejoined to the pool. To pre-empt that issue, you can either rename the poolpart folder before disconnecting/offlining the drive (though this may not be possible if there are any open file handles*) or you can open a command prompt as an administrator and enter the following command: dpcmd ignore-poolpart pooldriveletter poolpartfoldername where pooldriveletter is the drive letter (including the colon) of the pool - not the poolpart - and poolpartfoldername is the name of the poolpart folder that is on the drive you wish to remove. This will "tag" the poolpart folder so that DrivePool will immediately disconnect the poolpart and also ignore it in future. For example: dpcmd ignore-poolpart k: PoolPart.9be9560f-2871-4c99-9722-7269f0250cfa * the commands dpcmd list-open-files and dpcmd force-close-open-files can be useful for dealing with this. Just a reminder that if you are using SnapRAID with a pool, generally you should turn off balancing for that pool (to prevent DrivePool moving files between volumes).

@srcrist @Thronic Both of you have made valid points concerning the pros/cons of CloudDrive, rClone and cloud storage. However, unless specific information is still needed, I suggest this topic has been "sufficiently covered".

It requires a separate SSD volume per pool. Note the use of the word "volume" - you could use the same physical SSD for multiple pools by partitioning it into multiple volumes and adding each volume to a different pool, although you'd then have less SSD space available for each individual pool.