Leaderboard

Unfortunately there is still no Scanner support for reading SMART from SAS drives.

Drives need to be formatted as NTFS or ReFS to be added to a pool, as DrivePool uses features native to those file systems (e.g. exfat doesn't support alternate data streams, which DrivePool uses to assist with duplication).

Is this just kind of flaky, or am I maybe doing something wrong? I have 3 scanner installations on my lan. When I use the drop down in the Connect to I was seeing one of my remote installations, then I installed on another device yesterday, but now I don't see any other installations in the drop down. All devices have the LAN control option checked.

Hi mikernet. For whatever it is worth, I am successfully using SyncThing with my primary pool (currently only ~ 16TB before duplication) without any fine-tuning, but the content is reasonably static (e.g. photo albums, family videos, financial records, various documents) and I have DrivePool's Read Striping disabled. Note that SyncThing is not intended for situations where changes are likely to occur faster than the ability to distribute those changes between devices (e.g. if device A renames a file and device B also renames that file before it can receive the update from device A, then SyncThing is going to complain). Diving in... DrivePool FileID Notification bug: as I understand it, this risk involves both excessive notifications (reads being classified as writes) and insufficient notifications (e.g. moves not being reported), to which applications that rely on FileID notifications are vulnerable. The original thread for reference. SyncThing uses two independent mechanisms to detect changes, a scheduled full scan (default hourly) and a watcher that listens for notifications (default enabled); link to documentation. This means SyncThing's scheduled scans will catch any moves that are missed by its watcher, resulting in at most a delay of one hour (or as configured) to update other nodes. I do not know how well its watcher would handle excessive notifications if those occur, but it can be easily disabled if the load becomes a problem. DrivePool FileID Generation bug: as I understand it, this risk involves DrivePool's use of an incrementing-from-zero counter that resets on reboot, resulting in data corruption/loss for any application (local or remote) that relies on FileID having cross-boot permanence for identifying files (q.v. above original thread link). As far as I have determined SyncThing should not be affected by this bug so long as its watcher is not being used to monitor pool content via a remote share (e.g. if for some strange esoteric reason you mapped a share "\\pc\pool" to a drive "X:" and then told SyncThing to add "X:\stufftosync" as a folder instead of following the instruction to only add local folders). I'm... actually not sure if the watcher can even do that, but if so that's the only risk. DrivePool Read Striping bug: as I understand, this risk involves DrivePool sometimes returning invalid hashes or even invalid content to certain applications when this feature is enabled to read a duplicated file via concurrent access to the drives that file is stored on. Some systems apparently do not experience this bug despite using the same application as others. The original thread for reference. I have NOT tested striping with SyncThing, as I keep DrivePool's Read Striping pre-emptively disabled rather than have to exhaustively test all the apps I use (and given the possibility that the problem could involve some kind of race condition, which are a PITA to avoid false negatives on, I am happy to continue erring on the side of safety over performance).

If you're not doing anything complicated with your pool, that should be it. Manage Pool -> Re-measure and if you're using duplication Cog Icon -> Troubleshooting -> Recheck duplication if it doesn't do it automatically (if you've got duplication turned on for the whole pool then at the end you should see the big pie chart in the GUI showing no unduplicated data). You can also inspect/alter the duplication levels of folders and subfolders manually via the table provided by Manage Pool -> File Protection -> Folder duplication if you only want duplication for parts of the pool.

This issue is definitely NOT resolved and is really extremely serious. I can't believe it hasn't gotten more attention given the high potential for data corruption. I'm on v2.3.11.1663 (latest at this time) and was highly perplexed at seeing random corruption throughout thousands of files I copied to a linux server via an rsync command. This sent me on a wild goose chase trying to look into bad RAM modules or bugs in rsync but it is now clear that the issue was DrivePool all along (it didn't help that I actually did have some bad RAM on the linux server, but that was red herring as it has since been replaced with ECC RAM that has been tested). After noticing that the source data on a DrivePool volume "seemed" valid but thousands of the files copied to the linux server were corrupt I spent weeks trying to figure out what was going on. Trying to narrow down the issue I started working with individual files. In particular I looked at some MP3 files that were corrupt on the remote side. When I would re-copy a file via rsync with the --checksum parameter it would always report the mismatch and act like it was re-copying the file, but then sometimes the file would STILL be corrupt on the remote side. WTF? Apparently this bug was causing the rsync re-copy to send yet another corrupted version of the file to the remote side, though it would occasionally copy a good version. Super weird and very inconsistent. So then I wrote a Node.js script to iterate through a folder and generate/compare MD5 hashes of source files (on the DrivePool volume) and target files (on the remote linux server). I started with a small dataset of around 4000 files (22 of which were corrupt). Things got even weirder with multiple runs of this script showing different files with mismatched hashes, and I realized it was frequently generating an incorrect hash for the SOURCE file. There could be different results each time the script was run. Sometimes hundreds of files would show a hash mismatch. It's only been a short time since I disabled read-striping so I can't verify that has fixed everything but with read-striping disabled I haven't yet experienced a single corrupt transfer. An rsync command to compare based on checksum completed and fixed all 22 remaining corrupt files. And another couple runs of my hash compare script for the small 4000 file dataset shows no hash mismatches. The only thing preventing this from becoming an utter disaster is that I hadn't yet deleted the source material after copying to the remote server, so I still have the original files to compare and try to repair the whole mess. However, some of the files were already reorganized on the remote server, so it is still going to take a lot of manual work to get everything fixed. Sorry for the rant, but if the devs are not going to actually fix DrivePool I'm about done with this software. There are too many "weird" things going on (not just this particularly bad bug).