Jump to content
Covecube Inc.

Alex

Administrators
  • Content Count

    251
  • Joined

  • Last visited

  • Days Won

    46

Everything posted by Alex

  1. By default, the rules are now ordered like this (top being highest priority): Manually entered rules are inserted at the top. Folder based rules look for a place to insert themselves starting at the bottom and moving towards the top. Once a folder place rule encounters another folder placement rule with the same path depth or a higher path depth it inserts itself right under that rule. If it doesn't find such a rule then it inserts itself at the top of the topmost folder placement rule. The system tries to put any existing rules (prior to priority being implemented) into an order that fits
  2. I've just finished implementing priorities for file placement rules in an internal BETA (build 510). Available here: http://dl.covecube.com/DrivePoolWindows/beta/download/ I've done some preliminary testing and the new priority based rules seem to be working pretty well. I'm going to run this build through the official rounds of testing and release it as a public BETA, if no issues arise.
  3. Yep, that's another way to remove a drive and it's also a "very quick" way. BUT, you have to be absolutely sure that duplication is consistent when you do that. If you physically unplug a drive before every file on the pool is duplicated, some files on the drive that you pulled might not be on the pool. So overall, that's a more "manual" way to do it. When using "Duplicate later" you don't run that risk.
  4. Balancing can be complicated. Because of all the difference balancing scenarios that are possible it's difficult to understand what settings to use for each particular scenario. Christopher has suggested this, and I agree, we need some kind of guides for each particular setup. As for flushing the disk completely, I've implemented that and it's available in the latest BETA (CCR: https://stablebit.com/Admin/IssueAnalysis/2166). Dane, we can do remote support if you'd like, because I'm not entirely sure what's going on here.
  5. Balance immediately is now the default on new installations of 2.X. Overnight balancing started being the default in 1.X because that version was exclusive to WHS 2011 and those machines, I'd imagine for the most part, tend to be on 24 hrs. a day.
  6. Yes, this has come up before, but it would be impractical for me to implement new features into both versions. Even as it is now, with the existing bug reports and feature requests it's taking much longer than I would like to get release final builds out. But I can offer you my assistance in helping you upgrade to v2 on WHS 2011. If you'd like, and this is entirely up to you, we can set up a remote support appointment and I'll take a look at any issues that you're encountering with the upgrade. Just open up a contact request @ http://stablebit.com/Contact and mention this thread.
  7. The way that it's implemented right now is that the quick removal process checks whether the file is on the pool by comparing the file size and the last write time of the file on the pool to the file being removed. If those match the file on the drive being removed is deleted. Obviously hashing the file on the pool would not be so quick and so that would negate the quick removal aspect. I could easily change it to the way that you suggest, but in the past people have been very confused as to what to do with the files left behind by the removal process, so I've tried to minimize that whenev
  8. If your entire pool is duplicated, then the quickest way is to remove the first drive is with the Duplicate files later option. This will remove the drive but it will leave duplication process for later. But if you remove the second drive before the background duplication pass completes, with the same option, then it may take a bit longer because some files that may not have been re-duplicated yet will have to be migrated to the pool. Short answer: Use Duplicate files later. But it does carry some risk, if at the time of drive removal, the single copy that is still left on the pool is
  9. I think that you should remove the ordered file placement plug-in (or disable it by unchecking it) because it's not really doing what you want and will confuse you. From what I understand, to achieve what you want, all you have to do is uncheck Unduplicated under the File Placement Limiter for your C:\ drive. That way D:\ will store all of your unduplicated files and C:\ will store only the duplicated file parts. The other plug-ins can be left at their defaults.
  10. The rules that we're talking about here are a brand new feature of StableBit DrivePool 2.1.0.503 BETA. They are not available in any previous version. See my blog post for full details: http://blog.covecube.com/2014/04/stablebit-drivepool-2-1-0-503-beta-per-folder-balancing/ Download the latest BETA here: http://stablebit.com/DrivePool/Download
  11. As far as regular expression matching, well, first of all it's going to be slower, which is ok for the background balancing pass because that's only done when you change the rules. But we also pass these rules down to the file system driver and it has to evaluate them every time a file is created on the pool. Right now we use a built in function in the Windows kernel that is able to very quickly evaluate whether a path matches a pattern. Moving to regular expressions would mean putting a regular expression parser in the file system driver code. Not an easy task, and it will be slower. It "
  12. No, actually the rules are combined right now. So if you have a file \Media\Movies\MyMovie.MKV, that will match all the rules. The system now combines the rules in a restrictive way. In other words you've just told the system that you don't want that file on Drives 1, 2, 3. If you have no other drives in the pool it will continue on as if there are no restrictions. And you've kind of hit on something that I've been thinking of since I published build 503. I've talked to Christopher about this over here and I think that we have a nice solution to this. It will work like this: Rules de
  13. As far as performance, I did some informal testing today with a RAM disk, comparing file copying speeds of pool vs. direct to RAM disk. The numbers didn't show much of a difference when comparing pool I/O vs. non-pool I/O (about 700 MB/s). I tested both local file access and UNC file access. UNC access was slower both on the pool and direct to disk (~500MB /s). But this was with unduplicated files. I'll try to do some more formal performance testing with duplicated files and see what I can come up with. Perhaps we can optimize duplicated files further.
  14. Basically you might get into a race condition, where the state of all the files are not the same, and another operation might get data that is inconsistent. So you want to make sure that reads / writes are synchronized properly, and you also need to make sure that the cache is consistent, these are probably some of the most complicated things to code when writing a file system.
  15. Sounds like we can start testing with ReFS. We can surely add support, provided that there are no showstoppers. You would not be able to mix ReFS / NTFS on the same pool though.
  16. Very astute. You've just struck a very important aspect of disk I/O, it's called synchronization (by us programmers). Obviously, if such a thing were allowed then you would end up with file corruption. StableBit DrivePool has a well defined locking model to prevent such things from happening (as does the rest of the Windows kernel), and frankly this is most of the work of writing a reliable pooling solution.
  17. There is currently no way to force a full duplication consistency check in StableBit DrivePool 2.X (as there is in 1.X). Is this what you mean?
  18. As far as duplicated file performance, here's how it works: The Windows NT kernel is inherently asynchronous. In other words, performing an operation does not block the caller. You may have experienced this in other Windows applications, you click a button and then the whole Window freezes until some operation completes. This is not how the Windows kernel works (where our pooling is done). The original designers had the foresight to make the whole thing asynchronous. Which essentially means that nothing waits for something else to complete. For example, if I need to read a file I issue a
  19. Setting the slider to 100% should accomplish exactly that. It basically means "balance if any data needs to be moved". Also, the latest internal BETAs do a better job of moving every single file off of a disk when it's being emptied. In particular, here is info on that code change: https://stablebit.com/Admin/IssueAnalysis/2166
  20. That's odd, it was in the logs that I saw. Yes, absolutely. A CRC error should show up in the SMART data instantly and a surface scan will pick it up as well. We store the volume name in an alternate data stream on the pool's root directory. So it's likely that the issue has something to do with that. Can you capture a file trace log of the rename operation like this: Under Settings -> Troubleshooting enable File system logging. Perform the rename operation that results in the error. Then uncheck File system logging. This will capture a file system log of the error that I can look at t
  21. Piotr, I've examined your service dump and I believe that your crash is being caused by the Ordered File Placement plug-in v 1.0.0.1. It seems to be stuck in a perpetual calculating loop. If you update that to the latest (v 1.0.3.4), I think that your issue will be resolved. But reboot your system prior to upgrading to get it "unstuck". Download it here: http://stablebit.com/DrivePool/Plugins In fact, I just looked at the change logs for that plug-in and I can see that Issue #76 was fixed in that version and it was specifically causing perpetual ratio calculation. In addition, I've made 2
  22. Lurifax, Thanks for sending in that memory dump. I've analyzed it and unfortunately it indeed was a bug in build 420. It only affects x86 hosts, which is why you are seeing it. To resolve the issue you can download the latest public BETA right here: http://stablebit.com/DrivePool/Download Or you can get the very latest internal BETA here: http://wiki.covecube.com/Downloads (these contain the very latest fixes as people report them) I'm going to try to get 2.1 out as a release final sooner rather than later, so the fix will be in that as well. However, we have a few other impo
  23. Just to complete IAR 1154: I've done a number of controlled tests and didn't find any slowdowns. Full details here: https://stablebit.com/Admin/IssueAnalysis/1154
  24. I'll see if we can put in the Dashboard integration into the EXE setup package, like we do with StableBit DrivePool 2.X. (New CCR: https://stablebit.com/Admin/IssueAnalysis/1161 ) I'm currently experimenting with a "Metro" theme for the StableBit Scanner when installed on Windows 6.2+. You can see this in action in the latest internal BETAs ( http://wiki.covecube.com/Downloads ) I've fixed the case as well in the latest internal BETAs.( Completed CCR: https://stablebit.com/Admin/IssueAnalysis/1162 )
×
×
  • Create New...