Jump to content
Covecube Inc.

Umfriend

Members
  • Content Count

    936
  • Joined

  • Last visited

  • Days Won

    48

Posts posted by Umfriend

  1. 6 hours ago, bnhf said:

    I've been busy re-jiggering my drives after migrating a server from Windows Home Server 2011 (with DrivePool) to Windows 10. Several drives have SMART warnings, so I'm in the process of removing them from the pool and adding replacements.  It's going fine, but I've been doing them one at a time, which is a tad slow.  My question is, for those that have actually done it before -- is it OK to add and remove multiple drives at the same time from DrivePool?

    I assume you mean whether it is OK to remove multiple drives at the same time from the Pool through the GUI, not actually pulling them physically from the PC (which you might do after the removal from the Pool is done of course). In that case, yes, it is fine. However, I *think* it is sequential, it would not actually empty them simultaneously and in fact, the first drive you'll empty will actually places files on the other disks, including the one you want to remove (I am pretty sure I read on these forums). It may be better to use the Drive Usage Limiter balancer. If you, for the relevant disks, uncheck both duplicated and unduplicated on all drives you want to empty (and assuming you have suffient space available on the ones you do not want to empty) and the remeasure/rebalance, DP will empty those drives out without placing them on the others being emptied. Once that is done you can remove them through the GUI and that removal will be very fast.

  2. You might want to raise a ticket for support. AFAIK, this should not happen. When I copy a large file to my Pool (say 30GB), I get close to the 1GbE limit. Granted, it is a smaller Pool but also Hierarchical (so I would suffer any DP overhead roughly twice) and duplicated to two subpools, each 3 HDDs, no SSD Cache.

  3. So, thanks to this forum I got me an IBM M1015 (actually, a Dell H310, same difference) that is working out great for quite a few years now.

    But now I am wondering whether there is a 4/16-port SAS/SATA card with a wider and more recent PCIe bus that works well as an HBA and is cheap 2nd hand. Basically, is there an improved alternative to the IBM M1015 with a relatively similar value for money?

  4. 21 hours ago, fluffybunnyuk said:

    but it has the advantage of the free space being at the front of the disk, and the head doesnt have to seek any further than 33% into the platters unless

    Thats 12x per year or a workload of 144TB on a 12TB drive (Ironwolf have 155TB/year usage).

     

    LOL, I remember using only like 50% of a HDD (so just one 50% volume on a HDD, the rest not used/allocated) to have better seek times. Made a real difference in synthetics. IRL? Not so much IMHO.

    On workload, AFAIK, the stated limitation on (or better, guaranteed) use of the IronWolves is 180TB/year but that is in writes. Scanning reads.

  5. With DP, we don't talk about original and copies. You can have either a single instance of each file, that is x1. If x2, then you have two duplicates. So if you want two copies of each file, x2 is the way to go.

  6. 1 minute ago, Christopher (Drashna) said:

    In theory, that shouldn't be a problem, as the "CoveFs_SynchronizeDirectoryTimes" should be enabled by default, and should keep the directories in sync.  Though, if there are empty folders on one or more disks, it may be possible that it's not syncing the directory time.

    So the thing is I do not know whether this may be relevant to OP. However, the theory is refuted by practice as can be seen in thread I linked. It wasn't just me. And not with empty folders either. Has something be done since January 2020 to address this? If so, then I may be wrong now (but wasn't then).

  7. If the software uses/relies on timestamps of folders as well then this might be the problem:

    Basically, with x2 duplication, a folder may have different date modified on the two disks and any software querying the pool will only get one of them.

  8. Another thing I have is that if I copy files from a client to the Server of the network, it matters whether I access the Server through Explorer -> Network or through a mapped network drive. The latter sometimes fails but I am pretty sure it has to do with some sort of permission (SQL Server backup files I can not copy through the mapped network drive) and I get a different message anyway. So, completely OT.

    So basically, I have no clue. I hope someone else here as an idea on how to diagnose and/or fix. I would have a look at Event Viewer on both the client as the server. Not optimistic but I'd look.

  9. So there is a Pool that consist of only 1 4TB drive? If so, then yes, you can shut down, power down, remove 4TB drive and connect it again when you are done.

    In fact, you could install DP on another machine and connect that 4TB drive and DP should recognize it as a Pool.

  10. Whether it makes sense or not, a higher pool does not inherit duplication settings from a lower Pool. That might make it harder. I even think a higher Pool is rather restricted in what it can read from a lower Pool, I.e. check the duplication status.  But we'll see what Covecube says.

  11. I fear this may be hard to address actually. Whenever the Top Pool (consisting of Pool A and Pool B) starts to rebalance, it would have to take into account:
    (a) The use-gap between Pools A and Pool B
    (b) Then, for any file that is a candidate for move (CFM), check what the Use Of Space (UOS) is, i.e., what the duplication status is (If, for instance, you use folder level duplication, that file may have 1 or a gazillion duplicates),
    (c) Then, for any such CFM, determine what the UOS would be after the move. Again, the relevant duplication may be rather arbitrary.

    The real issue here may be that when checking the UOS, the Top Pool would actually have to read file parameters somehow. Either it would (1) have to read on a look-through basis, so an x2 duplicated file is returned to the balancer process twice or, (2) for each file, interpret the duplication settings as per Pool A/B, including folder level duplication settings. However, I suspect that the balancing process is only able to read the relevant data for itself w.r.t. duplication settings and through the Pools A and B, meaning that querying a file would only return one result (just like you only get one results when you look at Pool A or B through Explorer). I suspect that this is how they end up at "other" currently: The Top Pool queries Pools A and B but receives only one record/data item/hit per duplicated file. It also receives total space. So the difference between total space and space used [by single instances of the file because that is all the Top Pool receives when querying Pool A or B] by definition, is other.

    I am sure it can all be done but it does not seem simple to me and it may have an impact on performance as building the list of files to move will require some additional intelligence.

     

  12. @zeroibis Whatever works for you. To me, it seems like a lot have administrative hassle for a remote, really remote, probability that, assuming you have backups, can be recovered from one way or the other.

    As I said, my setup never requires a 20TB write operation to recover 10TB of data. It basically resembles your 3rd example although I would have Drives 1 to 4 as Pool 1, drives 5 to 8 as Pool 2 and then have Pool 3 consist of Pools 1 and 2. But I agree that if two drives fail, you need no restore unless the two drives are one while 2-drive Pool. In my case, I would need to restore some data if the two failing drives are split between Pool 1 and Pool 2. I wonder whether a triple duplication with one duplicate through CloudDrive would be helpful but I don;t use Clouddrive and it may be expensive at those sizes.

    I am wondering however, what do you use for backup?

  13. First of all, DP does not require drive letters. Some here have over 40 drives in a Pool. For individual access, you can map the drives to a folder.

    On the backup/restore thing. Yes. However, my setup does not require backing up duplicated data. I have two x1 Pools and then one x2 Pool consisting of the two x2 Pools (it is called Hierarchical Pooling). Each x1 Pool has three HDDs. I only need to backup 1 set of three drives. Now with 4 (or 6 in my case) drives, the probability of losing two drives at the same time (or at least loss of the 2nd prior to recovery of the first failure) is really small. I also wonder what the loss of two drives in one of your Pools would do if ever your Pool is larger than 2 drives. I don't know what kind of backup solution you use but I could simply recover the whole shabang to my x2 Pool and tell the recovery not to overwrite existing files in that x2 Pool. DP would have recovered files where one duplicate still exists, the other files will again be present on both sub-Pools.

  14. I don't know the ins- and outs but some of this stands to reason. So you have Pool X (x2, consisting of X1 and X2), Pool Y (x2) and Pool Z (x1) consisting of Pool X and Y. I think the issue is that when DP finds it needs to move 100MB from X to Y, it will select 100MB, measured at x1, of files to move but it must then move both duplicates. DP will not be able to move just one duplicate from X to Y because then the duplication requirement will not be met on either X or Y.

    Why you get a substantial "Other" I am not sure but it may be because from the perspective of Pool Z, the files are in fact unduplicated. On X1 and X2 both you'll have a PoolPart.* folder. Files written there are part of Pool X. Within these folders, you should have another PoolPart.* folder. Files written here are part of Pool Z. My guess is that when you write a file to Pool Z, it will feed through to, say, the inner PoolPart.* folder on X1. Then DP will try to duplicate this file in Pool X (so write to X2) and I think this duplicate may be considered "Other" from the perspective of Pool Z. Not sure where that one ends up physically (inner or outer PoolPart.* folder) on X2 but it can not be part of Pool Z (with respect to measurement) because if it were, it would be a duplicated file. That would be in violation of the duplication setting of Pool Z.

    Generally, I think, it is better to have Pool X and Y both at x1 (so no) duplication and then tell Pool Z to have x2 duplication. One big advantage is that if you ever need to recover, you only need either the drives from Pool X or Pool Y. I am pretty sure that would avoid balancing to overshoot and measurement being unclear.

     

  15. No worries, it was new to me too (and I still don't know that much about it). So a cheap SAS card, like the IBM 1015 (and there are many that are called differently but are the same. I have a Dell Perc H310 SAS controller which, I am pretty sure, is the exact same card.).

    Anyway, the cheaper SAS-card have two ports indeed. However, those are SAS ports. There are cables that allow you to connect up to four SATA drives to just one SAS port. For example: https://www.amazon.com/Cable-Matters-Internal-SFF-8087-Breakout/dp/B012BPLYJC.  If you look at the picture you see one weird connector, that's a SFF-8087, which splits into 4 SATA. Take two of these and you get 8 SATA connectors.

    A few things to be aware of:

    1. You want these SAS cards to have a BIOS for IT or HBA mode, not RAID. It is something you can do youself if you get a 2nd hand card in RAID mode. I found a guy who flashed it for me before shipping.
    2. Whatever card you get, check whether it is indeed using a SFF-8087 connector. No worries if it does not, it just means you need another type of breakout cables.
    3. In my case, the breakout cables included power delivery. That was a pain because the cables became less flexible than I wanted. The picture above is more to my liking, just data cables. Power to the drives seperately
    4. Which of course means you need a way to power 8, 12 or 14 HDDs... There are splitters for this as well. I think you do want to share the load a bit on the various cables that come out of the PSU
    5. Finally, these SAS cards are typically meant for servers that have plenty of airflow. The chip can run hot and most likeley only has a heatsink. You can attach a, I think, 40mm fan to it using philips screws that attach between ribs of the heatsink.
  16. Hi Newbie! ;D,

    DP needs very little, I had it running on an Intel Celeron G530 and could stream 1080p to at least one device. So a cheap build with, say, a Ryzen 3 3200G, 8GB of RAM, a decent 350W PSU and W10 would work like a charm as a file/stream server. The things you'd probably look for are SATA connectors (cheap boards often have only 4). Although you could get a cheap SAS card (IBM 1015 or somesuch, used.) which would provide plenty of expandability. The other thing is the network connection. 1Gb Ethernet, I think, should be "enough for anybody".

    It is a bit of a bad time as CPUs are in high demand relative to production capacity. Was a time, which will come again, when you could have a satsfactory CPU for like US$60.

    Edit and PS: Your English is fine. Just use capitals to start and periods to end a sentence and it'll be great.

  17. My storage needs don't really grow so I am just sticking with what I have, anything between 1.7 and 5.5 yo (6 drives). I bought them spread out over time. Mostly Toshiba and HGST as I had a few issues with Seagate and WD years ago. Only 2 HDDs of same type and purchase date so those are in seperate Pools.

    As long as they don;t fail, I'll run them until they do.

  18. You can have two PoolPart.* folders on one drive at one folder level (e.g. root) but that is a symptom of something gone a bit wrong. Long story. Only one of them would be the real currently active PoolPart.* folder.

    A PoolPart.* folder can in fact contain yet another PoolPart.* folder in the case where you use Hierarchical Pools.

    But mostly, no, normally a drive has one PoolPart.* folder in the root.

    You're most welcome.

×
×
  • Create New...