Jump to content
Covecube Inc.
  • 0

Turning off balancing will require better forethought?


Question

Hello,

 

I'm pretty sure I'll be buying DP soon. Will either use duplication for protection, or SnapRAID.

 

If I go with the former, and choose to use the plugin OrderedFilePlacement to disable balancing* and not make my HDDs spin all at once, then am I correct to assume that I will also need to think better about which HDDs I write to first.

 

Consider the following scenarios: (I know it's an extreme one in terms of HDD-size difference)

 

I currently have 1 x 8TB HDD and 2 x 2TB HDDs.

 

This will basically mean that if I want file-duplication with balancing off, I'll have to make sure that I fill the 2TB HDD first, before I start writing to the 8TB HDD? This is because, if I write to the 8TB first, and then this drive fails, its contents are too big for the 2 x 2TB HDDs to hold even with file duplication on. (12/2 = 6 TB, which is > 2 x TB HDDs).

 

Will I have to worry about stuff like that if I turn off balancing? I don't want to spin 2-3 HDDs just to play a movie file.

 

Note: I'm new to the concept of pooling and even newer to that of RAID so your patience is appreciated!

 

 

 

 

 

 

 

* That's what it does, right?

Link to post
Share on other sites

11 answers to this question

Recommended Posts

  • 0

Ok. Reading the manual one more time I think what I'm asking about is turning off read striping and not balancing. I'm still not sure what happens if I turn off balancing in the above scenario, so if you can answer that as well that'll be awesome.

 

As for read striping — should I keep it on? if I'm playing large media files, how likely is it that read striping will accelerate my HDDs conisderably?

Link to post
Share on other sites
  • 0

Playing media over an network? Probably 1GB Ethernet? That is about 125Mb/s max. The 8TB (Seagate Archive I assume) reads like crazy until the last, say, 2TB. IMHO< read striping is only sensible if you have heavy (read) I/O on the machine and streaming a video ain't that.

 

Also, I'd keep balancing as default. It will first fill up the 8TB HDD (most space free) and the other copies will be dividided over the 2 2TB HDDs (it needs to go there due to duplication and will select HDD with most space free).

 

I think, once you get to 4TB of data, DP will not allow you to write data to duplicated folders, sort of Disk Full, and at least report errors in duplication status if you force more data on the 8TB HDD. Christopher will tell.

Link to post
Share on other sites
  • 0

Playing media over an network? Probably 1GB Ethernet? That is about 125Mb/s max. The 8TB (Seagate Archive I assume) reads like crazy until the last, say, 2TB. IMHO< read striping is only sensible if you have heavy (read) I/O on the machine and streaming a video ain't that.

 

Also, I'd keep balancing as default. It will first fill up the 8TB HDD (most space free) and the other copies will be dividided over the 2 2TB HDDs (it needs to go there due to duplication and will select HDD with most space free).

 

I think, once you get to 4TB of data, DP will not allow you to write data to duplicated folders, sort of Disk Full, and at least report errors in duplication status if you force more data on the 8TB HDD. Christopher will tell.

 

Thanks for the input Umfriend. Yes, eventually I will be playing from a different machine over 1GB Ethernet. The 8TB was only a hypothetical. Right now I have 2 x 2TBs. When my new house is finished building, I'll add 2 x 4TBs.

 

 

 

The 8TB (Seagate Archive I assume) reads like crazy until the last, say, 2TB. IMHO< read striping is only sensible if you have heavy (read) I/O on the machine and streaming a video ain't that.

 

Yes, I'll keep balancing on, but should I turn  read-striping off as per your recommendation/exprience? What are the ramifications of read-striping in general?

 

Also, anything I should know if I plan to use this for a media server, anything that I should change out of the box? (except for setting up the drives as pool)

Link to post
Share on other sites
  • 0

Well, there is no issue at all turning read-striping off. However, it may be that one file will be read from one HDD and other from another. I do not actually know this but I suspect drives will spin up more than they would had drive pool been optimised to minimise spinning. At the en of the day, I simply do not think it will matter much either way.

Link to post
Share on other sites
  • 0

To clarify a few things:

 

By default, files are placed on the drive with the most available free space. That means that 8TB drive in your case.
However, if you enable duplication, it will place one copy on the 8TB drive (most free space) and then the duplicate on one of the 2TB drives (most likely). 

Once you have about 3.6 (aka 4) TBs of data, you'll run out of space on the 2x2TB drives.  At this point, for duplicated folders, you'll see "not enough free space" errors. For unduplicated data, you may see the same, but it may not. 

 

 

 

As for read striping, this feature can be a bit complicated and it only applies to duplicated data.

Specifically, if you have a disk that is on a much faster bus or a drive that has a lot of outstanding IO or is just performing much slower, the Read Striping feature will try to read from the other copy of the file.  Otherwise, it reads from both.

 

For the most part, there isn't a need to turn off read striping, as it does optimize performance. But if you want you disks to more aggressively idle, then it would be way to help do that (as it will only read from one disk/copy).

Link to post
Share on other sites
  • 0

To clarify a few things:

 

By default, files are placed on the drive with the most available free space. That means that 8TB drive in your case.

However, if you enable duplication, it will place one copy on the 8TB drive (most free space) and then the duplicate on one of the 2TB drives (most likely). 

Once you have about 3.6 (aka 4) TBs of data, you'll run out of space on the 2x2TB drives.  At this point, for duplicated folders, you'll see "not enough free space" errors. For unduplicated data, you may see the same, but it may not. 

 

 

 

As for read striping, this feature can be a bit complicated and it only applies to duplicated data.

Specifically, if you have a disk that is on a much faster bus or a drive that has a lot of outstanding IO or is just performing much slower, the Read Striping feature will try to read from the other copy of the file.  Otherwise, it reads from both.

 

For the most part, there isn't a need to turn off read striping, as it does optimize performance. But if you want you disks to more aggressively idle, then it would be way to help do that (as it will only read from one disk/copy).

 

 

Thank you, so I have one more related and another unrelated question:

 

 

 

Related: This nature of duplication means I should rather purchase 2 x 4TBs instead of 1 x 8TB? The way I understand it now, is that 2 x 4TBs vs 1 x 8TB will allow me to utilize the maximum 6TB instead of the 4TB in the 1 x 8TB scenario?

More generally and going forward, how can I make logical decisions regarding my purchase if I want x2 duplication and to utilize the maximum storage available?

 

Unrealted: Let's say I have two HDDs drives that contain the exact same files (using Microsoft's Syntoy). Is there any way to connect them both to the pool without it having to write from one of them to the other? (I'm assuming from the first one added to the second)

Link to post
Share on other sites
  • 0

If you want duplication initially, then yes, the 2x 4TB drives would be what you want to get.  
However, that depends on the amount of data. You can get a 8TB drive and a 2TB drive if you have less than 2TBs of data, and then swap out the 2TB drive for another 8TB drive (or the like).  As for what you could use the 2TB drive for? A dedicated system backup drive wouldn't be a bad idea.

 

But yes, for now, 2x4TB is the better option. 

 

 

As for the contents, you should be able to "seed" the pool, in this case.

http://wiki.covecube.com/StableBit_DrivePool_Q4142489

When you remeasure the pool (or reset the settings, as that will trigger a remeasure), it will check the "duplicated" files (files that exist in the same location on both disks). If the data modified info is not the same, it will checksum the data to ensure it's identical. 

Link to post
Share on other sites
  • 0

Thanks Christopher!

 

 

 

As for what you could use the 2TB drive for? A dedicated system backup drive wouldn't be a bad idea.

 

Which reminds me, how do people do system backups? Cloning? imaging? what system do *you* use? I can imagine losing my system preferences and software configuration being a much bigger headache than my easily-replaceable media library.

Link to post
Share on other sites
  • 0

Thanks Christopher!

 

 

Which reminds me, how do people do system backups? Cloning? imaging? what system do *you* use? I can imagine losing my system preferences and software configuration being a much bigger headache than my easily-replaceable media library.

I use Windows Server 2012R2 Essentials.  So I use the built in Windows Server Backup to backup just the system drive.  I've used it ... more than a few times to restore (even to move to a new drive).  And it's an incremental, full image backup. 

Link to post
Share on other sites
  • 0

Thanks for all the info. One last question:

 

Currently I have one folder on my HDD which is used for downloads. These downloads are "processed" once they are completed, which means they are moved to another folder on the same drive. These can be many small-medium files, or one or two large files being downloaded/processed at the same time. What would be the best practice for this folder when using DrivePool? I saw in your (Christopher) setup that you use a temp HDD designated for this especially, but seeing as I don't have nearly as much storage as you and I'm starting out modestly, I'm wondering what are my options here.

 

 

Thanks.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...