Feature request - Parity support ?

[vnguyenquangdo]

Good morning,

 

I have been testing Drive Pool over the last 24h, and this is indeed an interesting product. However I'm wondering if it would be possible to add 'parity type' security at the folder level

 

Kind of RAID5, but at the folder level. This would have the obvious benefit of not requiring twice the size of the data to be protected. It would obviously require three HDD (or more), the lowest size being able to cover the parity data required.

 

Example:

- 2 x 4TB HDD, with 1TB of 'duplicated folders'

- 1TB HDD

 

With only folder duplication, this would require 1TB of extra space

With parity, the parity and data information would be spread across three HDDs, resulting into an actual data usage of 1.5TB

 

Adding a fourth HDD would even further reduce the footprint to 1.25TB

 

Is this something currently investigated ?

 

Regards

 

Vincent

[Shane]

Hi Vincent, you've prompted me to write a FAQ on this topic.

 

http://community.covecube.com/index.php?/topic/52-faq-parity-and-duplication-and-drivepool/

 

The TLDR for your particular question is: the parity block has to be as big as the biggest data block you're protecting, so the moment you increase the size of your biggest parity-protected folder you're going to need a bigger parity block (or mathematical equivalent). A disk doesn't change in size; folders can vary dramatically. So the problems to overcome are more than what you have with a "simple" drive-level parity system, even without having to ensure the parity, duplication and balancing subsystems cooperate.

[weezywee]

Adding parity support would defeat the purpose of of DrivePool. This product is meant to be simple and easy, just pop drives in for more storage and remove them if they are damaged. You can take them out of the pool and put them in any machine that supports NTFS, you wouldn't be able to do this if there were parity data.

 

I know this isn't what you want to hear, but if you want parity, use raid 5 or 6.

[Shane]

Actually, parity support doesn't mandate that the user's content be unreadable on other machines; that depends on the implementation used.

 

The thing is that while it's possible to add even folder-level parity to JBOD-style pools in such a way as to maintain content readability on non-pool machines and still keep things simple for the end-user? Implementing it is way past non-trivial and I've no idea whether the performance costs would make it impractical. Sort of like hot fusion.

 

(but if any angel investors are out there with a loose seven figures to punt Covecube's way, maybe Alex would be happy to work on it for a few years)

[Alex]

With parity, the parity and data information would be spread across three HDDs, resulting into an actual data usage of 1.5TB

 

Adding a fourth HDD would even further reduce the footprint to 1.25TB

 

Is this something currently investigated ?

 

Regards

 

Vincent

 

Parity has some advantages and disadvantages.

 

The short answer is that it doesn't make sense to put parity into the current version of DrivePool. We would need a new product that is a block based pooling solution. This block based DrivePool would have some advantages and disadvantages over the current file based DrivePool.

 

The main disadvantage would be that your files would no longer be stored as standard NTFS files on the physical disk.

 

Overall I feel that there are a lot of other well established block based pooling solutions out there and I'm not sure that we would bring anything unique to the table. So I'm not in favor of doing a "me too" product.

 

In general I'd like to create products that are unique in some way.

[vnguyenquangdo]

Thanks for the feedbacks

 

To Shane, many thanks for the link, interesting read indeed.

 

I understand fully your position, and I also understand the added complexity. That would have been a nice to have, but I agree this use case has advantages and disadvantages. Not being able to read the HDDs on a standard PC would be one of these major disadvantages.

 

I'm still comparing FlexRaid and DrivePool for my usage. For now I'm tempted to use both :-). Although I must admit DrivePool is way easier to use (it did take a little bit of reading to understand FlexRaid mechanisms)

 

Which actually does raise a point: is there any known issues to use FlexRaid and DrivePool in combination (windows server 2012 essentials) ?

 

Regards

 

Vincent

[Christopher (Drashna)]

Vincent,

 

I've see a good number of people using FlexRAID with DrivePool without issue. 

http://forum.covecube.com/discussion/720/dp-with-flexraid-raidf-comments#Item_30

 

 

Regards