Jump to content
Covecube Inc.
  • 0
celegorm

Questions before switching

Question

I'm currently using FlexRaid and I'm getting sick of constant false reports of corrupt files.  All my drives check out OK and files are all good but it's validation checks fail immediately after re-doing the parity.  Needles to say, I don't trust it any more if a drive dies to successfully restore my info.

 

Before I switch over I have a few questions and things I'd like confirmed if anyone here can help me.

 

First, and most importantly can I add or create drives to a pool that already have data on the drives?  I have 15TB of data to move so this is huge for me.  Assuming I can add to the pool without destroying my data, is there anything else I need to do?

 

Second, after the pool is created, can I later add additional drives to expand my pool or am I stuck with the number of drives I start with in the pool?

 

 

Next, if I need to pull a drive out of the pool for any reason, am I able to read the data off the drive?  If there's one thing FlexRaid has done well is I can pull a drive out of the pool, plug it into another PC and read all the data off of it.

 

Last, how does the backup functionality work here?  My current setup has 7 4TB drives that are pooled with an 8TB drive used to store parity information.  After formatting, this gives me between 25 and 26 TB or usable space in the pool as the parity drive obviously isn't counted.  I'd like to not lose any of that space due to parity if I can help it but at least want to be properly informed on the expectations.

 

Thanks in advance!  It takes a while for Flexraid to rebuild after pulling things out so I'm a bit hesitant to go for the free trial until I know what I'm getting into.

 

Share this post


Link to post
Share on other sites

16 answers to this question

Recommended Posts

  • 0

 

 

First, and most importantly can I add or create drives to a pool that already have data on the drives?

 

When you add a drive to a pool, all that happens to the drive is that a folder named "PoolPart-<IDString>" is added to the root of the drive. You can then cut and paste content from the disk (or not) into that folder. What you put into that folder appears in the overall merged pool. If two matching files on different disks collide, they will both be added and the second/third/etc copy will just be utilized as duplicate copies.

 

 

 

Next, if I need to pull a drive out of the pool for any reason, am I able to read the data off the drive?

 

Yes, everything is still a plain file. All your DrivePool data on each disk just reside inside of the PoolPart folder on the disk.

 

 

 

Last, how does the backup functionality work here?

 

You should back up the pool drive which presents the merged view of all the individual disks. You could back up each individual disk's contents, but that gets messy imo. You can use any file-based backup tool. VSS is not supported, though.

 

 

 

space due to parity

 

Parity isn't supported - only duplication levels. You can set duplication levels either globally across the entire pool, or at a folder level instead. So, you could choose to not duplicate temporary/unimportant data, and then set 2x on others, and 3-4x on critical document folders/etc.

 

A popular choice is for people to use DrivePool with no replication along with "snapraid" to implement parity. That's a good choice for pools where data is rarely/never deleted/changed, but it's a bad choice for setups where data is being deleted/reworked/etc frequently, since it's file-based RAID which isn't real-time and there's risk of data loss in the event of a rebuild if parity hasn't been recalculated following significant data changes/deletions (additions are fine).

 

I would like parity too, but I much prefer the simplicity and sense of assurance I get from not having to worry about how much I change my data around or how frequently I do it. There really isn't a product which exists today in which there is no compromise whatsoever to running parity - normally the compromise is performance or complexity + lack of flexibility. ZFS is the best parity option imo, but you don't have raw files on the disks in the event of a catastrophe, and the logistics of disk acquisition to do expansion can be tricky. Personally, I prefer being able to just throw a single drive in as I need space, being able to easily just remove a single disk, never having to worry about drive sizes matching, etc. For me, sacrificing space efficiency to get those things is worth it. I understand if other people have different priorities, though.

Share this post


Link to post
Share on other sites
  • 0

To clarify when I was asking about backup options I was referring to duplication vs parity vs RAID.  I knew RAID wasn't an option with this software but I was hoping for parity as it wouldn't require that I buy any new disks :)  I think half the reason I'm having so many issues with FlexRaid's parity is that Plex scans the media and does god-knows-what with the meta data fairly often.  From what little I've found so far, it sounds like this might be a similar issue if I do anything involving snapraid.  This leaves me with duplication or a real RAID system.

 

 

One thing I have yet to find in an FAQ is let's say I want to replace an existing 4TB drive with an 8TB a few years down the road.  how would I go about expanding the pool with to include the new "4" extra terabytes?

Share this post


Link to post
Share on other sites
  • 0

 

 

To clarify when I was asking about backup options I was referring to duplication vs parity vs RAID.  I knew RAID wasn't an option with this software but I was hoping for parity

 

Ah, gotcha. Backups are different than RAID. If you accidentally deleted everything with a stray command, or got hit with a ransomware virus, for instance, all your data would instantly be lost with RAID. RAID's purpose is only to keep systems up in the event of hardware failure. If a copy of your data is not sitting in another system, with versioning handled, then your data is supremely vulnerable.

 

Also, "parity" is just something people say for short when referring to one type of RAID - RAID5 or RAID6 levels (or, in ZFS notation, "RAIDZ" or "RAIDZ2"). If you use DrivePool with 1 file duplication across the entire pool, then it's basically functioning as a RAID1 at that point. If you used no duplication, it would be equivalent to a RAID0 array (striping). Of course, you can customize it further at the folder level, which isn't typical with RAID, so "duplication" is probably the best way to refer to what DP does.

 

 

 

One thing I have yet to find in an FAQ is let's say I want to replace an existing 4TB drive with an 8TB a few years down the road.  how would I go about expanding the pool with to include the new "4" extra terabytes?

 

You wouldn't need to do anything. The disk would show up in the DrivePool UI, you'd click "add" and you would immediately have that much more space available in the pool. DrivePool takes care of balancing out the distribution of files across the disks automatically on its own. Because it's all file-based, there's no fixed filesystem that needs to be adjusted.

Share this post


Link to post
Share on other sites
  • 0

I apologize if a lot of this is already covered, but ....

 

 

First, and most importantly can I add or create drives to a pool that already have data on the drives?  I have 15TB of data to move so this is huge for me.  Assuming I can add to the pool without destroying my data, is there anything else I need to do?
  
 
Thanks in advance!  It takes a while for Flexraid to rebuild after pulling things out so I'm a bit hesitant to go for the free trial until I know what I'm getting into.

 
Add existing data to the Pool (seeding):
http://wiki.covecube.com/StableBit_DrivePool_Q4142489
 
As from the seeding guide, you can see that it's very easy to read data from the pool on other systems.
 

Second, after the pool is created, can I later add additional drives to expand my pool or am I stuck with the number of drives I start with in the pool?

You can add or remove disks from the pool at any time. 
 

 

Next, if I need to pull a drive out of the pool for any reason, am I able to read the data off the drive?  If there's one thing FlexRaid has done well is I can pull a drive out of the pool, plug it into another PC and read all the data off of it.

 

If you take a look at the seeding guide, it's pretty clear. 

 

But yes.  Everything is in the 'PoolPart.xxxxx" folder on normal NTFS drives.

The only issue is that the data may be spread out between the disks. 
 

 

Last, how does the backup functionality work here?  My current setup has 7 4TB drives that are pooled with an 8TB drive used to store parity information.  After formatting, this gives me between 25 and 26 TB or usable space in the pool as the parity drive obviously isn't counted.  I'd like to not lose any of that space due to parity if I can help it but at least want to be properly informed on the expectations.

 

StableBit DrivePool doesn't have any backup functionality built in.

 

Compounded by the fact that it doesn't support VSS, either (but i suspect that Flex RAID doesn't either, IIRC).

 

That said, duplication (redundancy) is done with 1:1 file copies.  So if duplication is enabled, it's going to use twice the space.   1TB of data will use 2TB of capacity in this case.

 

 

That said, we do have a number of users using SnapRAID (free, IIRC) in conjunction with StableBit DrivePool to get (snapshot) parity protection of the pool 

 

http://community.covecube.com/index.php?/topic/52-faq-parity-and-duplication-and-drivepool/

Share this post


Link to post
Share on other sites
  • 0

I too came from Flexraid and opted for drivepool leveraging snapraid. 4 drives for pooling and 1 parity drive. For simplicity purposes I've installed Elucidate which is a gui for snapraid.

https://elucidate.codeplex.com/

 

Drivepool is awesome for my needs, you set it and forget it. It scales perfectly, require more space simply add another drive or two.

 

Sent from my A0001 using Tapatalk

Share this post


Link to post
Share on other sites
  • 0

I too came from Flexraid and opted for drivepool leveraging snapraid. 4 drives for pooling and 1 parity drive. For simplicity purposes I've installed Elucidate which is a gui for snapraid.

https://elucidate.codeplex.com/

 

Drivepool is awesome for my needs, you set it and forget it. It scales perfectly, require more space simply add another drive or two.

 

Sent from my A0001 using Tapatalk

 

I've seen this mentioned a few times (and not just by you).  A GUI is definitely helpful if you're not used to command line (or just prefer a shiny UI :) )

 

.... future plans to migrate "critical" data to the pool via duplication...

 

Sent from my A0001 using Tapatalk

 

Well, "migrating" is just a few clicks away. Enable duplicated, done. :)

 

And duplication + snapshot parity isn't a bad way to go!

 

Awesome, well thank you everyone!  I'm pretty impressed so far with just the level of feedback from people here.  I'm going to have to switch things over later today and give this a play.

 

Glad to hear it, and let us know how it goes! 

And if you need help, just ask.  (and you can contact us directly at https://stablebit.com/Contact if you have any issues)

Share this post


Link to post
Share on other sites
  • 0

So far so good!  I'm doing the parity calculation with SnapRAID now.    It's going slow but calculating that much parity data I wouldn't expect anything fast.  For the time being, I disabled the data balancing as I cared more about getting a parity created than balance.  but I might turn that back on later.  worst case a drive dies before the next snapshot and things are restored how they were before the balance.

 

My initial thoughts so far are "wow".  super simple to set up and integrates very nicely.  My only complaint is that now "My Computer" is now littered with extra drives and drive letters that I don't need since I'll be doing everything in the pool.  That said, if that's my only complaint then I'm still in a much better position than I was before.

 

At this point I'm fairly confident that I will be buying once my trial is up.  I was talking to one of my friends who went the "UnRAID" route and with some of the things i've told him, he's regretted not trying this first.

 

Thanks again to all of you who answered my noobish questions and linked to the articles I couldn't find.  The prompt feedback was greatly appreciated and the activity within this community was a huge factory in me trying out the software.

Share this post


Link to post
Share on other sites
  • 0

 

 

My only complaint is that now "My Computer" is now littered with extra drives and drive letters that I don't need since I'll be doing everything in the pool

 

There's no need to have the disks mounted at all for DrivePool's purposes, so you can just go to diskmgmt.msc and remove the letters as far as it's concerned.

 

For snapraid, though, maybe you could mount the disks as folders? http://wiki.covecube.com/StableBit_DrivePool_Q4822624

Share this post


Link to post
Share on other sites
  • 0

So far so good!  Things have been stable and working well.  I am very impressed.

 

My only remaining question at this stage is how the drive balancing feature benefits me? I had turned it off to make sure it didn't clash with SnapRaid while it did the initial parity build.  I understand that if a drive fails  I lose less data (temporarily) as it's all balanced across my drives but now with SnapRaid managing a backup of sorts and no duplication is there really a point in me turning that back on? 

Share this post


Link to post
Share on other sites
  • 0

Well, depends on the pool usage and such. But for the most part, the balancers don't do a lot, except for in very specific cases. 

 

That said, I'd say disable all the balancers but the StableBit Scanner balancer, and leave it enabled.

This way, it should only balance data if there is a problem with one of the pooled disks.

 

 

But even that's optional.

Share this post


Link to post
Share on other sites
  • 0

Thanks for the info!

 

Sadly the Stable Bit Scanner won't be much use on the SMART front.  Most of my drives are connected to a high-point RAID card which is notorious for not passing through SMART data.  That said, I'll at least turn it on for the 3 drives that are not on the controller.

Share this post


Link to post
Share on other sites
  • 0

Thanks for the info!

 

Sadly the Stable Bit Scanner won't be much use on the SMART front.  Most of my drives are connected to a high-point RAID card which is notorious for not passing through SMART data.  That said, I'll at least turn it on for the 3 drives that are not on the controller.

 

 

Well, as long as the drives are not in RAID arrays, the HighPoint management software is installed and running, and "unsafe Direct IO" is enabled in StableBit Scanner, it *should* get SMART data.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...