Jump to content
Covecube Inc.
  • Announcements

    • Christopher (Drashna)

      Login issues   11/07/17

      If you have issues with logging in, make sure you use your display name and not the "username" or email.  Or head here for more info.   http://community.covecube.com/index.php?/topic/3252-login-issues/  
    • Christopher (Drashna)

      Getting Help   11/07/17

      If you're experiencing problems with the software, the best way to get ahold of us is to head to https://stablebit.com/Contact, especially if this is a licensing issue.    Issues submitted there are checked first, and handled more aggressively. So, especially if the problem is urgent, please head over there first. 
  • 0
MetalGeek

Replacing my Synology NAS

Question

Greetings,

I've been working out how I'm going to replace my aging synology DS413j NAS.  Crahsplan is no longer supported do to Java version limitations and it's not very fast, at all.  My research is pointing me to DrivePool + SnapRaid/disParity.  So here is what I want the rig to do and what I'm working with:

 

Functions

  • Media Storage
  • Standared File Server
  • Native Crashplan support
  • Media Server (Plex most likely)
  • iSCSI server for two workstations (Steam game Liibraries) using StarWind iscsi target
  • Backup destination for Macrium reflect disc images
  • Supports BOINC
  • Supports Encryption, preferably Bitlocker

Hardware (existing)

  • Xeon E3 1245 v3
  • Gigabyte Z87MX-D3H-CF Motherboard
  • 16Gb G.Skill PC3-1600 ddr3
  • Samsung 120gb SSD for Boot Drive
  • Nanoxia Deep Silence 1 Case
  • Antec HCG-620M power supply
  • 4 x 2TB WD RED (In the Synology)
  • Windows 10
  • Nvidia GTX960 2GB Video card

The hardware is my current desktop.  I used to work from home and run 4-5 VMs using Vmware workstation but that use case is not in play any longer.  I just game with the rig now.  So i'm going to build a mATX rig for gaming, move my current Toshiba 256Gb boot SSD and video cared into it and build my server around the current workstation.
 

I want this to a set it and forget it build.  What I'm not sure about is how to setup the storage.  Currently I have 5.35TB of storage with 1.07 TB free.  This includes all backups + media and files. Here is what I'm thinking, all drives encrypted with bitlocker:

  • DrivePool 1: 4x2tb WD Reds - Data and Media with duplication OR Parity for safe keeping, maybe both.
  • Drive Pool 2: single 3 or 4 TB drive for Backups. No duplication/parity
  • 1 3 or 4TB drive for iSCSI drives with parity (if supported).
  • Parity Drive if needed

I'm not planning an replacing any working hardware in my workstaion due to fiancial limititations. 
I'll need to add a 4-8port JBOD HBA as the Intel controller only has 6 sata slots and maybe a second NIC for the iSCSI target if performance warrents.
Thoughts?

Share this post


Link to post
Share on other sites

15 answers to this question

Recommended Posts

  • 0

I'm very sorry for the delayed response here.... It looks like the "view new content" is not ... well, getting all new content, so I missed this.

 

 

 

As for setup, aside from maybe dropping parity and just using duplication.... it looks good, mostly. 

We have a number of threads with guides (well, users talking about) how to use FlexRAID/SnapRAID with StableBit DrivePool.  You should definitely check them out, if you haven't already.

http://community.covecube.com/index.php?/topic/52-faq-parity-and-duplication-and-drivepool/

 

 

 

And StableBit DrivePool should see and utilize iSCSI drives without any issues.  I can't speak of the other products you've mentioned though. 

 

As for HBA's, I'd recommend LSI based cards. If you don't mind flashing cards, the IBM ServeRAID M1015's are relatively cheap, and work very well for HBA cards. And can support expanders, getting you the ability to attach a lot of drives). 

 

 

 

As for BitLocker, StableBit DrivePool absolutely supports it (the beta version enables the functionality by default, otherwise, you'll need to turn it on for proper functionality).  Though, we do recommend enabling automatic unlocking.

Share this post


Link to post
Share on other sites
  • 0

Thanks for the reply. Been playing with things with some old drives I have.  Good way for me to determine size needed for data, etc.  I'll need less storage than I anticipated for all my data which probably elimiantes the need for an addon HBA as well as parity but if I need one i'll go the IBM route.  I have no issues flashing the card.  Thanks for the tip, I've seen that other places as well. 

Right now my biggest issue is Windows 10 stability.  I know the stable version doesn't support it and the beta, well, it's beta so it's been hard to say "Yeah this will work".  The stable version causes explorer to hang on boot about 70% of the time so I cannot start programs, shutdown, restart. I have mouse and keyboard but everything is just locked.  I have to hard reset.  I'm trying to model my eventual setup with Emby server, Crashplan, etc but so far I cannot get Stablebit to a place I'm happy with so I'm stalled at the first step. storage.    I just installed the latest beta and i'll report errors and see if I can at least move forward with my testing.

Share this post


Link to post
Share on other sites
  • 0

Well, I definitely understand not wanting to use a beta product. Not everyone is willing to do so.

 

 

Right now my biggest issue is Windows 10 stability.  I know the stable version doesn't support it and the beta, well, it's beta so it's been hard to say "Yeah this will work".  The stable version causes explorer to hang on boot about 70% of the time so I cannot start programs, shutdown, restart. I have mouse and keyboard but everything is just locked.  I have to hard reset.  I'm trying to model my eventual setup with Emby server, Crashplan, etc but so far I cannot get Stablebit to a place I'm happy with so I'm stalled at the first step. storage.    I just installed the latest beta and i'll report errors and see if I can at least move forward with my testing.

 

That is with StableBit DrivePool? If so, that's definitely unusual. 

If that continues with the beta build, could you get a crash dump of the system when that occurs?

http://wiki.covecube.com/StableBit_DrivePool_System_Freeze

Share this post


Link to post
Share on other sites
  • 0

Yes this is with DrivePool.  So far explorer has not hung on me but I've had 1 app crash, 1 service crash and a BSOD since installing the Beta of DrivePool.  I uploaded the logs already after the crash.  I've placed all of DrivePool into my AV (BitDefender) exclusions including the running process to elimiante any colisions there.  My system runs rock solid until I introduce DrivePool and then.. not so much.  If the Beta doesn't play nice I'll go back to the released version and work on the crash dump.

Share this post


Link to post
Share on other sites
  • 0

I also use Bitdefender and made zero change in it regarding DrivePool.

I even have DrivePool with "Bypass file system filters" disabled and have no issue so far.

It must be something else in your config.

Share this post


Link to post
Share on other sites
  • 0

With the exception of 1 bad day, the day I installed the beta, it has been runnign without a hitch.  Much more stable than the current production release, for me.  I've been trying to break things by moving large amounts of dat on/off the pool as well as placing all of my local email storage there is I have active file activity.  So far, no issues.  Next will be start bitlocker on the pool drives.  I have it on all non pooled drives that are unlocked at bootup via PW.  The rest will be the same.  

Share this post


Link to post
Share on other sites
  • 0

With the exception of 1 bad day, the day I installed the beta, it has been running without a hitch.  Much more stable than the current production release, for me.  I've been trying to break things by moving large amounts of data on/off the pool as well as placing all of my local email storage there is I have active file activity.  So far, no issues.  Next will be start bitlocker on the pool drives.  I have it on all non pooled drives that are unlocked at bootup via PW.  The rest will be the same.  

Considering the huge list of bug fixes between the Release version (2.1.1.561) and the Public Beta (2.2.0.651), this doesn't surprise me. 

 

I know there was some locking and performance fixes in there as well.

 

 

So if the beta is working much better, I'm very glad to hear that.

If you're still having issues or have any other issues, do let us know (and what OS you're using specifically) 

Share this post


Link to post
Share on other sites
  • 0

​​​​​

 

 

Functions

  • ​Media Storage
  • Standard File Server
  • Native Crashplan support
  • Media Server (Plex most likely) Emby
  • iSCSI server for two one workstations (Steam game Liibraries) using StarWind iscsi target
  • Backup destination for Macrium reflect disc images
  • Supports BOINC
  • Supports Encryption, preferably Bitlocker

Hardware (existing)

  • Xeon E3 1245 v3
  • Gigabyte Z87MX-D3H-CF Motherboard
  • 16Gb G.Skill PC3-1600 ddr3
  • Samsung 120gb SSD for Boot Drive
  • Nanoxia Deep Silence 1 Case
  • Antec HCG-620M power supply
  • 4 x 2TB WD RED (In the Synology)
  • Windows 10
  • Nvidia GTX960 2GB Video card

 

​So this project was put into the Freezer for quit some time but I'm reviving it.   My first dilemma:  Build a top heavy system using Windows 10 Pro or Virtualize the stack and split out functions? Both introduce different sets of complexities and in the end I want set it and forget it.  I check backups and patching, the rest runs itself.  I have this today with my Nas + desktop + Firewall but they eat lots of real estate in my office. No basement to hide stuff in and it's far too hot in Northern CA to use a closet without having to to air flow modifications that are very wife unfriendly.   I'm leaning heavily towards Hyper-V but this is my first whatck with Hyper-V so I may be missing he boat on something.

​​

Windows 10 Pro Server
​This smells like recipe for instability at some point and it leaves my firewall appliance on my desk still.

  • ​Stablebit + Drive Scanner
  • ​Snapraid for Disc Rot protection
  • ​Starwind iSCSI
  • ​Crashplan
  • ​Marcrium Reflect
  • ​Emby
  • ​BOINC
  • ​General Desktop usage for Email, WWW, Music ripping (Im old scool, by CD and rip to flac)

​Moving onto Hyper-V, which looks to be a "better" solution if I deploy it properly.  My main question is whether to pass all the storage discs directly through or just fill each with a single VHDX file and assign that to the Guest OS?  I could care less really, as long as one isn't  a setup for failure or a technological cul-de-sac.  My readings indicate I should just create the VHDX files and be done with it.
Am not married to Hyper-V.  I can use KVM or ESXi as well.  Feel free to make eidts, recommendations, etc.

Hyper-V Host Server

  • ​Hyper-V Server 2012 R2
  • Xeon E3 1245 v3 processor
  • Gigabyte Z87MX-D3H-CF Motherboard
  • 16Gb G.Skill PC3-1600 ddr3
  • ​OCZ Trion 150 240GB SSD for Boot Drive running off the Integrated SATA
  • Toshiba 256GB SSD for VM boot VHDX files running off the Integrated SATA
  • ​Nanoxia Deep Silence 1 Case
  • Antec HCG-620M power supply
  • ​LSI SAS9211-8I 8PORT Controller with 2 3WARE Cable Multi-lane Internal Cables (SFF-8087)
  • 5 x 2TB WD RED- running off the LSI controller
  • ​2 x 1TB drives running off the LSI controller
  • ​1 Seagate 5TB UDB Drive
  • ​HP (Intel) dual port Server NIC

Pfsense Guest

  • HP Dual NIC will be dedicated to this VM​
  • 1GB Ram
  • 2 cores

​File Server Guest

  • ​Windows 10 Pro or Server 2012 R2
  • Stablebit Drivepool
  • StawrWind Iscsi
  • snapraid
  • 4GB+ RAM
  • 2 cores
  • ​5x2TB Drives via VHDX
  • ​2x1TB Drives via VHDX
  • ​USB Drive via pass though - this will be for snapraid parity files.

Emby/BOINC Server

  • ​Windows 10 or Linux - don't care really.
  • 2+ cores
  • 4GB RAM
  • Macrium reflect for OS Backup of Windows
  • ​limited trans-coding.  1 Roku at a time

 

 

Share this post


Link to post
Share on other sites
  • 0

Well, you know that Windows 10 Pro includes HyperV as well, right? :)

 

As for passing the disks through .... StableBit Scanner won't get SMART data in most cases.  It *may* with ESXi, if the controller is passed through, IIRC.  But not with HyperV at all, and I'm not sure about KVM.  But SMART may not be necessary on the guest if you're using HyperV.  (you can use it for SMART data on the host, and another copy (or copies) on the guest(s), for surface and file system scans. 

 

Running VMs off a SSD is a fantastic idea (I do this, and ... run ~20 VMs simultaneously without any issues).  If you have access to Server Standard/Datacenter, Data Deduplication is fantastic (900GBs of VMs using ~300GBs)

 

 

The only thing is that I would recommend running your router OS on bare metal.  I've seen some reports of misbehavior while virtualizing.  Also, if you need to do maintanence on the host, you essentially take your entire network down. 

Also, check out Sophos UTM (not their XG Firewall crap), or Untangle.  They're both very "turn key" solutions, but may work better for you.  (Also, opnSense may be a better alternative to pfSense). 

 

As for Emby, if you're doing transcoding, the more cores you can throw at it, the better (I'd honestly recommend 8, if you can)

Share this post


Link to post
Share on other sites
  • 0

 

Well, you know that Windows 10 Pro includes HyperV as well, right?

 

I do but I'm concerned it's not 24x7 ready.   I'll ponder this a bit more.  

 

 

The only thing is that I would recommend running your router OS on bare metal.  I've seen some reports of misbehavior while virtualizing.  Also, if you need to do maintanence on the host, you essentially take your entire network down. 

Also, check out Sophos UTM (not their XG Firewall crap), or Untangle.  They're both very "turn key" solutions, but may work better for you.  (Also, opnSense may be a better alternative to pfSense).

Thanks for the alternatives.  I've used both Sophos and Untangle although it's been awhile.  I was also an Untangle partner about 5-6 years ago. I prefered Kerio Control to both of them for SOHO/SMB applications.  For my use case, pfsense is a much better fit. Lean, easy to setup with simple OpenVPN client setup for my PIA account.  I have little use for all of the UTM features in a home environment.  I did see Untangle finally added OpenVPN but I still prefer pfsense.  Been using it it in my vm labs and my home network for 3 years.  Good point on staying with bare metal.  More pondering.

 

So aside from the SMART issue, are there any other reasons to pass thru vs. creating full disk VMDK's? 

Share this post


Link to post
Share on other sites
  • 0

It's 24/7 ready.   I have 4 Windows 10 systems that run 24/7.  one is a media server, and two are HTPCs.  All powered by Emby (the server only runs emby, no storage on it, as it connects to the storage server).   The 4th runs BlueIris and records to disk. 

 

The systems reboot maybe once a week, for updates.  Aside from that, they run very well. 

 

 

Thanks for the alternatives.  I've used both Sophos and Untangle although it's been awhile.  I was also an Untangle partner about 5-6 years ago. I prefered Kerio Control to both of them for SOHO/SMB applications.  For my use case, pfsense is a much better fit. Lean, easy to setup with simple OpenVPN client setup for my PIA account.  I have little use for all of the UTM features in a home environment.  I did see Untangle finally added OpenVPN but I still prefer pfsense.  Been using it it in my vm labs and my home network for 3 years.  Good point on staying with bare metal.  More pondering.

 

Ah, okay. :) 

 

I'm personally invested in Sophos UTM, and absolutely love it (using a Home license though). 

 

I don't use OpenVPN because Windows Server Essentials has a L2TP and SSTP VPN built into the Remote Access stuff. 

 

 

 

So aside from the SMART issue, are there any other reasons to pass thru vs. creating full disk VMDK's? 

 

Performance and portability, maybe. 

 

Honestly, I'm not too clear.  I keep my storage server separate from my VM stuff, and access everything over windows file shares. So I haven't looked into it too much. 

 

Though, if you're using HyperV, and are able to get Windows Server at a low price, that may be worth it.  Deduplication seriously does get you 60-70% savings on VHDs. But the OS is rather expensive.

Share this post


Link to post
Share on other sites
  • 0

I'v decided to take the KISS principle with this migration. Focus on the desired results and not solve prolbems that don't yet exist.  To accomplish that I'm going to start with my Win10 Pro box, strip out everything except teh "server" and feature software:

  • Stablebit + Drive Scanner
  • ​Snapraid for Disc Rot protection
  • ​​Crashplan
  • ​Marcrium Reflect
  • ​Emby
  • ​BOINC/Virtualbox (Project Atlas uses tiny headless VMs)
  • Bitdefender AV

As long as the box is stable 24x7, I'll leave it as is.  If something becomes unstable, I'll address it as they come.  I can always split out something to a VM later on.   I've migrated all desktop activites do a laptop so checking email, research, CD ripping all happens there now.   I've go my rial license up so I can get things going and when payday hits next week I'll register and go from there.  Now I need to RTFM to sort out my Pools, folder duplication and SnapRAID considerations.

 

Thanks for the nput Chris, it helps me sort out ideas by chatting with someone.  I engineer by talking aloud. 

Share this post


Link to post
Share on other sites
  • 0

The "KISS" principle is a very good idea. :) 
 
I'm "quoting" to break this up in an easier to read way.  Not actually quoting. 
 

As for the disk rot, that's ... a controversial subject.  I cover it a lot of that in the blog post I did a while ago here:
http://blog.covecube.com/2014/10/why-using-stablebit-scanner-is-a-good-idea/
 
But the TL;DR: of it is: In my opinion, bit rot doesn't happen. And modern disks are VERY, VERY good at making sure it doesn't. A simple read pass of the disk periodically is enough to ensure that doesn't happen, to correct it *if* (not when) it does, or at least let you know. 
However, SnapRAID *is* a good integrity check, and a way to recover data from a failed disk.

 
 
 

Though. If you're using Windows 10 Pro, it does include HyperV.  This is a type 1 (bare metal) hypervisor, as opposed to virtualbox's type 2 (software).  You'll get better performance out of HyperV, with less of a hit on the host.



 

We have some experience with BitDefender (namely with StableBit CloudDrive), but it does some really weird things. And by weird, I mean, from the looks of it, "badly programmed" is what I mean. 

You can see what I mean here. 
https://stablebit.com/Admin/IssueAnalysis/15661
But what you don't see is the resolution of the issue, and WHY it is occurring:

  • It turns out trufos is attempting to open: "\Driver\cloudfs_dis" (yes, notice the last k is missing).
  • That's because when they are constructing this path, they use the length of the real driver object (\Driver\CloudFSDisk), which is 1 character shorter, but when copying the data into the path they use the name of the driver file!
The problem is, that any antivirus solution will add additional overhead and problems.   Installing them is another layer of complexity and another vector for attack.    But not running one can be even more dangerous. 

What is my recommendation? 
Run a UTM.  Be it pfSense with intrusion detection, and web filtering.  Or Sophos UTM (not XG Firewall).  Or Untangle.  Or something better than a "dumb router".  This may serve you better than antivirus on your system.  

But having antivirus installed may be necessary as well.  And everyone has their own experiences and recommendations....

 
ddd

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×