Jump to content
  • 0

Volume does not show up in DrivePool


zeroibis

Question

16 answers to this question

Recommended Posts

  • 0

I also had trouble with a volume showing up in Drivepool, never was able to resolve it.  It appears that a few of my drives randomly and infrequently add or remove themselves to the list of available Non-Pooled drives.

 

I tried a few things with Christopher's help, to no avail.  You may want to step through some of them and see if they help you resolve why it isn't appearing there.  If nothing seems to, Christopher may ask for you to submit a ticket.

Link to comment
Share on other sites

  • 0

Looking around it appears the issue is that creating raid 0/1 arrays from disk management is not supported in combination with Drivepool and that you are supposed to be using Storage Spaces instead for that. I will be attempting to create the RAID 0 in Storage Spaces instead.

Link to comment
Share on other sites

  • 0

Ok solution is that you need to manually create the virtual drive in powershell after making the pool:

1)      Create a storage pool in the GUI but hit cancel when it asks to create a storage space

2)      Rename the pool to something to identify this raid set.

3)      Run the following command in PowerShell (run with admin power) editing as needed:
New-VirtualDisk -FriendlyName VirtualDriveName -StoragePoolFriendlyName NameOfPoolToUse -NumberOfColumns 2 -ResiliencySettingName simple -UseMaximumSize

Link to comment
Share on other sites

  • 0

Odd.  I steer clear of storage spaces whenever possible.  And I've used soft RAID 0 volumes (created in Windows Disk Management) in Drivepool *many* times for testing, so the assumption that Disk Management isn't a valid tool for RAID with Drivepool is incorrect.  I highly doubt storage spaces has anything to do with it showing up.

I've never seen Christopher or Alex comment about needing a PowerShell command to get a soft/hard RAID volume to show up.  But as I suspect @Christopher (Drashna) will be reading this topic, we'll hear what he has to say about your solution.  The issues that I linked to in my original reply were about normal (non-RAID) volumes and their odd behavior in the Non-pooled list.

 

Edited by Jaga
Update to fix incorrect info
Link to comment
Share on other sites

  • 0
34 minutes ago, Jaga said:

Odd.  I steer clear of storage spaces whenever possible.  And I've used soft RAID 0 volumes (created in Windows Disk Management) in Drivepool *many* times for testing, so the assumption that Disk Management isn't a valid tool for RAID with Drivepool is incorrect.  I highly doubt storage spaces has anything to do with it showing up.

I've never seen Christopher or Alex comment about needing a PowerShell command to get a soft/hard RAID volume to show up.  But as I suspect @Christopher (Drashna) will be reading this topic, we'll hear what he has to say about your solution.  The issues that I linked to in my original reply were about normal (non-RAID) volumes and their odd behavior in the Non-pooled list.

 

Interesting that RAID 0 in Disk Management worked for you as I found here it explicitly is not supposed to work:

" Dynamic Disks. Because of the added complexity of Dynamic disk, we don't support them being added to a pool.  Specifically, we take into account the physical disk when determining where duplicates reside, and that adds a lot more complexity and overhead when you start dealing with Dynamic Disks due to the complex arrays that you can create with them."

When you create a RAID 0 in Disk Management the disks become Dynamic and should not work. This is why I switched to using Storage Spaces. However if you create a normal pool in the Storage Spaces GUI it is not RAID 0 but instead just a pool of disks as the default value for -NumberOfColumns is 1 when using simple in the GUI. Thus the only way to use Storage Spaces for RAID 0 performance is to use the CLI so you can enter the correct settings in manually.

Problem with using Storage Spaces is then the underline drives can not be accessed by Stablebit Scanner as referenced here:

You can also see it listed as an official limitation:

SMART Data from Storage Spaces array. Because of the low level implementation of Storage Spaces, the disks in the array are hidden from access normally.  This makes getting information about the disk (including SMART data) significantly more difficult. 
We are looking into implementing this feature in the future, but there is no ETA.

Now I will state that HWInfo is still able to see the disks when they are being used in Storage Spaces so hopefully it is not to hard to get working.

It appears that both of these issues came up in the past but there was no Dev time to spare due to work being done on CloudDrive but now that it has released hopefully we will see some progress made on both of these issues.

Link to comment
Share on other sites

  • 0

Further testing on my end with a single Basic disk being converted to Dynamic showed that DP recognizes the Simple disk, but doesn't recognize the Dynamic one - so you are right.  I must have been thinking of spanned testing with another utility.  That's one of the inherent dangers of doing so many architecture/metrics tests, though I do love to do them.

I'll tinker a bit with Storage spaces if I can find some spare physical drives around here to use.

Link to comment
Share on other sites

  • 0

So I did some testing with Storage Spaces in Windows 10 (Build 17134.228), with two connected separate physical drives (4TB WD Reds in external WD USB 3.0 enclosures).  The Storage Space I setup was to stripe the drives (no resiliency) to best imitate a RAID 0 architecture.  The results are interesting:  Stablebit Drivepool can see and add the Storage Spaces drive (S:) to a pool without any problems.  Stablebit Scanner also sees the Storage Spaces drive as a single large virtual disk (no underlying drives are displayed, nor is any SMART data).

sKZq1Zu.png

 

g8vqp5F.png

 

RzLUIGG.png

 

gUpX5Ef.png

 

Note:  I didn't specify NumberOfColumns or Interleave when setting up the no Resiliency Storage Space.

I then did a performance test with CrystalDiskMark against the Storage Space volume (not the Pool volume), and ended up with speeds that I'd consider normal for a single WD 4TB Red drive:

ddgf0Rz.png

 

After this, I did the same performance test against the Pool drive (E:) to see what if any impact Drivepool has when using Storage Spaces:

cTB9byK.png

 

Performance appears to be unaffected, and is normal for the drives per the manufacturer specs (150MB/s read/write sequential). 

Link to comment
Share on other sites

  • 0

The issue I had with Storage Spaces was not that it did not show up, it was that it did not achieve RAID 0 performance by default. I tested my speed with disk management raid 0 and got for example 368MB/s Write speeds. When I setup a Storage Space just like you I ended up with the performance of only a singe drive 18XMB/s. So I created it again using the CLI method so that I could define the Columns as 2. When I retested after this I once again had raid 0 like performance and was achieving 368MB/s writes.

Link to comment
Share on other sites

  • 0

Ah right - only the Disk Management stripe didn't show up for you.  Let me change the config on my current storage space and see how that affects the performance.  Not sure how the USB connection will do with performance however.

Edit:  Changed the columns setting to 2, and re-created the volume.  The performance went up for sequential reads by 83%, and sequential writes by 69%.  The bad news is that anything non-sequential lost performance significantly.

Zgj9Oci.png

I'm not sure personally if the ~75% gain in sequential speeds (offset by losses for random access) are worth double the chance of volume failure, but that's entirely up to you.  Seems like you found the right answer for interleaving with Storage Spaces and Drivepool after all.  And I may have to take another look at why I don't use Storage Spaces, since it's possible to present a volume from it to Drivepool now.  This is why I love metrics and architecture testing - you find out something new all the time.

Link to comment
Share on other sites

  • 0
On 10/8/2018 at 3:38 PM, zeroibis said:

I have a RAID 0 volume (E) but it does not show up in DrivePool as a possible drive.

 Why is it not showing up and how can I fix this? Is there another software RAID 0 solution I can use that DrivePool will like?

StableBit DrivePool does not support Dynamic disks. Period.  

There are a number of reasons for why, but the main one is that we are unable to (efficiently/quickly) determine what physical disk a file resides on. So, it becomes much harder and much more to ensure that duplicate files never reside on the same physical disk. 

18 hours ago, zeroibis said:

SMART Data from Storage Spaces array. Because of the low level implementation of Storage Spaces, the disks in the array are hidden from access normally.  This makes getting information about the disk (including SMART data) significantly more difficult. 
 We are looking into implementing this feature in the future, but there is no ETA.

 Now I will state that HWInfo is still able to see the disks when they are being used in Storage Spaces so hopefully it is not to hard to get working.

 

We know how, and it's simple.  Microsoft added a new data structure for the disks when they're in a Storage Spaces array. 

The problem isn't changing the code to handle this, it's redesigning the code to display it in a meaningful way.  Which will basically require a drastic change to the UI.

 

 

Link to comment
Share on other sites

  • 0
30 minutes ago, Christopher (Drashna) said:

StableBit DrivePool does not support Dynamic disks. Period.  

There are a number of reasons for why, but the main one is that we are unable to (efficiently/quickly) determine what physical disk a file resides on. So, it becomes much harder and much more to ensure that duplicate files never reside on the same physical disk. 

We know how, and it's simple.  Microsoft added a new data structure for the disks when they're in a Storage Spaces array. 

The problem isn't changing the code to handle this, it's redesigning the code to display it in a meaningful way.  Which will basically require a drastic change to the UI.

 

 

Thanks for the info!

One note about the issue with dynamic disks, why not just assume that the underline physical disks do not matter to the user insofar as file placement goes. For example if I am creating dynamic disks to have a raid 1/0/5 etc and then trying to place such a volume into drivepool I am saying that I want that entire volume to be treated as a singe disk (basically exactly how it works in storage spaces). Thus the duplication check does not need to be looking at the underline drives.

I am sure it is actually a lot more complex than that and you are likely greatly simplifying the explanation so that we can understand it but just wanted to put up some food for thought.

Interesting that the Scaner issue is actually UI related. I suppose this must be also why the UI looks different when viewing the smart data for the same drive when connected to a HBA vs the motherboard.

Link to comment
Share on other sites

  • 0

Because you can have multiple volumes on a single disk.  Even normally.  And with dynamic disks, you can have multiple arrays be located on a single disk. 

So, if that disk fails.... and both duplicates were on that disk, then there was no point in duplicating.   We make sure that the software places duplication on different physical disks.  

The problem there is that dynamic disks are slow. The API for them is slow.   And the software has to decide what disk to place files on in the kernel. So that HAS to be fast. Otherwise, when you write 3000 files, it will take magnitudes longer to write that data. 

So, we made the conscious choice to not support dynamic disks.  Performance is important. It's not as important as reliability, stability and data integrity, but if your product is dirt slow, people will move on. ESPECIALLY when it comes to storage. 

On 10/10/2018 at 10:43 AM, zeroibis said:

Interesting that the Scanner issue is actually UI related. I suppose this must be also why the UI looks different when viewing the smart data for the same drive when connected to a HBA vs the motherboard.

Not just that. But how do we display a Storage Spaces array? 

Show the array in Scanner, like a normal disk?  Okay. how do we handle displaying SMART data, and the disk details for each disk?  Do we surface scan each disk or the entire array (trick question, it's the array)?

What about displaying the individual disks?  Okay, how do we show info about the file system scan? or the surface scan (since it can recover files from unreadable sectors, it would need to be run on the array, not the individual disks).  

See the problem here?  There isn't a good way with how the UI is designed now to show all of this information in a meaningful and accurate way. 

 

And all this translates over to RAID pretty directly, as well

 

Link to comment
Share on other sites

  • 0
4 hours ago, Christopher (Drashna) said:

Because you can have multiple volumes on a single disk.  Even normally.  And with dynamic disks, you can have multiple arrays be located on a single disk. 

So, if that disk fails.... and both duplicates were on that disk, then there was no point in duplicating.   We make sure that the software places duplication on different physical disks.  

The problem there is that dynamic disks are slow. The API for them is slow.   And the software has to decide what disk to place files on in the kernel. So that HAS to be fast. Otherwise, when you write 3000 files, it will take magnitudes longer to write that data. 

So, we made the conscious choice to not support dynamic disks.  Performance is important. It's not as important as reliability, stability and data integrity, but if your product is dirt slow, people will move on. ESPECIALLY when it comes to storage. 

Not just that. But how do we display a Storage Spaces array? 

Show the array in Scanner, like a normal disk?  Okay. how do we handle displaying SMART data, and the disk details for each disk?  Do we surface scan each disk or the entire array (trick question, it's the array)?

What about displaying the individual disks?  Okay, how do we show info about the file system scan? or the surface scan (since it can recover files from unreadable sectors, it would need to be run on the array, not the individual disks).  

See the problem here?  There isn't a good way with how the UI is designed now to show all of this information in a meaningful and accurate way. 

 

And all this translates over to RAID pretty directly, as well

 

For the dynamic disks stuff:

Yea that makes sense and I understand more why MS considers dynamic disks to be decrepit (as they stated with Win Server 2012 release). Also good point about all the volume options, I had forgotten about that possibility.

 

For the scanner stuff:

You have it show up as one unit like it does now as logically your going to scan the array and not the individual drive but you can just show data for each drive within that section. Think of how storage spaces itself displays the info you can see what drives are in the underline pool. You could even make it so there is just pages and you page though each drive you want to view data on. Honestly though, the data pages are not that important compared to the alerts themselves. As long as the scanner can tell me which of the underline drives is having a problem or if it is overheating etc that is super useful. It is not as though someone sits there all day looking at all the pages anyways. What they do care about is getting the various alerts and having as much detail about the source of the problem in the alert as possible. If you wanted to see the array and the part of the block display for each drive you could just make boxes to represent what part belongs to what drive assuming that info can be known.

Link to comment
Share on other sites

  • 0
19 hours ago, zeroibis said:

Yea that makes sense and I understand more why MS considers dynamic disks to be decrepit (as they stated with Win Server 2012 release). Also good point about all the volume options, I had forgotten about that possibility.

Yeah. Exactly.  

19 hours ago, zeroibis said:

You have it show up as one unit like it does now as logically your going to scan the array and not the individual drive but you can just show data for each drive within that section. Think of how storage spaces itself displays the info you can see what drives are in the underline pool. You could even make it so there is just pages and you page though each drive you want to view data on. Honestly though, the data pages are not that important compared to the alerts themselves. As long as the scanner can tell me which of the underline drives is having a problem or if it is overheating etc that is super useful. It is not as though someone sits there all day looking at all the pages anyways. What they do care about is getting the various alerts and having as much detail about the source of the problem in the alert as possible. If you wanted to see the array and the part of the block display for each drive you could just make boxes to represent what part belongs to what drive assuming that info can be known.

The problem is that would require a drastic change to the UI.  So that's not an insignificant change. 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...