Jump to content
  • 0

How do I correctly backup Stablebit Drivepool?


riahc3

Question

Ill never understand why Drivepool and VSS are not compatible.

Samba, which is open source software, is fully compatible with VSS. So its not about "there isnt documentation about it"; There is an implementation already!

 

Anyways, I want to know what is Stablebit's official procedure on making a backup of a pool and how to restore from backup.

 

Thanks 

Link to comment
Share on other sites

12 answers to this question

Recommended Posts

  • 0
3 hours ago, riahc3 said:

Ill never understand why Drivepool and VSS are not compatible.

 Samba, which is open source software, is fully compatible with VSS. So its not about "there isnt documentation about it"; There is an implementation already!

Because there is no documentation on how to support VSS on the file system level.  
There is documentation on how to access VSS, and plenty of it.  But that's not the issue.  The problem is how the file system is supposed to handle the VSS calls.  There is NO documentation on this, in the wild.   Any documentation that may exist is internal Microsoft documentation. 

If by Samba, you mean Samba/SMB/CIFS/Windows Shares, then you're just connecting to the API.  You're relying on the underlying drive that the SMB share resides on supporting VSS.   This is the top level VSS stuff, we need the bottom/low level info, how you'd implement it on a different file system.

So, right now, we'd have to reverse engineer exactly how VSS interacts with NTFS, at the file system level.  That is not a simple thing, at all. And it would be incredibly time consuming.

If you mean a specific software, could you link it? 

3 hours ago, riahc3 said:

 Anyways, I want to know what is Stablebit's official procedure on making a backup of a pool and how to restore from backup.

Back up the underlying disks in the pool, not the pool drive. 

As for restoring .... basically the same. 

That or used something file based, or a sync utility (such as AllWays sync, good sync, free file sync, synctoy, etc).

Link to comment
Share on other sites

  • 0
On 4/5/2019 at 2:42 AM, Christopher (Drashna) said:

Because there is no documentation on how to support VSS on the file system level.  
There is documentation on how to access VSS, and plenty of it.  But that's not the issue.  The problem is how the file system is supposed to handle the VSS calls.  There is NO documentation on this, in the wild.   Any documentation that may exist is internal Microsoft documentation. 

If by Samba, you mean Samba/SMB/CIFS/Windows Shares, then you're just connecting to the API.  You're relying on the underlying drive that the SMB share resides on supporting VSS.   This is the top level VSS stuff, we need the bottom/low level info, how you'd implement it on a different file system.

So, right now, we'd have to reverse engineer exactly how VSS interacts with NTFS, at the file system level.  That is not a simple thing, at all. And it would be incredibly time consuming.

If you mean a specific software, could you link it? 

Back up the underlying disks in the pool, not the pool drive. 

As for restoring .... basically the same. 

That or used something file based, or a sync utility (such as AllWays sync, good sync, free file sync, synctoy, etc).

4

Im sorry but I dont understand....

 

Every software on planet earth can use VSS perfectly. I dont see any reason your software can (even if it has to do it on each individual drive instead of the pool)

 

https://wiki.samba.org/index.php/Virtual_File_System_Modules

https://github.com/zfsonlinux/zfs-auto-snapshot/wiki/Samba

https://forum.openmediavault.org/index.php/Thread/7331-GUIDE-Windows-Previous-Versions-and-Samba-Btrfs-Atomic-COW-Volume-Shadow-Copy/

These documents/links show how to leverage VSS over Samba as long as the underlying file system supports snapshots (such as ZFS). AFAIK, the underlying file system (the rest of the disks) are NTFS so they can be VSSd...

 

About backing up, IMO you need to be much clearer and it needs to be added in official documentation...

"Backup the underlying disks in the pool"...So what you are saying is that instead one volume being backed up, its necessary to back up several (in the sense that one disk = one volume)? And then restore them?

 

I dont think its that difficult; Drivepool should provide a easy way for backup software using VSS to backup its pool. Not allowing so breaks compatibility with a lot of backups software.

 

Link to comment
Share on other sites

  • 0

There is a difference between using VSS and supporting it. If you use VSS, then VSS will do all sorts of stuff for you and in turn use, for instance, NTFS. NTFS must therefore be able to support functions that VSS uses. It is not that easy to know what functions VSS uses and that is what you need to know to support it.

As a, probably faulty, analogy, say you want to program something for your video card to do. You need to make calls to the, say, nVidia driver. These will be documented as nVidia wants people to program for their drivers. Now suppose that driver makes calls to the OS and you want to write your own OS on which that nVidia driver runs. Then you need to know what that driver needs from the OS and facilitate that. However, what that driver needs from the OS may not be documented.

And there is a really good reason not to want to backup the entire volume. Let's say you have a 4 x 4TB Pool, duplicated. You may have 6TB of (unduplicated) data in that Pool. If you then try a BMR, you would need one HDD of at least 6TB to be able to restore that _or_ the recovery software must be able to load/support DrivePool as well. I don;t know what you use but the Windows Server Backup recovery software won't do that.

So yes, I backup the underlying disks.

Link to comment
Share on other sites

  • 0
On 4/7/2019 at 2:50 PM, riahc3 said:

Every software on planet earth can use VSS perfectly. I dont see any reason your software can (even if it has to do it on each individual drive instead of the pool)

Use.  They use VSS, they talk to it.  They pull data from it.  

To support VSS, we have to support the API that VSS uses to talk to the drive.  Not that programs use to talk to VSS. 

The difference is that you're talking about the front end.   which has plenty of documentation.  We need to support the backend.  which has NONE. 

 

 

 

Or in more detail, when SAMBA uses VSS, it makes an API call to VSS, which then makes an API call to the file system.  

The  API calls to VSS are documented.  The API calls to the file system are a black box. 

Link to comment
Share on other sites

  • 0
On 4/9/2019 at 1:27 AM, Christopher (Drashna) said:

Use.  They use VSS, they talk to it.  They pull data from it.  

To support VSS, we have to support the API that VSS uses to talk to the drive.  Not that programs use to talk to VSS. 

The difference is that you're talking about the front end.   which has plenty of documentation.  We need to support the backend.  which has NONE. 

 

 

 

Or in more detail, when SAMBA uses VSS, it makes an API call to VSS, which then makes an API call to the file system.  

The  API calls to VSS are documented.  The API calls to the file system are a black box. 

Could you explain how other software leverages it then?

Link to comment
Share on other sites

  • 0
On 6/1/2019 at 2:30 AM, riahc3 said:

Could you explain how other software leverages it then?

That depends on what you mean.

Most other programs make calls to VSS. 

It would ONLY be disk drivers that would need to deal with VSS calls to the file system.  And then, only file based solutions that would need to, as block based solutions are file system agnostic. 

Link to comment
Share on other sites

  • 0

@riahc3: How much data are we talking about and what kind of backup solution(s) do you (want) to use? As an example, I can not imagine image restore or BMR to work. Personally, I use Windows Server Backup (which uses VSS) to backup individual drives (and yes, I have x2 duplication but manage to backup the files only once by using hierarchical Pools).

Link to comment
Share on other sites

  • 0
On 6/14/2019 at 1:39 AM, Umfriend said:

Now I am rather ignorant on these kind of things but... how do other disk driver creators deal with this? Or is MS the only one?

They don't.  Or rather, most of them are block based, so they don't need to deal with this.

As for drivers like ours.... either they don't support it at all, or they have support, but the code is closed source and obfuscated.  

As in, I know we've talked about this internally, and Alex has an answer: "Yes, it can be done" EldOS does it, I think.  But if you check the price .... it's not cheap.  It also runs in userspace.... 

Link to comment
Share on other sites

  • 0
On 7/9/2019 at 2:38 AM, Christopher (Drashna) said:

They don't.  Or rather, most of them are block based, so they don't need to deal with this.

As for drivers like ours.... either they don't support it at all, or they have support, but the code is closed source and obfuscated.  

As in, I know we've talked about this internally, and Alex has an answer: "Yes, it can be done" EldOS does it, I think.  But if you check the price .... it's not cheap.  It also runs in userspace.... 

So there IS a solution

Link to comment
Share on other sites

  • 0
On 8/18/2019 at 10:52 PM, riahc3 said:

So there IS a solution

Correct.  But it's not documented, at all, publically.  

That means that we would have to reverse engineer it, and test it extensively. Which is a large commitment of time, for a small company. 

If there was a simple solution, we'd love to implement that, but .... there isn't. 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...