Jump to content

ElectroStrong

Members
  • Posts

    3
  • Joined

  • Last visited

Everything posted by ElectroStrong

  1. Another comment on this topic - I have discovered the following when a OneDrive folder is located on a drivepool (in addition to not supporting Files On Demand/Reparse Points): DOS/PowerShell commands in a folder that is considered a OneDrive folder do _not_ allow for basic DOS commands. An example, if you try to use a "move" command it will fail with "access denied". When the file is copied to a folder outside of the OneDrive folder on a drivepool, the system will allow the move command. This also affects tools like ffmpeg. Interestingly, the Windows Explorer does support the movement of files through cut and paste...but not PowerShell/Cmd. OneDrive's sync engine does not seem to catch some files - I have files now in my folder structure that are setup for syncing but are not syncing as expected. Windows Backup, when trying to create a system image backup and save that in a drivepool, gives an error (The bootfile is too small to support persistent snapshots) when the VSS provider closes the snapshot. This is caused by the drivepool as backing up to a drive that is not a drivepool succeeds. It looks like the virtual driver that is used has a lot of limitations. Are these also known issues as well?
  2. Thanks for the response Christopher. I apologize for the confusion with the terminology - you are correct that they are not sparse files but they are considered reparse points. It looks like Microsoft has documented the interaction with the placeholder files here: https://learn.microsoft.com/en-us/windows/win32/cfapi/build-a-cloud-file-sync-engine There is also additional information on reparse points which are used for the placeholder files: https://learn.microsoft.com/en-us/windows/win32/fileio/reparse-points Finally - there are a bunch of examples on how to manage reparse points with a device driver perspective that could be used to establish a base design. A basic example in C identifies some of the core concepts and methods that need to be implemented, which will probably vary significantly from your device driver design, but the net result is that this is a well documented feature that shouldn't require a lot of reverse engineering. In fact, the core to the reparse points logic is in ntifs.h which is included in the Windows SDK: https://github.com/tpn/winsdk-10/blob/master/Include/10.0.16299.0/km/ntifs.h An example that I asked our favorite little AI tool to generate an example, which obviously has flaws in it but shows some of the core functions to call, is here as well: #include <ntddk.h> #include <ntdddisk.h> #include <mountdev.h> // Reparse point buffer structure typedef struct _REPARSE_DATA_BUFFER { ULONG ReparseTag; USHORT ReparseDataLength; USHORT Reserved; union { struct { USHORT SubstituteNameOffset; USHORT SubstituteNameLength; USHORT PrintNameOffset; USHORT PrintNameLength; ULONG Flags; WCHAR PathBuffer[1]; } SymbolicLinkReparseBuffer; struct { USHORT SubstituteNameOffset; USHORT SubstituteNameLength; USHORT PrintNameOffset; USHORT PrintNameLength; WCHAR PathBuffer[1]; } MountPointReparseBuffer; struct { UCHAR DataBuffer[1]; } GenericReparseBuffer; }; } REPARSE_DATA_BUFFER, *PREPARSE_DATA_BUFFER; #define REPARSE_DATA_BUFFER_HEADER_SIZE FIELD_OFFSET(REPARSE_DATA_BUFFER, GenericReparseBuffer) #define IOCTL_DISK_GET_REPARSE_POINT CTL_CODE(IOCTL_DISK_BASE, 0x0003, METHOD_BUFFERED, FILE_ANY_ACCESS) #define IOCTL_DISK_SET_REPARSE_POINT CTL_CODE(IOCTL_DISK_BASE, 0x0004, METHOD_BUFFERED, FILE_ANY_ACCESS) #define IOCTL_DISK_DELETE_REPARSE_POINT CTL_CODE(IOCTL_DISK_BASE, 0x0005, METHOD_BUFFERED, FILE_ANY_ACCESS) typedef struct _DEVICE_EXTENSION { PDEVICE_OBJECT DeviceObject; PDEVICE_OBJECT LowerDeviceObject; PUNICODE_STRING DeviceName; } DEVICE_EXTENSION, *PDEVICE_EXTENSION; // Reparse point filter functions NTSTATUS ReparsePointFilter_DispatchCreate(PDEVICE_OBJECT DeviceObject, PIRP Irp); NTSTATUS ReparsePointFilter_DispatchIoctl(PDEVICE_OBJECT DeviceObject, PIRP Irp); NTSTATUS ReparsePointFilter_DispatchClose(PDEVICE_OBJECT DeviceObject, PIRP Irp); NTSTATUS DriverEntry(PDRIVER_OBJECT driverObject, PUNICODE_STRING registryPath) { PDEVICE_OBJECT deviceObject = NULL; UNICODE_STRING deviceName; UNICODE_STRING linkName; PDEVICE_EXTENSION deviceExtension; NTSTATUS status; RtlInitUnicodeString(&deviceName, L"\\Device\\MyVirtualDisk"); RtlInitUnicodeString(&linkName, L"\\DosDevices\\MyVirtualDisk"); // Create the device object status = IoCreateDevice(driverObject, sizeof(DEVICE_EXTENSION), &deviceName, FILE_DEVICE_DISK, 0, TRUE, &deviceObject); if (!NT_SUCCESS(status)) { return status; } // Initialize the device extension deviceExtension = (PDEVICE_EXTENSION) deviceObject->DeviceExtension; deviceExtension->DeviceObject = deviceObject; deviceExtension->DeviceName = &deviceName; // Create the symbolic link status = IoCreateSymbolicLink(&linkName, &deviceName); if (!NT_SUCCESS(status)) { IoDeleteDevice(deviceObject); return status; } // Register the dispatch functions driverObject->MajorFunction[IRP_MJ_CREATE] = ReparsePointFilter_DispatchCreate; driverObject->MajorFunction[IRP_MJ_DEVICE_CONTROL] = ReparsePointFilter_DispatchIoctl; driverObject->MajorFunction[IRP_MJ_CLOSE] = ReparsePointFilter_DispatchClose; // Set the device object as ready deviceObject->Flags |= DO_DIRECT_IO; deviceObject->Flags &= ~DO_DEVICE_INITIALIZING; return STATUS_SUCCESS; } NTSTATUS ReparsePointFilter_DispatchCreate(PDEVICE_OBJECT DeviceObject, PIRP Irp) { PDEVICE_EXTENSION deviceExtension = (PDEVICE_EXTENSION) DeviceObject->DeviceExtension; PIO_STACK_LOCATION stack = IoGetCurrentIrpStackLocation(Irp); PFILE_OBJECT fileObject = stack->FileObject; // Check if the file being opened is a reparse point if (fileObject->FileName.Length >= sizeof(WCHAR) && fileObject->FileName.Buffer[fileObject->FileName.Length/sizeof(WCHAR) - 1] == L'$') { // Do something with the reparse point, such as redirecting the file to a different location // ... // Complete the IRP and return success Irp->IoStatus.Status = STATUS_SUCCESS; IoCompleteRequest(Irp, IO_NO_INCREMENT); return STATUS_SUCCESS; } // Forward the IRP to the next driver in the stack return IoPassIrpDown(DeviceObject, Irp); } NTSTATUS ReparsePointFilter_DispatchIoctl(PDEVICE_OBJECT DeviceObject, PIRP Irp) { PIO_STACK_LOCATION stack = IoGetCurrentIrpStackLocation(Irp); ULONG ioControlCode = stack->Parameters.DeviceIoControl.IoControlCode; PVOID inputBuffer = Irp->AssociatedIrp.SystemBuffer; ULONG inputBufferLength = stack->Parameters.DeviceIoControl.InputBufferLength; PVOID outputBuffer = Irp->AssociatedIrp.SystemBuffer; ULONG outputBufferLength = stack->Parameters.DeviceIoControl.OutputBufferLength; ULONG bytesReturned = 0; NTSTATUS status = STATUS_SUCCESS; switch (ioControlCode) { case IOCTL_DISK_GET_REPARSE_POINT: // Retrieve the reparse point data for the specified file // ... bytesReturned = sizeof(REPARSE_DATA_BUFFER); break; case IOCTL_DISK_SET_REPARSE_POINT: // Set the reparse point data for the specified file // ... break; case IOCTL_DISK_DELETE_REPARSE_POINT: // Delete the reparse point for the specified file // ... break; default: // Forward the IRP to the next driver in the stack return IoPassIrpDown(DeviceObject, Irp); } // Complete the IRP and return success Irp->IoStatus.Information = bytesReturned; Irp->IoStatus.Status = status; IoCompleteRequest(Irp, IO_NO_INCREMENT); return status; } NTSTATUS ReparsePointFilter_DispatchClose(PDEVICE_OBJECT DeviceObject, PIRP Irp) { // Do any necessary cleanup when a file handle is closed // ... // Forward the IRP to the next driver in the stack return IoPassIrpDown(DeviceObject, Irp); } This example does implement the IRP concept for the device stack as well. There are also examples as to how the Cloud API filter driver works in Windows which also supports reparse points: https://github.com/microsoft/Windows-classic-samples/tree/main/Samples/CloudMirror There's quite a bit of documentation out there on this topic from what I've seen...is this candidly a feature that does not have as much importance as opposed to other features to be added? Reparse points are used in other situations beyond this - they are also used in volume mountpoints and another common use for reparse points is backup files that are archived that are then retrieved (not using the Cloud API from Microsoft for example). I apologize for the length of this post - just trying to help out as I think this feature would be used by others - it's just the most obvious hole I have in my current setup and figured I'd inquire.
  3. I am in the process of evaluating DrivePool as a replacement to the Windows RAID mirroring. One of the problems that I've run into revolves around the fact that I can not longer use the OneDrive Files on Demand feature when the OneDrive folder is located in a drive pool. This is consistent with feedback that I've seen in the reddit community in which other users have stated that they have the same issue. This is a deal-breaker for me - I use multiple machines and sparse files allow for me to access these files without storing them locally on my drive configuration. With Windows RAID, it worked without issues as you can imagine. My main focus around using DrivePool is to have a redundant copy of data in case of a drive failure so the other use cases of DrivePool, such as SSD caching and multi-drive configurations are not as useful to me as having the ability to support sparse files with OneDrive. Is this a known issue? Is there any plans to add support for this capability? I'd appreciate any input on this topic so I can make a decision on what my path forward is for my use case.
×
×
  • Create New...