[Beware of DrivePool corruption / data leakage / file deletion / performance degradation scenarios Windows 10/11]

I would add that if Stablebit have gone to the effort of providing a unique and stable 128bit object-ID for every file in the pool, why not use the same code base to also provide a stable 64bit FileID?

[Beware of DrivePool corruption / data leakage / file deletion / performance degradation scenarios Windows 10/11]

Nice find. Makes sense to me.

Hopefully Chris/Alex can make Drivepool compliant with those requirements.

[Beware of DrivePool corruption / data leakage / file deletion / performance degradation scenarios Windows 10/11]

 

Quote

 

Zenju12 Dec 2023, 14:33

Let's see...
File object IDs are supported only on NTFS volumes

But for NTFS we have:
In the NTFS file system, a file keeps the same file ID until it is deleted

Ergo:
Object ID = useless
File ID = perfect as persistent file identifier (on NTFS).

 

Response from Freefilesync developer. 
 

I read through the Microsoft docs you posted earlier and others, and I agree with the Freefilesync developer. 

It appears the best way to track all files on a volume on NTFS is to use fileid which is expected to stay persistent. This requires no extra overhead or work as the Filesystem maintains FileID’s automatically. 
ObjectID requires extra overhead and is only really intended to track special files like shortcuts for link tracking etc.

 

Any software that is emulating an NTFS system should therefore provide FileID’s and guarantee they stay persistent with a file on that volume.

 

I am seeing the direct performance impact from this and agree with Mitch that there can be other adverse side affects potentially much worse than just performance issues if someone uses software that expects FileID’s to behave as per Microsoft’s documentation. 
 

Finally also note that ObjectID is not supported by the Refs filesystem, whereas FileID is.

https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/ns-ntifs-_file_objectid_information

ReFS doesn't support object IDs. ReFS uses 128-bit file IDs, so can't cleanly distinguish between file ID versus object ID when processing an open by ID. 

 

[Beware of DrivePool corruption / data leakage / file deletion / performance degradation scenarios Windows 10/11]

Thanks for the reply Chris.

Note: the beta does not fix the FileID change on rename or copy issue.

I have posted your comment on the Freefilesync forums and will see if Object-ID is an option for consideration there.

Meanwhile I'd still think that it would be better if file-id behaved more like regular ntfs volumes and stayed persistent.

From the same document you referenced....

Quote

In the NTFS file system, a file keeps the same file ID until it is deleted. You can replace one file with another file without changing the file ID by using the ReplaceFile function. However, the file ID of the replacement file, not the replaced file, is retained as the file ID of the resulting file.

It mentions that with the FAT file system it is not safe to assume file-id will not change over time, but with NTFS it appears the file-id is indeed persistent for the life of the file.

[Beware of DrivePool corruption / data leakage / file deletion / performance degradation scenarios Windows 10/11]

Quote

It depends what the application uses the FileID for but it may assume the FileID should stay the same when a file moves, as it doesn't with DrivePool this might mean it reads or backs up, or syncs the entire file again if it is moved (perf issue).

Very well explained Mitch. I just discovered this issue as well while trying to work out why my installation of Freefilesync wasn't behaving as expected.

Drivepool indeed changes fileid every time a file is renamed or moved which is not correct NTFS behaviour.

The result is that if i move say 100GB of data on my drivepool from one folder to another (or rename a large group of files) when I run freefilesync for backup instead of mirroring the file moves or renames, it needs to delete and recopy every moved or renamed file. Over the network this can take hours instead of less than a second so the impact is substantial.

[When do file ids/inodes change for files in a pool?]

I am also seeing this behaviour and it affects performance greatly of Freefilesync. This software uses fileid to track moved files and keep folders in sync.

When I move some files on the drivepool (or rename them) the fileid changes on the Drivepool. This does not happen for any of my normal or networked drives which maintain the fileid for moves, renames etc.

The result is that if i move say 100GB of data on my drivepool from one folder to another (or rename a large group of files) when I run freefilesync for backup instead of mirroring the file moves or renames it needs to delete and recopy every moved or renamed file. Over the network this can take hours instead of less than a second so the impact is substantial.