Duplication warning; CRC error

[JohnT]

Hi, just wondering what the best way is to deal with a CRC error warning during duplication?

After duplication is complete, there is a warning, with a CRC error. Shouldn't the 'invalid' file be corrected from the cuplicate (I have x2)

 

Thanks,

 

John

[Shane]

CRC error indicates a problem reading from / writing to a physical drive (you may find this link helpful); that drive may be going bad and you may need to remove it from the pool (if you do so, I'd recommend ticking "Force damaged file removal"). If DrivePool encountered the error while attempting to create a duplicate of a new/updated file you may need to restore the file from a backup. If DrivePool encountered the error while checking the consistency of an existing duplication, you'll need to identify which is the bad copy and remove it (the file and/or the drive) so that DrivePool can then re-duplicate from the good copy.

[JohnT]

On 3/11/2024 at 3:30 AM, Shane said:

CRC error indicates a problem reading from / writing to a physical drive (you may find this link helpful); that drive may be going bad and you may need to remove it from the pool (if you do so, I'd recommend ticking "Force damaged file removal"). If DrivePool encountered the error while attempting to create a duplicate of a new/updated file you may need to restore the file from a backup. If DrivePool encountered the error while checking the consistency of an existing duplication, you'll need to identify which is the bad copy and remove it (the file and/or the drive) so that DrivePool can then re-duplicate from the good copy.

Thanks, I understand the CRC error, but don't understand why Drivepool isn't dealing with it. Since it isn't, is the suggestion above the best way for me to deal with it? Why doesn't Drivepool do this itself? And if so, then what is the best way to locate the CRC file?

I'm also rather concerned at the suggestion to 'restore the file from a backup'. My understanding of duplication is that (and certainly that is the way I use it) it should prevent data loss from a single file/drive failure. Drivepool IS my backup!

Thanks,

John

[Shane]

The CRC error may be on the source drive or the destination drive (as described in the page I linked). At least it's letting you know there is a problem even if it can't fix it for you. Can you post a screenshot of the warning that DrivePool is displaying?

"I'm also rather concerned at the suggestion to 'restore the file from a backup'. My understanding of duplication is that (and certainly that is the way I use it) it should prevent data loss from a single file/drive failure. Drivepool IS my backup!"

Your understanding is correct, DrivePool does provides a RAID-like defense against data loss from drive failure(s), but there's a phrase in IT: "RAID is not Backup!" This is because there are many more ways to lose data than a drive dying. You could accidentally delete a file. A virus could deliberately delete a file (or all of them). A bug in a program or a fault in a controller could corrupt a file. Duplication does not protect against the first two and provides only limited protection against the third. There is also the very rare but not impossible chance of multiple drives dying simultaneously (not just by bad hardware but also by disaster such as fire or flood). This is why also having separate backups is important too.

[JohnT]

21 hours ago, Shane said:

The CRC error may be on the source drive or the destination drive (as described in the page I linked). At least it's letting you know there is a problem even if it can't fix it for you. Can you post a screenshot of the warning that DrivePool is displaying?

"I'm also rather concerned at the suggestion to 'restore the file from a backup'. My understanding of duplication is that (and certainly that is the way I use it) it should prevent data loss from a single file/drive failure. Drivepool IS my backup!"

Your understanding is correct, DrivePool does provides a RAID-like defense against data loss from drive failure(s), but there's a phrase in IT: "RAID is not Backup!" This is because there are many more ways to lose data than a drive dying. You could accidentally delete a file. A virus could deliberately delete a file (or all of them). A bug in a program or a fault in a controller could corrupt a file. Duplication does not protect against the first two and provides only limited protection against the third. There is also the very rare but not impossible chance of multiple drives dying simultaneously (not just by bad hardware but also by disaster such as fire or flood). This is why also having separate backups is important too.

OK, by bad usage of the term backup :-). I meant; 'relatively safe data storage, with reduced change of (single) mechanical drive failure or bitrot causing data loss'. I understand that it is not a replacement for (e.g.) off-site offline tape storage. And I agree that knowing is better than not. But fixing (as it does when it detects duplicate parts with different DTS) would be even better.

It isn't a new file, so I don't understand the source/destination reference. I thought that I had posted an image with the initial post, but apparently not. Hopefully that works this time 🙂

[Christopher (Drashna)]

Yeah, that typically is a file system error.   The path is the path on the pool, and best bet is to copy the file from elsewhere. 

Worst case, the "dpcmd" tool can be used to identify which drives that this file exists on.  Likely one of these will have read errors, and will need to be resolved.  A CHKDSK pass on the disk *may* help, but emphasis on "may" here.