Jump to content
Covecube Inc.
  • 0
Sign in to follow this  
Catch-22

Pool file duplication causing file corruption under certain circumstances

Question

I am using StableBit DrivePool and noticed that I am getting file corruption that is somewhat reproducible. I particularly noticed this with FLAC files as I was attempting to verify my library of music using flac -t which checks the MD5 signature of the decoded WAV file.

My setup is DrivePool with Folder Duplication enabled on specific folders, "Bypass filesystem Filters" checked, "Read striping" checked and "Real-time duplication checked".

Note that it appears to be "Read striping" that is the culprit for this but I am not 100% sure. Particularly concerning to me is that this happens even with "Verify after copy" checked.

Steps for me to reproduce are to download a FLAC to a DrivePool location that matches the above parameters, verify it with flac -t (ensure that it verifies OK), copy the file to a different location (doesn't even have to be a DrivePool location), verify this copied file with flac -t and see that the file does not verify anymore.

Checking with a hex editor, I can see it's not even just 1 byte difference, usually something like the first 32kb is fine, then I get random jumbled up data for a 128kb or so then the remainder of the file is correct (and the file size is correct).

Here's something I would never expect to see in a working filesystem:

PS F:\test\> flac -t .\test.flac

flac 1.3.2
Copyright (C) 2000-2009  Josh Coalson, 2011-2016  Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

test.flac: ok
PS F:\test\> copy .\test.flac test2.flac
PS F:\test\> flac -t .\test2.flac

flac 1.3.2
Copyright (C) 2000-2009  Josh Coalson, 2011-2016  Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

test2.flac: *** Got error code 2:FLAC__STREAM_DECODER_ERROR_STATUS_FRAME_CRC_MISMATCH


test2.flac: ERROR while decoding data
            state = FLAC__STREAM_DECODER_ABORTED

Edit:

It seems I am not the only one running into this problem, see this recent Reddit thread:

 

Share this post


Link to post
Share on other sites

5 answers to this question

Recommended Posts

  • 0

FLAC is a file format for lossless audio compression, you can download the flac utility from https://xiph.org/flac/download.html

The reason I use it as an example is because the FLAC format has built-in corruption checks, so it's easy to verify if the file contents is not what the user expects.

I suspect that the file format is largely irrelevant, however this does seem to be more likely to happen with larger files (ie. you probably can't reproduce this by copying small text files around).

Share this post


Link to post
Share on other sites
  • 0

Ok - i have downloaded the flac.exe

have tested on two different machines (win10 and 2012r2) with drivepool installed (2.3.0.1124 beta)

with same settings as you listed - i am not seeing any corruption of flac files - or any other files

flac.exe (1.3.2) always comes back "OK"

tried

copying to same folder as original

copying from pool to a non pool drive

tried several files - all copied fine and came back as "OK"

Share this post


Link to post
Share on other sites
  • 0

Well that's a 2 years old version - and there have been numerous changes/bug fixes since then

if you want to stay on the stable version 2.2.3.1019 is available from October 2019 - which is the latest

or move to the beta

and see if your problem goes away or not

 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
Sign in to follow this  

×
×
  • Create New...