Jump to content
Covecube Inc.
  • 0

Quorum based file movement by hash?


Midoxai
 Share

Question

Hello,

Recently i read about HDFS. I think rebalancing the files on file hashes with a quorum would be a cool feature.

This would work in the following way:

  1. Rebalancing is scheduled by another plugin, e.g StableBit Scanner Plugin
  2. File movements are calculated
    1. Before a file is moved, it is read on multiple Disks and a hash (e.g md5, sha256, ... ) is calculated
    2. If the hash is the same on all disks, all disks are used to move the file
    3. If the hash is diffrent on the disks a Quorum is used to identify the disks where the file is read from
  3. Files getting copied/moved

 

The Balancing could also do the following

  1. On a schedule, the hash for every file on every disc is calculated.
  2. If the hashes are diffrent on some disks, Quorum is used to determine from wich disk the files are replaced.
  3. Files are copied

This would add a new level of integrity for the files.

 

Link to comment
Share on other sites

1 answer to this question

Recommended Posts

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

×
×
  • Create New...