Simultaneous mass NTFS corruption: 2 Pools (2 pairs of 2 drives in 4 drive USB chassis...)

[CosmicPuppy]

Just sharing the mini-nightmare I experienced this week, in case y'all have any recommendations:

 

• I have an IcyDock external 4 drive JBOD array, running SATA disks over its single USB interface because it is connected to an Intel NUC with no eSATA ports (there's one SATA connected to an external BluRay drive).
   
• I have a UPS and the array has been running for months with no problems.
   
• Early yesterday I had some USB device errors (keyboard or something) and accidentally pull the array's USB cable. Plugged it back in and everything seemed fine ... but not for long.
   
• After a reboot, all 4 drives in the array came up with their NTFS partitions marked as "RAW". Geesh! I thought NTFS was more tolerant to a little glitch than that!
   
• I felt some comfort that because I had DrivePool with full duplication enabled, I would have two opportunities to recover the data. I purchased "GetDataBack Simple" for this purpose. It recovered most of the folders and files intact, though a few folders lost their names and proper locations in the tree. I recovered by writing to a pair of larger external USB drives already connected to the PC.

 

 

So... Now I have to put my array back together again... But am rethinking the layout:

 

1. Since a "hiccup" on the one USB cable and/or chassis has proven that it can destroy both supposedly independent NTFS file systems, obviously one duplicate of each drive should be on a different USB bus ... either a separate chassis or independent external drives.
    
2. The 4 chassis drives are 1tb each and I have a pair of 3tb external drives. So my plan is to partition the 3tb drives into 3 (1tb, 1tb, and slightly under 1tb), and pair up chassis drive to each partition. But wait ... that doesn't quite balance out!
    
3. So I guess I could create 1 or 2 Pools and just create balancing rules that ensure all duplicates span across the total of USB ports (1 chassis USB with 4 drives + 2 external USB drives)?

 

Question:

• Is it possible for a partition (a single NTFS "drive") to belong to more than one Pool? The 3tb drives, for example, could then belong to 1 Pool for mirroring 2 of the chassis disks each, and also belong to a separate Pool to duplicate each other?
   
• Of course... I can do the above with partitioning, but since DrivePool uses a top-level NTFS Folder to hold the pooled data, can't it just have 2 (or more!) folders on each single partition, one such folder for each pool the partition belongs to?

 

Thanks for indulging my insanity. DrivePool is a powerful product, so there's bound to be several bad ways and a couple good ways to handle this.

 

I'm presuming that DrivePool itself didn't play any role in the simultaneous NTFS corruption ... but let me know if that is not a safe assumption.

 

 

 

[grimpr]

Looks to me you lost the partition tables, in which case you had the hard disks as MBR, switch to GPT to stay on the safe side of partition table corruption.

[Christopher (Drashna)]

First, I'm sorry to hear about that! 

 

Unfortunately, this is relatively common with USB drives.  it's actually "allowed" by the USB specification. 

Specifically, the drives can periodically disconnect, spontaneously.  This can cause write errors and corruption. If the drive (or drives) were writing to the partition table at that time, it can corrupt the entire partition, causing it to show up as "RAW".  

 

But Grimpr is correct here. For data drives, I would HIGHLY recommend initializing as GPT (at least in the future). This is because the partition information is stored twice. At the beginning and end of the disk.  This helps reduce issues like this. 

 

 

However, the first thing you should do this this case is run a CHKDSK pass. In some cases, that can actually fix the issue and return the drive to working order.  I've had a few drives "brought back to life" by doing this. 

 

 

 

So... Now I have to put my array back together again... But am rethinking the layout:

 

1. Since a "hiccup" on the one USB cable and/or chassis has proven that it can destroy both supposedly independent NTFS file systems, obviously one duplicate of each drive should be on a different USB bus ... either a separate chassis or independent external drives.
    
2. The 4 chassis drives are 1tb each and I have a pair of 3tb external drives. So my plan is to partition the 3tb drives into 3 (1tb, 1tb, and slightly under 1tb), and pair up chassis drive to each partition. But wait ... that doesn't quite balance out!
    
3. So I guess I could create 1 or 2 Pools and just create balancing rules that ensure all duplicates span across the total of USB ports (1 chassis USB with 4 drives + 2 external USB drives)?

 

Question:

• Is it possible for a partition (a single NTFS "drive") to belong to more than one Pool? The 3tb drives, for example, could then belong to 1 Pool for mirroring 2 of the chassis disks each, and also belong to a separate Pool to duplicate each other?
   
• Of course... I can do the above with partitioning, but since DrivePool uses a top-level NTFS Folder to hold the pooled data, can't it just have 2 (or more!) folders on each single partition, one such folder for each pool the partition belongs to?

 

Thanks for indulging my insanity. DrivePool is a powerful product, so there's bound to be several bad ways and a couple good ways to handle this.

 

I'm presuming that DrivePool itself didn't play any role in the simultaneous NTFS corruption ... but let me know if that is not a safe assumption.

 

No, a volume (or partition) can only belong to a single pool at a time.  

The reason for the limit is that it would become incredibly complex when balancing data... 

 

However, a feature we have planned for the future is "duplication grouping". So you could specify a group of disks to be used for one copy, and another group for the other copy.  That may be better for what you want. 

But we don't have an ETA for this feature, unfortunately.  But it is a higher priority (in a large part because of StableBit CloudDrive).