Drive must have died is there a way to tell what was on it?

[chrismilne]

How do i tell what is missing now?  I only have one folder duplicated (home photos) in my WHS drivepool settings.  The disk is clicking when i try to hook it up via an external interface and does not show up in windows...It is one of 11 drives, so i have no idea what was on it, is there a log or a file i can get any ideas?  I remember when drives lasted more than 2 years or so...

[Christopher (Drashna)]

First, stop using that drive. Unplug it, and leave it that way for now. IMMEDIATELY.

 

Unfortunately, StableBit DrivePool doesn't maintain a list of files. We rely on the drives for that, and in most cases, that should be more than sufficient. 

 

In theory, the drive should give warnings before it starts head crashing. Did you have any SMART monitoring software on the system?

That clicking sound is a head crash and VERY bad. And potentially damaging to the disk.

 

 

Data recovery may work, but chances are....

[dbailey75]

How do i tell what is missing now?  I only have one folder duplicated (home photos) in my WHS drivepool settings.  The disk is clicking when i try to hook it up via an external interface and does not show up in windows...It is one of 11 drives, so i have no idea what was on it, is there a log or a file i can get any ideas?  I remember when drives lasted more than 2 years or so...

I hope you have a back up set, if so you should be able to do a restore of only the missing files.

[airjrdn]

I just got bit by this exact same thing.  A drive died, I didn't have all data set to at least 2x duplication, and I have no idea what I lost.  I know I didn't lose anything too critical, but the fact that I don't know what I lost is concerning.  It isn't the fault of DrivePool, but if DrivePool isn't going to at least be able to report what was lost, I guess I'll have to come up with something.  It would be nice if that were a DrivePool feature though.

 

I am curious though, as to why Scanner didn't report anything ahead of time.

[dbailey75]

I just got bit by this exact same thing.  A drive died, I didn't have all data set to at least 2x duplication, and I have no idea what I lost.  I know I didn't lose anything too critical, but the fact that I don't know what I lost is concerning.  It isn't the fault of DrivePool, but if DrivePool isn't going to at least be able to report what was lost, I guess I'll have to come up with something.  It would be nice if that were a DrivePool feature though.

 

I am curious though, as to why Scanner didn't report anything ahead of time.

 

Agreed like a file inventory with hash, and a way to compare the inventory at various points in time, I'd buy it.    I've brought this up a number times,  for now your best bet is a file hash too like  exact file or fileverifier++, both will create an export inventory of all your files.

[Christopher (Drashna)]

Unfortunately, no DrivePool doesn't keep a separate list of the drive contents, nor a hash of them. We rely on NTFS for that. However, obviously, that's not always a great idea.

 

Alex has heard the requests, and ... well plans on changing that:

http://community.covecube.com/index.php?/topic/689-worrying-issue-with-pictures-etc/page-2&do=findComment&comment=4596

[bytemaster]

I have considered the situation that with a large library I would not know what I had lost in the event of disk failure.

 

What I have done is mount all pool drives as folders under a common folder "/mount". I then catalogue this folder using a program called VVV (Virtual Volumes View). I used to use Kathy for cataloguing but VVV allows a csv export of the entire catalogue. This way I have a record of the location of every file within the pool.

 

My only problem is remebering to keep the VVV catalogue updated

[Christopher (Drashna)]

I have considered the situation that with a large library I would not know what I had lost in the event of disk failure.

 

What I have done is mount all pool drives as folders under a common folder "/mount". I then catalogue this folder using a program called VVV (Virtual Volumes View). I used to use Kathy for cataloguing but VVV allows a csv export of the entire catalogue. This way I have a record of the location of every file within the pool.

 

My only problem is remebering to keep the VVV catalogue updated

That sounds like a great solution. 

 

But let me make it even better for you. (I doubt you'd object).

VVV appears to support command line switches. This is awesome, because you can set up Task Scheduler to run the program at specific intervals. Such as once a day, or week. Even if you're not logged in.

http://vvvapp.sourceforge.net/command_line_switches.html

 

It's for "advanced users", so it may be a PITA to setup, but I suspect that it may be worth it.

[4Frame]

That really sucks, and hope you're able to recover your lost data. 

I, too, lost a drive the other day (WD2002FYPS, Mfg Date: 14 May 2012) and it sucked. It died with no imminent SMART warnings at all. But I suspect it died because of the high load cycle count (which still baffles me, as this is an enterprise-grade drive).

I use Drivepool in combination with SyncBack pro to mirror/backup the entire pool to an external disk. On SyncBack I set up versioning so that if files are missing/deleted on the next source->destination check, they will get placed into a versioning directory. At the very least, if a drive died on my pool, the next Sync on SyncBack will show me what files I lost, if any, and I could recover them.

 

[Asturia]

You can think of integrating this onto your server:

http://snapraid.sourceforge.net/

 

This way, you can recover from a disk that dies.

[Christopher (Drashna)]

You can think of integrating this onto your server:

http://snapraid.sourceforge.net/

 

This way, you can recover from a disk that dies.

Yup. A number of users use SnapRAID or similar products in conjunction with StableBit DrivePool to protect your data.

[kihimcarr]

What other products can be used with DP to protect data? I just had 2x drives fail within days of each other and wasn't able to recover any of the data. This has been a sad Thanksgiving holiday!

[Christopher (Drashna)]

Ouch, I'm really sorry to hear that.

 

As for other software to use with DrivePool... that depends on what you're trying to do.

 

SnapRAID and FlexRAID are good ways to add parity protection to the pool. But that depends on not losing too many disks, as well.

 

You could use something like CrashPlan to backup the pool to the Cloud (and they offer a "direct to door" recovery service, IIRC).

 

You could use a file inventorying tool to see what you've lost. (I think fv++ will do this).

 

You could backup or copy the contents of the pool to another disk or pool, using something like SyncBack, Allways Sync, Free File Sync, etc.

[kihimcarr]

Going the SnapRAID or FlexRAID route, the only thing is I will not be able to use balancing since the parity cannot be moved. Am I thinking correctly here?

 

For the file inventory tool. I would have to have an inventory completed already to see what the differences are.

 

There is no silver bullet, so I have some planning to do. Thanks for the reply.

[puppetj]

You can think of integrating this onto your server:

http://snapraid.sourceforge.net/

 

This way, you can recover from a disk that dies.

hi im instrissed in snapraid, but how would it work along with drive pool from stable bit? since you cant use the drives that are already add to the pool, you would need more drives just for snapraid right?

 

if not please explain, as i did look at the getting started and .conf file, and im truely confused!

[Christopher (Drashna)]

Going the SnapRAID or FlexRAID route, the only thing is I will not be able to use balancing since the parity cannot be moved. Am I thinking correctly here?

 

For the file inventory tool. I would have to have an inventory completed already to see what the differences are.

 

There is no silver bullet, so I have some planning to do. Thanks for the reply.

yes, you're correct there. You'd want to avoid balancing. IIRC, the best option actually is to use the Ordered File Placement balancer, to fill up a disk (or two) at a time.

 

And yeah, you'd need to do this ahead of time.

If you have the opportunity now, "fv++" has been mentioned previously, and may work for you. 

 

And no, there are no silver bullets. IIRC, SnapRAID and FlexRAID can do pooling, as well, but it's nowhere near as simple or transparent as StableBit DrivePool is.

 

hi im instrissed in snapraid, but how would it work along with drive pool from stable bit? since you cant use the drives that are already add to the pool, you would need more drives just for snapraid right?

 

if not please explain, as i did look at the getting started and .conf file, and im truely confused!

We have a bunch of links, in fact:

http://community.covecube.com/index.php?/topic/52-faq-parity-and-duplication-and-drivepool/

 

The post talks about parity, and the reply to it includes links to other threads talking about the products, and how to set them up with StableBit DrivePool. 

[Pål Andreassen]

If you have your drives mapped with drive letters you can run the following script (say daily) to have it generate a file list per drive and place in DropBox:

 

@echo off

set dropbox=C:\Users\SomeUser\Dropbox\DrivePoolContent\

for %%i in (C D E F G H I J K L M N O P Q R S T U V W X Y Z) DO (

  if exist %%i: (

    dir %%i:\ /ahs /s /b  > "%dropbox%\FileList_%%i.txt" 2> nul

  )

)

 

If you don't want to assign driveletters you can also mount all your disks under a single mount point instead of drive letters (say C:\PoolDisks\) and just dir that folder.

[duelistjp]

gotta agree with pal i have a script that runs tree on my media drives every week and then saves it in the google drive folder.  every couple months i delete the older ones but no real need for that honestly. google gives me plenty of space for some text files

[rtech73]

I had a drive die last night, kind of took the whole pool drive with it (in that it would essentially bog down or even lock up the system when the pool was accessed).   I knew which drive was the culprit, removed it and the DrivePool took care of the rest and everything is now fine.   To me, all the above is negated by simply having everything duplicated (and telling it not to put ANY unduplicated files on any drives showing SMART errors, or anything else).  When I removed the drive (which I had to do by force and have DP recover from a "missing" drive), of course the system knew what I was missing because there was a duplicate copy of it elsewhere!)   Of course, to do that, I have 20 drives!   :-)

 

I guess I'm just curious why one would go through all the trouble of having a list of files, when you could just duplicate everything on the pool?   (I don't mean this as judgmental or anything, I'm actually curious as to the rationale!)

 

As far as my own drive dying, it was a PERFECT example of "the system works".    Huzzah for DrivePool!

[Christopher (Drashna)]

I had a drive die last night, kind of took the whole pool drive with it (in that it would essentially bog down or even lock up the system when the pool was accessed).   I knew which drive was the culprit, removed it and the DrivePool took care of the rest and everything is now fine.   To me, all the above is negated by simply having everything duplicated (and telling it not to put ANY unduplicated files on any drives showing SMART errors, or anything else).  When I removed the drive (which I had to do by force and have DP recover from a "missing" drive), of course the system knew what I was missing because there was a duplicate copy of it elsewhere!)   Of course, to do that, I have 20 drives!   :-)

 

I guess I'm just curious why one would go through all the trouble of having a list of files, when you could just duplicate everything on the pool?   (I don't mean this as judgmental or anything, I'm actually curious as to the rationale!)

 

As far as my own drive dying, it was a PERFECT example of "the system works".    Huzzah for DrivePool!

 

First, I'm sorry to hear about the hard drive dying, as that sucks regardless of the status of your data.

 

Because duplicating 15TB of video is ... painful. If it's not "critical", then it's not as important to duplicate and takes up a lot of space. 

 

So if you don't have the disk space to do so... then duplicating it isn't really an option.

 

Also, a 1:1 mirror is a huge use of space, and some people aren't really fans of that, either.

 

 

 

But yes, duplicating everything makes recovery incredibly simple. Just remove the bad drive, and let StableBit DrivePool "do it's thing". And we're glad that you appreciate that!

[singatdcds]

Ah, I see. It's really just a matter of "do you have enough space". Cool! I certainly do so I am glad I'm not missing anything. DP plus online backup for the MOST critical stuff like family photos and movies is just about as safe and secure as it gets!!

 

 

Sent from my iPhone using Tapatalk

[Christopher (Drashna)]

Ah, I see. It's really just a matter of "do you have enough space". Cool! I certainly do so I am glad I'm not missing anything. DP plus online backup for the MOST critical stuff like family photos and movies is just about as safe and secure as it gets!!

 

 

Sent from my iPhone using Tapatalk

Yes, it's mostly a matter of having enough space to duplicate the data ... and then to "reduplicate" in the case of a failed drive.

 

Also, as I've mentioend before, the Seagate Archive drives work very well with StableBit DrivePool. Especially, if you mainly have videos or other files like that. Write performance isn't fantastic (but it's still pretty good actually!), but if it's a concern, then get the SSD Optimizer balancer and a couple of SSDs

And they're the best price per TB ratio on the market (reliably at least). 

 

 

 

And yes, having online backup is important as well.