thnz

Still running smoothly after 24 hours. It's looking like this issue could well be resolved. It's thrown a few red 'error downloading from Amazon Cloud Drive' errors - throttling I guess - although the message mentions possible data stability issues if it keeps occuring. How will DrivePool respsond to this kind of thing? If the internet connection goes down, will DrivePool handle read errors gracefully? As its all duplicated data I would hope so. I'm assuming reads would be done with the local drive copy, and writes would be cached for later upload.

Disk grouping sounds ideal. Previously I found the i/o deadlock seemed to kick in after maybe 5mins or so, so having gone several hours without issue is certainly promising. Fingers crossed I don't wake up to a BSOD tomorrow morning! Will keep you guys updated.

It's been duplicating all afternoon without any crashes, so its looking promising. Are there any plans for further integration with DrivePool? For instance, to force duplications to the cloud disk? At the moment it needs a fair bit of micromanagement, individually assigning rules to folders as to which disk they should be on. When theres more than two disks in the pool, its a bit fiddly forcing them on to the cloud disk.

That's great news. I'll grab that build later and put it through its paces - will do my best to crash it! Just need to pass the Amazon review process then, and I'll be able to ditch CrashPlan for DrivePoolDuplication+CloudDrive.

It BSOD'd twice since .352. The first time I didn't catch the message as it had restarted itself, and I didn't keep the dump. It happened during drivepool duplication (so, heavy i/o). The second time was a clock_watchdog_timeout - I'm not sure under what circumstances it happend (i/o etc.), as I had just came back to the machine and it was sitting on a BSOD (I had disabled auto-restart on crash after the first one so I could see what it was). I'm compressing the second dump atm and will upload it shortly.

I took .352 for brief spin and am getting clock_watchdog_timeout BSODs.

I might wait for a later build then before restesting. Good luck!

It's great that you guys have (hopefully!) reproduced it. I see a note in the changelog mentioning deadlock resolution code. Is that for this issue? If so, is it worth testing yet?

I've been fiddling around this afternoon and I can now semi-reliably reproduce it. If it doesn't lock up first time, then just keep doing it until it does. I don't think using drivepool is relevant, but its what I've used, and it does induce plenty of i/o. 1. Created a new single disk drive pool. 2. Copied ~20GB worth of files across (10K files - mix of large and small) 3. Created a cloud drive. I've had it crashing on Amazon Cloud Drives, File Shares, and Local Disks. I'm not sure if the provider is relevant, but I've used a 1GB cache / 100GB drive for my tests. Enabled full drive encryption (again, not sure if relevant). 4. Added the cloud drive to the drive pool 5. Enabled duplication on everything. 6. Clicked the 'increase priority' button - not sure if this is relevant, but I'd it was clicked for the last 3 crashes. Then cross your fingers and hope it locks up. If not, remove the cloud drive from the pool, and re-add it to restart the duplication process. This has caused a lock up the past 3/5 times I've tried. Give it a go and see if you guys can reproduce it. *edit* Full drive encryption isn't relevant, nor is local cache size. It still locked up with a local disk/no cache/no encryption on the second attempt at duplicating. Task manager (I couldn't open it in Windows 8.1 once its locked up, but can in 10) shows the cloud drive as 100% active time, 0b/s read, 0b/s write once its locked up.

Windows Memory test thing reports no errors. I've just ran memtest86 too just to be certain, and that hasn't found anything wrong either. I've had the lockup on two separate machines, so I'd be surprised if it was a hardware issue. I've only seen it occur during cloud disk i/o - it's possible something was accessing the disk when it crashed this morning too, though as it was throwing constant red auth errors it could have been something else. It went several weeks without occurring when the disk was just sitting there attached but idle. The last 3 times I've tried adding the cloud disk to drivepool, its locked up within 5-10mins of duplication beginning. *edit* I just reproduced it on another machine. Made a new drivepool+cloud disk on it. Copied ~20GB of files to the drivepool disk, then added the clouddisk to the pool. Locked up at about 80% duplication. Was still able to move open windows around just like the lock up on the other machine, but wasn't able to do much else.

FWIW .336 has fixed the red auth errors. Thanks for explaining the situation with Amazon Cloud Drive. Hopefully Amazon don't take too long in reviewing your application. If anyone else is curious, there is a bit more info here. Is there a possibility that once this is sorted, upload verification will no longer be needed? Back on topic, I had another lockup this morning. There wasn't any specific read/write activity happening on the disk at the time that I was aware of, although it had been throwing those red errors all night.

After updating to .332, and then .333, Im getting red 'is not authorized with Amazon Cloud Drive' messages. Clicking 'reauthorize' takes me to the cloudbit site with no access key to copy. 'An+unknown+scope+was+requested' appears in the URL.

Had no effect - it still locked up within a few minutes of beginning duplication in drivepool. Will upload both yesterday's and today's dumps in a bit.

It's reporting version 1.0.0.330 BETA in the UI and 1.0.330 in control panel's 'programs and features'. C:\WINDOWS\system32>fltmc Filter Name Num Instances Altitude Frame ------------------------------ ------------- ------------ ----- eamonm 8 328700 0 luafv 1 135000 0 npsvctrig 1 46000 0 FileInfo 8 45000 0 Wof 0 40700 0 Shall I upload today's dump too? Also, would logfiles help at all?

Done. FWIW none of the file modified times changed after doing this, so I assume it had previously updated successfully. Will try my best to crash it again! *edit* It just locked up - again while drivepool was duplicating to the clouddrive. Done a manual dump again and can upload it if it'll help.

Just had the 'deadlock' lockup again. Happened while drivepool was duplicating files to the clouddrive. Will upload a dump shortly. *edit* Uploaded here. Drive is doing recovery right now.

Great to see progress on this. Unfortunately I'm still unable to reattach drives in this build (bug in other thread), so can't really give it a proper testing yet.

I found after initially installing .321 on top of .307 that the GUI was still reporting it as .307. I then uninstalled and reinstalled to get it up to .321. I found that doing a complete uninstall and deleting the programdata subdir, and then reinstalling fixed the 'full drive encryption' bug, in that it was then selectable. However as drives still couldn't be reattached, I've downgraded back to .307 for the time being.

Deleting the cloudpart.xxxx folder had no effect - it was still unable to reattach. IIRC there was no error displayed. I've uploaded the log folder here. I can't remember the exact times I was fiddling around, but it should be somewhere in yesterday's log (Aug 6). FWIW both problems happened on two different computers - both upgrading from .307 to .321. It's good that you can reproduce them. I see that in .320 a fix was made for a 'deadlock with the cache manager'. Is this the same problem from the other thread? I see that issue #17719 is still open.

Upgrading from 1.0.0.307. When creating a new cloud disk, I'm unable to select 'full drive encryption' in 1.0.0.321. When checking the check box nothing happens. The 'settings' button becomes clickable, but clicking that also does nothing. I'm also unable to reattach another disk. It says it attached successfully, but I never get a prompt to unlock it. I can click 'attach' again, but then it warns that its already attached to another (THIS) computer. When attaching, and clicking 'advanced settings' it lists the drive as having a 0b sector size (not sure if this is relevant). Downgrading back to 1.0.0.307 and everything works again.

Do you know if data loss/corruption can occur when this happens? I copied several hundred 500mb-1gb files onto a cloud disk, and during upload this deadlock thing happened twice. Once uploading had finished, several of the files became corrupted (they had different file hashes than the originals). The corrupted files seemed to be in two distinct clusters when sorted by file modified time (ie. they were copied sequentially). I'm wondering if chunks of these corrupted files were being uploaded at the time of the deadlock, and weren't caught by recovery after a restart. Is it possible that chunks being uploaded when this crash occurs can become corrupted?

A shame it turned out to be so complex. Great to know that its being worked on though. Good luck!

Sure. Create a new cloud drive (I'm using Amazon Cloud Drive) Leave everything as default (10GB drive etc). I enabled encryption, though not sure if this is relevant. Format drive + assign drive letter Create a new VM in VMWare Player (might be slightly different in Workstation or other VMWare stuff) Select - install OS later Select Linux / Ubuntu x64 (not sure if this is relevant, but its what I selected) Set VM location - set to a local physical disk Use default disk settings, then click Finish Edit the VM settings and remove the HDD Add - Hard Disk - SCSI - Use a physical disk Select the virtual drive in the dropdown - name correlates to disk # in disk management Either (a) Choose 'use individual partitions'. Assuming the virtual drive was formatted in step 2, no partitions will be listed - its expected that a single 10GB NTFS partition should be there. Or ( Choose to use the entire disk, then save the .vmdk. The disk will appear as 1.3GB in the VM settings rather than 10GB as expected. Also, if the drive is set to 20GB in step 2, it will appear as a 2.5GB drive.

I don't plan on using a CloudDrive disk in this way - too many things to go wrong - but it does appear that the virtual disk is seen differently to a physical disk by VMware Player. Not sure if this is a Player issue or CloudDrive issue. * Create a virtual disk in CloudDrive (say 20GB), but don't format * Create new VM in Vmware Player * Add the CloudDrive disk as an exisiting physical disk to the new VM * The disk appears as a 2.5GB disk Also when adding a pre-partitioned and formatted CloudDrive as a physical disk, existing partitions aren't listed like they would be on a real disk. Possibly relevant stuff: Windows 8.1 x64 VMWare Player 7.1.2

Have you guys found the problem yet? Would more dumps help, or is it now a matter of implementing a fix?