thnz
-
Posts
139 -
Joined
-
Last visited
-
Days Won
7
Posts posted by thnz
-
-
I'm curios as to whether your machine ever crashed, or was restarted while CloudDrive was still uploading data? I've found that corruption seems prone to occur when this happens.
-
I would have thought Amazon would be a bit more keen to support you guys. With the way you guys have made intelligent local caching, I would have thought there would be far less network usage at their end (though maybe a higher overall file and request count) than other offerings that they do support.
It's bizarre they would revoke production status of an entire product due to the way a single user was using it. Seems rather disproportionate - throttle the user sure, but not the whole product. Are you sure that was their reasoning? Perhaps the default thread count of 10 was causing a higher load than it was built for, though I do see you've since lowered it to 2. Would be nice if they were more communicative with you guys. I've mentioned it before, but there is an amazon dev forum in which Amazon employees post that might be of use to you guys.
Seeing as I'm now locked in with Amazon for at least a yaer (I had purchased with the sole intention of using it with CloudDrive), I've started playing with acdcli for backups in order to get my moneys worth. It's quite nice seeing full 2MB/s uploads to Amazon now, though without the local caching reads aren't as fast (as its backup only its not so much of an issue), and its certainly not as ideal as CloudDrive will be once things finally get sorted.
-
After updating to .393 (from .379) I had two BSODs yesterday, though I also installed some Windows Updates too and am unsure if it was that or CloudDrive that triggered them. No drives were attached at the time of either crash. I'll upload the memory dump from the second BSOD in a bit - unfortunately it overwrote the dump of the first BSOD. The second one was a CLOCK_WATCHDOG_TIMEOUT - I didn't note what the first one was, though IIRC it was different.
*edit* The first BSOD was defiantly Windows Update related, and not CloudDrive, as I can reproduce it. The second one though I'm still not sure of.
-
I've seen some more corruption occur, though I'm unsure when and how. This time there was definitely no ungraceful restarts or crashes. This morning DrivePool was reporting inconsistent duplication, and removing the clouddrive from drivepool gave the following error:
DrivePool.Service.exe Warning 0 [RemoveDriveFromPool] Error scanning for and closing existing open handled to pool part. The file or directory is corrupted and unreadable 2015-09-25 22:51:04Z 152107466120 DrivePool.Service.exe Warning 0 [RemoveDriveFromPool] Error removing \\?\Volume{acaa6ea1-74c0-4651-ba49-56abf3959b7f}\PoolPart.42720267-d5e1-4ddc-bd63-6f9f8310234d\ from the pool. The file or directory is corrupted and unreadable 2015-09-25 22:51:06Z 152114294960The CloudDrive involved was created a few days ago and a single folder was duplicated to it. The system had been restarted a few times since, although it was always done gracefully. I believe that data was currently being uploaded at the times the computer was restarted, though aren't certain if this is relevant or not. I had received the same error after the non-graceful-restart corruption issue, though as all restarts were graceful this time I can't say if it was the same issue or not.
If I get time later, I'll try to reproduce it.
*edit* It could possibly have occurred after installing .379 over .378 and then restarting while data was uploading, though it would have been a normal, safe restart. I'm unsure of the order of events.
-
36 hours on and still no progress. It's either crazy throttling or something is going wrong.
Error: HTTP Status BadRequest
This error has occured 24,228 times.
-
Is this also throttling? Its been sitting like this all night with no progress.
Logfile is full of
CloudDrive.Service.exe Warning 0 [WholeChunkIoImplementation] Error on read when performing master partial write. HTTP Status BadRequest 2015-09-22 21:01:07Z 216674686527 CloudDrive.Service.exe Warning 0 [WholeChunkIoImplementation] Error on read when performing shared partial write. HTTP Status BadRequest 2015-09-22 21:01:07Z 216674806938 CloudDrive.Service.exe Warning 0 [WholeChunkIoImplementation] Error on read when performing shared partial write. HTTP Status BadRequest 2015-09-22 21:01:07Z 216674809296 CloudDrive.Service.exe Warning 0 [IoManager] Error performing I/O operation on provider. Retrying. HTTP Status BadRequest 2015-09-22 21:01:07Z 216674811931 CloudDrive.Service.exe Warning 0 [WholeChunkIoImplementation] Error on read when performing shared partial write. HTTP Status BadRequest 2015-09-22 21:01:07Z 216674814877 CloudDrive.Service.exe Warning 0 [WholeChunkIoImplementation] Error on read when performing shared partial write. HTTP Status BadRequest 2015-09-22 21:01:07Z 216674817146 CloudDrive.Service.exe Warning 0 [IoManager] Error performing I/O operation on provider. Retrying. HTTP Status BadRequest 2015-09-22 21:01:07Z 216674820551 CloudDrive.Service.exe Warning 0 [IoManager] Error performing I/O operation on provider. Retrying. HTTP Status BadRequest 2015-09-22 21:01:07Z 216674822976 CloudDrive.Service.exe Warning 0 [IoManager] Error performing I/O operation on provider. Retrying. HTTP Status BadRequest 2015-09-22 21:01:07Z 216674825401 CloudDrive.Service.exe Warning 0 [IoManager] Error performing I/O operation on provider. Retrying. HTTP Status BadRequest 2015-09-22 21:01:07Z 216674827225
Is a disk on Amazon CloudDrive.
-
FWIW AWS had a big outage earlier in the day, though is now apparently fixed - I'm not sure if this effected Amazon Cloud Drive, though as Amazon CloudDrive is throwing errors (even in a web browser) I assume it did (and still is).
-
Data loss happened when restarting (manual power cycle as the system was unresponsive) after the system crash in post #107 (which hasn't happened again since) although its very possible that the initial crash wasnt even clouddrive related. I then found that by manually hard restarting (ie restart button on case - a non graceful restart) I could reproduce the data loss on data that was previously 'written' to the cloud drive and was currently being uploaded. In this instance, data loss only occurred the once following a system crash and non-graceful restart - the rest was me manually non-gracefully restarting in order to reproduce it.
I understand that data loss can be expected on power loss, or a non graceful shutdown - especially on active writes - and thankfully they happen very rarely. However seeing as the data had previously been written a good day or two prior (although still uploading), I thought it reasonable for CloudDrive to be able to successfully recover and continue uploading from where it left off.
Also this might well be the same thing seen back in posts #19-#20.
Hopefully that clears things up
-
Data loss has a chance to occur if you manually hard restart the computer (or something else crashes it/lose power etc) while CloudDrive is uploading data. CloudDrive itself doesn't cause a crash or restart. Data loss doesn't happen every time.
-
It took several attempts, but I finally reproduced it again after lowering the local cache size to 20MB (though that could just be coincidence). 'Drive tracing' seems to have turned itself off after the hard restart, but hopefully it caught enough to be helpful. I've uploaded it via the form on that log collection page.
Quick summary of how I reproduced it:
- New 1GB unencrypted drive on DropBox with 20MB cache
- Copied ~700mb file across
- Hard reset after the file finished copying (ie. disk activity in resource monitor had finished), but still uploading (was maybe 50mb into the upload)
- File is now corrupt (has different hash) after drive recovers
Just want to add, that its times like this (constant restarting) that you really appreciate having an SSD. Reboots so fast!
-
It happened on a new Amazon Cloud Drive, created using .378. It also happens on other providers though - both encrypted and unencrypted - I've tested on DropBox with the same result - so isn't Amazon specific. It's much easier to reproduce by copying across a single large file, and hard restarting once its finished copying and is well into uploading - when uploading a group of files, it shows the corruption occurs in several places as it effects several files at once. It doesn't allways happen though - it might take a couple of hard restarts before the corruption occurs.
-
I had a crash last night, though I'm unsure if it was related to CloudDrive or not, although CloudDrive is the only thing to have caused any crashes on that machine all year. I've got a memory dump, though I didn't see any BSOD messages as the screen wouldn't turn on. Looking at the event log, the last thing to happen was a wake by Windows to run scheduled maintenance tasks. The event log also mentions queuing the update to Windows 10 (the machine is currently running 8.1), though I wouldn't have thought that would have started without user intervention first, so might just be incidental. I'll submit the dump in a bit, so hopefully that'll show whether its CloudDrive related or not.
To add to this, there appears to have been some data loss when this occurred, similar to what would happen after restarting from the I/O deadlock issue. A group of sequentially named files have become corrupted - ie. they were copied to the disk in order and I guess the corruption likely happened on the chunks involved. The chunks would have been being uploaded at the time of the crash.
*Edit* I can reproduce this on another machine. Simply copy a folder to the cloud disk, say a video folder or something with a good selection of reasonably sized files. Hard restart the computer while the data is uploading. After recovery, there will be a group of sequentially named corrupt files. They stand out as they fail to generate thumbnails, and when checked their hashes differ from the original files.
-
I'm assuming its either some kind of throttling, or CloudDrive erroneously aborting something that was still active as resource monitor shows plenty of network activity while CloudDrive shows very slow progress. It seems to show up regularly (~2m intervals) during active uploads when it happens. Perhaps its left over from the measures in place to counter throttling in v1?
That's my random, totally uninformed guesswork anyway!
-
I had a crash last night, though I'm unsure if it was related to CloudDrive or not, although CloudDrive is the only thing to have caused any crashes on that machine all year. I've got a memory dump, though I didn't see any BSOD messages as the screen wouldn't turn on. Looking at the event log, the last thing to happen was a wake by Windows to run scheduled maintenance tasks. The event log also mentions queuing the update to Windows 10 (the machine is currently running 8.1), though I wouldn't have thought that would have started without user intervention first, so might just be incidental. I'll submit the dump in a bit, so hopefully that'll show whether its CloudDrive related or not.
Also, after a few solid hours last night, it looks like the I/O errors are back again and speed has slowed right down.
-
Its sprung back to life over the past hour or two. The last I/O error was ~2hrs ago.
*edit* Has now sped up too.
-
Done. Looks like the logfiles are capped at 20mb each and have been outputting lots of stuff. Hopefully the important stuff hasn't been cut off. The i/o errors started maybe 24 hours ago.
-
I am yes. Want the log file?
-
It worked great for the first 24 hours, though checking on it now it's slowed right down and is throwing lots of 'Error: The request was aborted: the request was cancelled' I/O errors.
-
I'll give .378 a try later. Reading the changelog atm and it looks like you guys have been making some solid progress.
I'm curious as to whether the 'post processing' that Amazon is/was doing on unencrypted files, along with the not-a-true-404 could have been responsible for the 'disappearing files' issue that you were having with them.
*edit* It's working great. Could definatley get used to these speeds!
-
Just to clarify - the upload stall I thought I saw was just bad maths on my part, and those 'error write range' warnings occured when I destroyed an old drive, so I assume are nothing of importance.
The log shows the following when refreshing the list of unatached disks.
CloudDrive.Service.exe Warning 0 [RefreshProviderCredential] Error retrieving metadata from cloud drive '131bd8b7-3e83-4c55-8692-6492c7974588'. HTTP Status NotFound 2015-09-02 02:15:17Z 9411891508 CloudDrive.Service.exe Warning 0 [RefreshProviderCredential] Error retrieving metadata from cloud drive '55a11e0e-2b21-4fb3-9eb4-291c12f28437'. HTTP Status NotFound 2015-09-02 02:15:18Z 9416008001 CloudDrive.Service.exe Warning 0 [RefreshProviderCredential] Error retrieving attachment info from cloud drive '55a11e0e-2b21-4fb3-9eb4-291c12f28437'. HTTP Status NotFound 2015-09-02 02:15:19Z 9418497390
Will revert to .371 in the meantime.
-
Doing a clean install stopped the auth errors, though detached drives still appear as 'destroyed'. Just to make thing even more complicated, its been throwing some file verification inconsistency fail errors too. And also uploads stalling after a while with lots of:
CloudDrive.Service.exe Warning 0 [IoManager] Error writing range: Offset=13,792,968,704. Length=1,048,576. Cannot access a disposed object.
in the log.
-
Grab build 373 if you want to see the difference.
Don't mind if I do! Looks like its been a good past week all round for you guys.
Would you guys be able to investigate if upload verification is still required? Being able to half traffic usage would be great.
*edit* I recieved some red authentication errors earlier, which reappeared again straight after re-authenticating. The red errors have now stopped, but now my Amazon Cloud Drive provider appears 'red' with the mouseover text saying 'Cant connect to Amazon Cloud Drive. Security error' when adding a new drive. However it still appears to be uploading alright to the currently connected drive.
On another machine, after a fresh install, existing cloud disks now appear as 'destroyed drive'
-
You could always try pinging them on their clouddrive dev forums if you've reached a brick wall.
-
And yeah, Amazon is throttling the connection pretty bad, but that shouldn't' be an issue after we get approved for production use... whenever that happens....
And any data that fails to upload is retried. And specifically, for Amazon Cloud Drive, we use "Upload verification", which redownloads the data after it's been uploaded to make sure that the upload process occurred properly. And once that's verified, then we may prune that data from the cache.
Have you guys heard back from Amazon at all since emailing them about getting approved?
Possible bug with VMWare Player
in General
Posted
Done. FWIW I used VMWare Player 12 this time with the same result.