Jump to content

  • Log in with Twitter Log in with Windows Live Log In with Google      Sign In   
  • Create Account

Photo

Amazon Cloud Drive - Why is it not supported?


  • Please log in to reply
196 replies to this topic

#21 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,208 posts
  • LocationSan Diego, CA, USA

Posted 05 December 2015 - 08:13 PM

I realize that this is mostly out of your control, but as far as timeframe for exiting "development status"... are we thinking that it could be any day now? Or is it likely it may be months away, or never?

Well, as it took 2 weeks and several emails originally ..... the answer is "I don't know". Sorry. :(
 
We've had much better communication with Amazon lately (eg, they're actually replying).
 
In fact, right now, the emails are about the exact API usage and clearing up some functionality, between Alex and Amazon.  So, things are looking good.   
 
 If you check the first post, Alex has updated it with the latest responses.


  • WBadminUser, Talyrius, Ginoliggime and 2 others like this

Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#22 Talyrius

Talyrius

    Member

  • Members
  • PipPip
  • 22 posts
  • LocationPortland, OR, US

Posted 12 December 2015 - 05:46 AM

I have no idea what they mean regarding the encrypted data size. AES-CBC is a 1:1 encryption scheme. Some number of bytes go in and the same exact number of bytes come out, encrypted. We do have some minor overhead for the checksum / authentication signature at the end of every 1 MB unit of data, but that's at most 64 bytes per 1 MB when using HMAC-SHA512 (which is 0.006 % overhead). You can easily verify this by creating an encrypted cloud drive of some size, filling it up to capacity, and then checking how much data is used on the server.

Are chunks fully or partially allocated upon creation? If they're fully allocated, this could explain Amazon's confusion.

#23 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,208 posts
  • LocationSan Diego, CA, USA

Posted 12 December 2015 - 09:15 PM

Are chunks fully or partially allocated upon creation? If they're fully allocated, this could explain Amazon's confusion.

They are not allocated upon creation.  

I double checked on my Amazon Cloud Drive account, I have a 50TB drive on it, but only using about 50GBs of space right now.  We we're definitely not pre-allocating. 

 

Basically, unless Amazon's servers are doing something to the files on their side... we're not really sure what's going on.  All the providers use the same framework, and work in pretty much exactly the say way (there are slight differences between each provider due to specific requirements).  

 

 

Though, it could be that they're trying to decompress files that look like Archives (statistically speaking, it's going to happen).  Aside from that... without more details, it's impossible to know what Amazon is seeing, specifically. 


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#24 Talyrius

Talyrius

    Member

  • Members
  • PipPip
  • 22 posts
  • LocationPortland, OR, US

Posted 30 December 2015 - 02:50 PM

Any news?

#25 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,208 posts
  • LocationSan Diego, CA, USA

Posted 30 December 2015 - 08:11 PM

From our end, not really.  Given the holiday seasons, it is to be expected.

 

However, while I can't comment specifically on some of the new replies, I suspect that the latest reply (about the "already exists under" errors) have to do with the "missing" files we see and re-upload. But I'm not sure.

 

There is definitely ongoing dialog, but progress is slow. But at least they're replying!


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#26 steffenmand

steffenmand

    Advanced Member

  • Members
  • PipPipPip
  • 296 posts

Posted 08 January 2016 - 04:50 PM

Got any good replies ? :) Are we maybe even able to avoid upload verification looking forward ? :)



#27 mentalinc

mentalinc

    Member

  • Members
  • PipPip
  • 13 posts

Posted 09 January 2016 - 12:51 AM

Could they look to add a checksum verification on Amazon end, this would save everyone a pile of bandwidth, but add a pile of compute time on Amazon end I guess.

 

1. Compute chunk checksum 

2. Upload chunk

3. Run Amazon checksum API

4. Receive Amazon generate checksum (will be only a few bytes depending on MD5 or SHA1 etc)

5 compare checksums from step 1 & 4. If same move to next file, if cehcksum is  different, error restart at step 1 for current file



#28 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,208 posts
  • LocationSan Diego, CA, USA

Posted 09 January 2016 - 04:36 AM

Could they look to add a checksum verification on Amazon end, this would save everyone a pile of bandwidth, but add a pile of compute time on Amazon end I guess.

 

1. Compute chunk checksum 

2. Upload chunk

3. Run Amazon checksum API

4. Receive Amazon generate checksum (will be only a few bytes depending on MD5 or SHA1 etc)

5 compare checksums from step 1 & 4. If same move to next file, if cehcksum is  different, error restart at step 1 for current file

There is a bunch of stuff that Amazon should be doing on their end. Starting with properly throttling API calls instead of punishing us for using too many calls. 

 

If that sounds a bit bitter .... It is. And it's meant to be an example of how poorly Amazon has handled their entire Cloud Drive service. Not just the API. 

 

Basically the entire backend is poorly designed and managed, and the Amazon Cloud Drive team really needs to call the S3 team and step up their game. 


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#29 WBadminUser

WBadminUser

    Advanced Member

  • Members
  • PipPipPip
  • 82 posts

Posted 09 January 2016 - 07:54 PM

There is a bunch of stuff that Amazon should be doing on their end. Starting with properly throttling API calls instead of punishing us for using too many calls. 

 

If that sounds a bit bitter .... It is. And it's meant to be an example of how poorly Amazon has handled their entire Cloud Drive service. Not just the API. 

 

Basically the entire backend is poorly designed and managed, and the Amazon Cloud Drive team really needs to call the S3 team and step up their game. 

What is the harm in giving us a few more threads for amazon? I used the old build that didnt have the 2 thread cap and it worked great. However you guys kept adding new features that eventually i said ok i gotta upgrade. I am as made at amazon as everyone else but really if i just had a few more threads problem solved for me.



#30 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,208 posts
  • LocationSan Diego, CA, USA

Posted 09 January 2016 - 09:40 PM

What is the harm in giving us a few more threads for amazon? I used the old build that didnt have the 2 thread cap and it worked great. However you guys kept adding new features that eventually i said ok i gotta upgrade. I am as made at amazon as everyone else but really if i just had a few more threads problem solved for me.

Right now? We're stuck in development status, that means the more active connections from our software, the more the entire product gets throttled.  More threads means more connections, which means a worse experience for everyone.  As in, unusable.  And that includes for Amazon (who needs to test our program, now).


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#31 WBadminUser

WBadminUser

    Advanced Member

  • Members
  • PipPipPip
  • 82 posts

Posted 09 January 2016 - 10:11 PM

Right now? We're stuck in development status, that means the more active connections from our software, the more the entire product gets throttled.  More threads means more connections, which means a worse experience for everyone.  As in, unusable.  And that includes for Amazon (who needs to test our program, now).

How are all these other tools that are popping up on github that access ACD not having the same issue? for example https://github.com/yadayada/acd_cli

*EDIT*

Actually upon reading a bit into it i think they are approved.

Just baffled as why the hell you guys cant get approved.



#32 thnz

thnz

    Advanced Member

  • Members
  • PipPipPip
  • 139 posts

Posted 09 January 2016 - 10:22 PM

How are all these other tools that are popping up on github that access ACD not having the same issue? for example https://github.com/yadayada/acd_cli

 

They're probably running in production status.

 

Perhaps the 'chunky' nature of clouddrive makes it stand out with high calls/sec, even though the intelligent caching side of it will cause less data to be transferred. Storing a 100mb file in 1mb chunks could mean 100 upload requests rather than just 1. Also the upload verification would cause a double up in data used when uploading, though IIRC this is necessary as files sometimes 'disappear' unexpectedly at Amazon's end.



#33 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,208 posts
  • LocationSan Diego, CA, USA

Posted 10 January 2016 - 03:52 AM

That's why....

They're cheating... Heavily.

 

 

https://github.com/y...thorization.rst

 

Specifically, they're requiring you to sign up for an account as a developer, and then input that info in, and use it. Meaning that each and every user is using a different developer account. 

Something you're *not* supposed to do. You're supposed to use a single developer account for your entire program.

 

In fact, this may be part of why Amazon has been a nightmare about approval.... because they have hundreds or thousands of developer accounts being used by people using ACD_CLI. 


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#34 WBadminUser

WBadminUser

    Advanced Member

  • Members
  • PipPipPip
  • 82 posts

Posted 10 January 2016 - 04:05 AM

That's why....
They're cheating... Heavily.


https://github.com/y...thorization.rst

Specifically, they're requiring you to sign up for an account as a developer, and then input that info in, and use it. Meaning that each and every user is using a different developer account.
Something you're *not* supposed to do. You're supposed to use a single developer account for your entire program.

In fact, this may be part of why Amazon has been a nightmare about approval.... because they have hundreds or thousands of developer accounts being used by people using ACD_CLI.

Can't beat em, join em.

Sent from my Nexus 6P using Tapatalk

#35 thnz

thnz

    Advanced Member

  • Members
  • PipPipPip
  • 139 posts

Posted 10 January 2016 - 04:08 AM

 

Is the first/default method not the same as what you guys are doing?



#36 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,208 posts
  • LocationSan Diego, CA, USA

Posted 10 January 2016 - 04:25 AM

Is the first/default method not the same as what you guys are doing?

This first method is what we're doing. 

 

But the offer the option to use your own security account.

 

Can't beat em, join em.

Sent from my Nexus 6P using Tapatalk

 

We've talked about it. Extensively.

 

However, the issue isn't just the production status. The entire service has some pretty fundamental issues, as well. Which we've reported to Amazon.  


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#37 WBadminUser

WBadminUser

    Advanced Member

  • Members
  • PipPipPip
  • 82 posts

Posted 10 January 2016 - 07:03 PM

This first method is what we're doing. 

 

But the offer the option to use your own security account.

 

 

We've talked about it. Extensively.

 

However, the issue isn't just the production status. The entire service has some pretty fundamental issues, as well. Which we've reported to Amazon.  

 

Well i might try to sign up for a google for work account.

Tired of amazon. Its weird normally their customer service is top notch.



#38 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,208 posts
  • LocationSan Diego, CA, USA

Posted 12 January 2016 - 06:39 AM

Well i might try to sign up for a google for work account.

Tired of amazon. Its weird normally their customer service is top notch.

May be a good idea for the meanwhile.

 

And yes, it is. But then again, S3 is also very top notch, in regards to API and documentation. So it really seems that this is a COMPLETELY different team, and they have absolutely no interaction with the S3 team (which is a shame, and ... troubling IMO). 


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#39 evulhotdog

evulhotdog

    Newbie

  • Members
  • Pip
  • 7 posts

Posted 16 January 2016 - 12:07 AM

May be a good idea for the meanwhile.

 

And yes, it is. But then again, S3 is also very top notch, in regards to API and documentation. So it really seems that this is a COMPLETELY different team, and they have absolutely no interaction with the S3 team (which is a shame, and ... troubling IMO). 

Well i might try to sign up for a google for work account.

Tired of amazon. Its weird normally their customer service is top notch.

 

Do you think that Google Drive can support everything, if not work much better than the current state of AWS? Particularly for media streaming?



#40 Talyrius

Talyrius

    Member

  • Members
  • PipPip
  • 22 posts
  • LocationPortland, OR, US

Posted 16 January 2016 - 12:21 AM

I hope some progress is made soon on getting Amazon Cloud Drive approved for production.

In the meantime, does anyone know of a way to sign up for a Google Apps Unlimited account without having to pay for four other users? An organization requires five users to qualify and Google Apps Unlimited is $10 per user per month.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users