Google Drive: The limit for this folder's number of children (files and folders) has been exceeded

[KiLLeRRaT 6]

Hi All,

I've recently noticed that my drive has 40GB to be uploaded, and saw there are errors in the top left.  Clicking that showed 

Quote

The limit for this folder's number of children (files and folders) has been exceeded

What's the deal here, and is this going to be an ongoing issue with growing drives?

My drive is a 15TB drive, with 6TB free (so only using 9TB).

Attached is my drive's info screen.

EDIT: Cloud Drive Version: 1.1.5.1249 on Windows Server 2016.

 

Cheers,

[JulesTop 4]

2 hours ago, KiLLeRRaT said:

Hi All,

I've recently noticed that my drive has 40GB to be uploaded, and saw there are errors in the top left.  Clicking that showed 

What's the deal here, and is this going to be an ongoing issue with growing drives?

My drive is a 15TB drive, with 6TB free (so only using 9TB).

Attached is my drive's info screen.

EDIT: Cloud Drive Version: 1.1.5.1249 on Windows Server 2016.

 

Cheers,

I just started getting the same thing today. Based on the coincidence of both of us having the issue in the same day, this may be a Google issue.

[kird 0]

I know Googledrive has a file and folder limit, but now I don't know what the number is. In my case I'm not having, for now, this error but due to the large amount of folders and files I have on my drive I don't think I'll have much left to overcome it.  Good news that our DriveCloud will warn us about that error.

[JulesTop 4]

18 hours ago, kird said:

I know Googledrive has a file and folder limit, but now I don't know what the number is. In my case I'm not having, for now, this error but due to the large amount of folders and files I have on my drive I don't think I'll have much left to overcome it.  Good news that our DriveCloud will warn us about that error.

I doubt I hit my file and folder limit. The reason I say that is that I deleted a few folders and their files, and nothing changed. I still get this error and my uploads are hanging.

I'm wondering if this is happening to anyone else...

[kird 0]

4 hours ago, JulesTop said:

I doubt I hit my file and folder limit. The reason I say that is that I deleted a few folders and their files, and nothing changed. I still get this error and my uploads are hanging.

I'm wondering if this is happening to anyone else...

Figuring it out, have you deleted the files from the recycle bin? If you have not configured it in a special way, by default the deleted files go there and if they are not deleted they still count for space.

[JulesTop 4]

6 hours ago, kird said:

Figuring it out, have you deleted the files from the recycle bin? If you have not configured it in a special way, by default the deleted files go there and if they are not deleted they still count for space.

My windows recycling bin? I went there and it is empty. Just in case, I just changed the settings for my cloud drive to permanently be deleted without being stored in the recycling bin.

I also went to my actual Google drive we interface, however, the bin is empty. I'm still having the issue... It's weird, I deleted a bunch of files and still have the issue. I have it setup with 20MB chunks... So my drive would likely have to be humongous to reach the file limit.

[kird 0]

Yes, i was referring logically to the recycle bin of your gdrive account, the windows one here plays no role. As I understand it here the point is not the amount of Gb/Tb in files on the drive but the total number of folders that can be nested in the account.

If I understand correctly, according to this information the error occurs when it reaches 500.000 folders, files... inside a root folder in the drive.

"A numChildrenInNonRootLimitExceeded error occurs when the limit for a folder's number of children (folders, files, and shortcuts) has been exceeded. There is a 500,000 item limit for folders, files, and shortcuts directly in a folder. Items nested in subfolders do not count against this 500,000 item limit."

[JulesTop 4]

6 hours ago, kird said:

Yes, i was referring logically to the recycle bin of your gdrive account, the windows one here plays no role. As I understand it here the point is not the amount of Gb/Tb in files on the drive but the total number of folders that can be nested in the account.

If I understand correctly, according to this information the error occurs when it reaches 500.000 folders, files... inside a root folder in the drive.

"A numChildrenInNonRootLimitExceeded error occurs when the limit for a folder's number of children (folders, files, and shortcuts) has been exceeded. There is a 500,000 item limit for folders, files, and shortcuts directly in a folder. Items nested in subfolders do not count against this 500,000 item limit."

In that case... there is definitely something wrong... I don't believe I should be anywhere near that as Stabelbit is the only thing that manages my google drive. You can see in the attachment that my root directory only has one folder, and deeper to that, it's all managed by stablebit clouddrive.

 

[kird 0]

From the numbers shown there, it doesn't seem to be your case at all. This can be best traced from the unit mounted on your PC and looking at your Gdrive's statistics. 

Let's see if the developers can shed some light on the subject.

[JulesTop 4]

3 hours ago, kird said:

From the numbers shown there, it doesn't seem to be your case at all. This can be best traced from the unit mounted on your PC and looking at your Gdrive's statistics. 

Let's see if the developers can shed some light on the subject.

I'm not sure if this is helpful...

[kird 0]

That information gives you the error we talked about but it doesn't show anything of the total files/folders contained in a root folder. At least I can't see it. 

When I say look at your locally mounted unit (it's decrypted) I mean this. As you can see in the image it's still far away from the 500,000

 

 

[JulesTop 4]

1 hour ago, kird said:

That information gives you the error we talked about but it doesn't show anything of the total files/folders contained in a root folder. At least I can't see it. 

When I say look at your locally mounted unit (it's decrypted) I mean this. As you can see in the image it's still far away from the 500,000

 

 

BTW, I really appreciate the help. Here it is.

I wonder if there is a way of counting the number of chunks in google drive... since that is really what adds up

[kird 0]

In your image I don't see any appreciation for having exceeded the limit. I wouldn't know what else to tell you. Let's wait and see what the developers say, it's a error which must be verified where it can come from and it's basic to know. I guess now your gdrive is not operational, right?

[JulesTop 4]

Just now, kird said:

In your image I don't see any appreciation for having exceeded the limit. I wouldn't know what else to tell you. Let's wait and see what the developers say, it's a error which must be verified where it can come from and it's basic to know. I guess now your gdrive is not operational, right?

I can download from the gdrive without issue. I can also upload to the root of the gdrive without problem (manually). When I tried adding a file to the clouddrive content folder manually (via the gdrive web interface), it fails.

It looks like there is an issue with the content folder... Maybe @Alex or @Christopher (Drashna) can help.

[kird 0]

3 minutes ago, JulesTop said:

When I tried adding a file to the clouddrive content folder manually (via the gdrive web interface)

I have never been able to download or upload anything via gdrive web, all content is encrypted. How do you do it? In my case to upload/download content always through the mounted network drive or Air Explorer (great for managing clouds) but never from the google web interface.

Yup, hopefully they can fix this nasty error or give information.

 

[JulesTop 4]

So the issue has been "fixed". There must be a bug or something, but not sure if it's on Google's side or stablebit.

I deleted the file that I figured it was stuck trying to upload. The file I deleted was about 20GB, and there was over 65GB in queue to upload. As soon as I deleted the 20GB file from my computer (which was on the clouddrive), my upload queue seemed to just resume. It didn't drop by 20GB... It just resumed.

 

Anyway, everything seems to be totally back to normal.

[kird 0]

I'm glad you were able to fix that nasty error on your own. By the way, which version of CloudDrive are you using? I can tell you that in my case there is a version that I keep as a real gem because it is like the "holy grail" for all kinds of errors that the drive may involve. This version is v1.1.0.1051

[JulesTop 4]

On 6/20/2020 at 2:12 PM, kird said:

I'm glad you were able to fix that nasty error on your own. By the way, which version of CloudDrive are you using? I can tell you that in my case there is a version that I keep as a real gem because it is like the "holy grail" for all kinds of errors that the drive may involve. This version is v1.1.0.1051

I was on the latest stable (I think the last 4 digits we're 1249) when this occured. After a couple of days, I updated to the latest Beta to try and solve it.

Also, I realized that at some point I removed my personal gdrive API credentials from the config file, and this is about the time the issue resolved. I just put back my personal API credentials and re-authorized and the issue came back. I yet again switched back to 'null' (the default stablebit API credentials) and the issue went away again... I wonder if there are higher API limits with the Stablebit credentials.

 

Maybe @Christopher (Drashna) can shed some light.

[kird 0]

I don't have my api configured and I have never met that error, maybe these google guys have touched again something related to the API causing the program to fail. Probably because of what you're saying. Looks like the developers are on vacation.

[Jellepepe 4]

This issue appeared for me over 2 weeks ago (yay me) and it seems to be a gradual rollout.

The 500.000 items limit does make sense, as clouddrive stores your files in (up to) 20mb chunks on the provider, and thus the issue should appear somewhere around the 8-10tb mark depending on file duplication settings. 
In this case, the error actually says 'NonRoot', thus they mean any folder, which apparently can now only have a maximum of 500.000 children.

I've actually been in contact with Christopher since over 2 weeks ago about this issue, and he has informed Alex of the issue and they have confirmed they are working on a solution. 
(other providers already has such issues, and thus it will likely revolve around storing chunks in subfolders with ~100.000 chunks per folder, or something similar.)

It is very interesting that you mention reverting to the non-personal api keys resolved it for you, which does suggest that indeed it may be dependent on some (secret) api level limit.
Though, that would not explain why adding something to the folder manually also fails...

@JulesTop Have you tried uploading manually since the issue 'disappeared' ? If that still fails, it would confirm that the clouddrive api keys are what are 'overriding' the limit, and any personal access, including web, is limited.

Either way hopefully there is a definitive fix coming soon, perhaps until then using the default keys is an option.

 

[JulesTop 4]

1 hour ago, Jellepepe said:

This issue appeared for me over 2 weeks ago (yay me) and it seems to be a gradual rollout.

The 500.000 items limit does make sense, as clouddrive stores your files in (up to) 20mb chunks on the provider, and thus the issue should appear somewhere around the 8-10tb mark depending on file duplication settings. 
In this case, the error actually says 'NonRoot', thus they mean any folder, which apparently can now only have a maximum of 500.000 children.

I've actually been in contact with Christopher since over 2 weeks ago about this issue, and he has informed Alex of the issue and they have confirmed they are working on a solution. 
(other providers already has such issues, and thus it will likely revolve around storing chunks in subfolders with ~100.000 chunks per folder, or something similar.)

It is very interesting that you mention reverting to the non-personal api keys resolved it for you, which does suggest that indeed it may be dependent on some (secret) api level limit.
Though, that would not explain why adding something to the folder manually also fails...

@JulesTop Have you tried uploading manually since the issue 'disappeared' ? If that still fails, it would confirm that the cloudflare api keys are what are 'overriding' the limit, and any personal access, including web, is limited.

Either way hopefully there is a definitive fix coming soon, perhaps until then using the default keys is an option.

 

You called it. I just tried uploading a small file to the content folder directly from the Google drive web interface and got an 'upload failure (38)', same as before.

It looks like this restriction rollout is definitely happening. It will be fantastic if @Alex has a solution coming soon! But also good that the default keys are keeping us going for now.

[steffenmand 60]

Maybe this is related to the issues we have with Internal Error etc. if we are already above the 10 TB mark. Then it fails when trying to write new chunks because we are already above the 10 TB mark

[kird 0]

4 hours ago, steffenmand said:

Maybe this is related to the issues we have with Internal Error etc. if we are already above the 10 TB mark. Then it fails when trying to write new chunks because we are already above the 10 TB mark

So far, with google's own api, I haven't had that kind of error and my files take up more than 100 TB

[Jellepepe 4]

4 hours ago, steffenmand said:

Maybe this is related to the issues we have with Internal Error etc. if we are already above the 10 TB mark. Then it fails when trying to write new chunks because we are already above the 10 TB mark

I also doubt this is related to this issue. I have well over 10TB used and i have never had a 'internal error' error. 

The only errors i have seen are user rate limit exceeded (when exceeding the upload or download limits) and since about 2 weeks this exceeded maximum number of children error.

[Christopher (Drashna) 606]

Just a heads up, this is a known issue, and we are looking into a solution for this issue (both to prevent it for new drives and for existing drives).

I don't have an ETA, but we are actively working on it. 

 

[Christopher (Drashna) 606]

On 6/21/2020 at 11:58 AM, JulesTop said:

Also, I realized that at some point I removed my personal gdrive API credentials from the config file, and this is about the time the issue resolved. I just put back my personal API credentials and re-authorized and the issue came back. I yet again switched back to 'null' (the default stablebit API credentials) and the issue went away again... I wonder if there are higher API limits with the Stablebit credentials.

If that is the case, this makes things easier to fix, at least. 

And it's very possible that our API key is treated differently. However, it's something we'd rather not assume, just in case.