Google Drive: The limit for this folder's number of children (files and folders) has been exceeded

[KiLLeRRaT 6]

Hi All,

I've recently noticed that my drive has 40GB to be uploaded, and saw there are errors in the top left.  Clicking that showed 

Quote

The limit for this folder's number of children (files and folders) has been exceeded

What's the deal here, and is this going to be an ongoing issue with growing drives?

My drive is a 15TB drive, with 6TB free (so only using 9TB).

Attached is my drive's info screen.

EDIT: Cloud Drive Version: 1.1.5.1249 on Windows Server 2016.

 

Cheers,

[steffenmand 60]

On 7/7/2020 at 10:04 PM, Chase said:

All of my drives are working and I have no errors...EXCEPT the last beta that changed the GoogleDrive_ConcurrentRequestCount to "2" appears to be causing issues. My drives tend to do a lot of reading and writing...it's for a plex server. And 2 concurrent causes CloudDrive to tell me I don't have enough bandwidth. I have tried to change the setting in the "settings.json" but it isn't accepting the change.

 

 

The newest beta works great for me - i still have a great speeds and the drives are upgrading fine (25% per day)!

 

Only annoying bug i have atm is that if i try to attach a new drive, then it will fail and give me a folder i cant delete. After a reboot the drive will mount (although it said it failed, but due to the folder being there) but not upgrade as it is supposed to.

So since im moving my drives to a new server im only able to move and upgrade 1 drive at a time (although my drives are split over 5 google accounts and in theory have api limits on each account).

 

So my wish would be a new version which supports mounting new drives, while an existing is upgrading  - and fixing so that a drive which fails at being mounted doesnt get undeleteable due to being in use

[srcrist 60]

I finally bit the bullet last night and converted my drives. I'd like to report that even in excess of 250TB, the new conversion process finished basically instantly and my drive is fully functional. If anyone else has been waiting, it would appear to be fine to upgrade to the new format now.

[JulesTop 4]

2 hours ago, KiLLeRRaT said:

Hi All,

I've recently noticed that my drive has 40GB to be uploaded, and saw there are errors in the top left.  Clicking that showed 

What's the deal here, and is this going to be an ongoing issue with growing drives?

My drive is a 15TB drive, with 6TB free (so only using 9TB).

Attached is my drive's info screen.

EDIT: Cloud Drive Version: 1.1.5.1249 on Windows Server 2016.

 

Cheers,

I just started getting the same thing today. Based on the coincidence of both of us having the issue in the same day, this may be a Google issue.

[kird 0]

I know Googledrive has a file and folder limit, but now I don't know what the number is. In my case I'm not having, for now, this error but due to the large amount of folders and files I have on my drive I don't think I'll have much left to overcome it.  Good news that our DriveCloud will warn us about that error.

[JulesTop 4]

18 hours ago, kird said:

I know Googledrive has a file and folder limit, but now I don't know what the number is. In my case I'm not having, for now, this error but due to the large amount of folders and files I have on my drive I don't think I'll have much left to overcome it.  Good news that our DriveCloud will warn us about that error.

I doubt I hit my file and folder limit. The reason I say that is that I deleted a few folders and their files, and nothing changed. I still get this error and my uploads are hanging.

I'm wondering if this is happening to anyone else...

[kird 0]

4 hours ago, JulesTop said:

I doubt I hit my file and folder limit. The reason I say that is that I deleted a few folders and their files, and nothing changed. I still get this error and my uploads are hanging.

I'm wondering if this is happening to anyone else...

Figuring it out, have you deleted the files from the recycle bin? If you have not configured it in a special way, by default the deleted files go there and if they are not deleted they still count for space.

[JulesTop 4]

6 hours ago, kird said:

Figuring it out, have you deleted the files from the recycle bin? If you have not configured it in a special way, by default the deleted files go there and if they are not deleted they still count for space.

My windows recycling bin? I went there and it is empty. Just in case, I just changed the settings for my cloud drive to permanently be deleted without being stored in the recycling bin.

I also went to my actual Google drive we interface, however, the bin is empty. I'm still having the issue... It's weird, I deleted a bunch of files and still have the issue. I have it setup with 20MB chunks... So my drive would likely have to be humongous to reach the file limit.

[kird 0]

Yes, i was referring logically to the recycle bin of your gdrive account, the windows one here plays no role. As I understand it here the point is not the amount of Gb/Tb in files on the drive but the total number of folders that can be nested in the account.

If I understand correctly, according to this information the error occurs when it reaches 500.000 folders, files... inside a root folder in the drive.

"A numChildrenInNonRootLimitExceeded error occurs when the limit for a folder's number of children (folders, files, and shortcuts) has been exceeded. There is a 500,000 item limit for folders, files, and shortcuts directly in a folder. Items nested in subfolders do not count against this 500,000 item limit."

[JulesTop 4]

6 hours ago, kird said:

Yes, i was referring logically to the recycle bin of your gdrive account, the windows one here plays no role. As I understand it here the point is not the amount of Gb/Tb in files on the drive but the total number of folders that can be nested in the account.

If I understand correctly, according to this information the error occurs when it reaches 500.000 folders, files... inside a root folder in the drive.

"A numChildrenInNonRootLimitExceeded error occurs when the limit for a folder's number of children (folders, files, and shortcuts) has been exceeded. There is a 500,000 item limit for folders, files, and shortcuts directly in a folder. Items nested in subfolders do not count against this 500,000 item limit."

In that case... there is definitely something wrong... I don't believe I should be anywhere near that as Stabelbit is the only thing that manages my google drive. You can see in the attachment that my root directory only has one folder, and deeper to that, it's all managed by stablebit clouddrive.

 

[kird 0]

From the numbers shown there, it doesn't seem to be your case at all. This can be best traced from the unit mounted on your PC and looking at your Gdrive's statistics. 

Let's see if the developers can shed some light on the subject.

[JulesTop 4]

3 hours ago, kird said:

From the numbers shown there, it doesn't seem to be your case at all. This can be best traced from the unit mounted on your PC and looking at your Gdrive's statistics. 

Let's see if the developers can shed some light on the subject.

I'm not sure if this is helpful...

[kird 0]

That information gives you the error we talked about but it doesn't show anything of the total files/folders contained in a root folder. At least I can't see it. 

When I say look at your locally mounted unit (it's decrypted) I mean this. As you can see in the image it's still far away from the 500,000

 

 

[JulesTop 4]

1 hour ago, kird said:

That information gives you the error we talked about but it doesn't show anything of the total files/folders contained in a root folder. At least I can't see it. 

When I say look at your locally mounted unit (it's decrypted) I mean this. As you can see in the image it's still far away from the 500,000

 

 

BTW, I really appreciate the help. Here it is.

I wonder if there is a way of counting the number of chunks in google drive... since that is really what adds up

[kird 0]

In your image I don't see any appreciation for having exceeded the limit. I wouldn't know what else to tell you. Let's wait and see what the developers say, it's a error which must be verified where it can come from and it's basic to know. I guess now your gdrive is not operational, right?

[JulesTop 4]

Just now, kird said:

In your image I don't see any appreciation for having exceeded the limit. I wouldn't know what else to tell you. Let's wait and see what the developers say, it's a error which must be verified where it can come from and it's basic to know. I guess now your gdrive is not operational, right?

I can download from the gdrive without issue. I can also upload to the root of the gdrive without problem (manually). When I tried adding a file to the clouddrive content folder manually (via the gdrive web interface), it fails.

It looks like there is an issue with the content folder... Maybe @Alex or @Christopher (Drashna) can help.

[kird 0]

3 minutes ago, JulesTop said:

When I tried adding a file to the clouddrive content folder manually (via the gdrive web interface)

I have never been able to download or upload anything via gdrive web, all content is encrypted. How do you do it? In my case to upload/download content always through the mounted network drive or Air Explorer (great for managing clouds) but never from the google web interface.

Yup, hopefully they can fix this nasty error or give information.

 

[JulesTop 4]

So the issue has been "fixed". There must be a bug or something, but not sure if it's on Google's side or stablebit.

I deleted the file that I figured it was stuck trying to upload. The file I deleted was about 20GB, and there was over 65GB in queue to upload. As soon as I deleted the 20GB file from my computer (which was on the clouddrive), my upload queue seemed to just resume. It didn't drop by 20GB... It just resumed.

 

Anyway, everything seems to be totally back to normal.

[kird 0]

I'm glad you were able to fix that nasty error on your own. By the way, which version of CloudDrive are you using? I can tell you that in my case there is a version that I keep as a real gem because it is like the "holy grail" for all kinds of errors that the drive may involve. This version is v1.1.0.1051

[JulesTop 4]

On 6/20/2020 at 2:12 PM, kird said:

I'm glad you were able to fix that nasty error on your own. By the way, which version of CloudDrive are you using? I can tell you that in my case there is a version that I keep as a real gem because it is like the "holy grail" for all kinds of errors that the drive may involve. This version is v1.1.0.1051

I was on the latest stable (I think the last 4 digits we're 1249) when this occured. After a couple of days, I updated to the latest Beta to try and solve it.

Also, I realized that at some point I removed my personal gdrive API credentials from the config file, and this is about the time the issue resolved. I just put back my personal API credentials and re-authorized and the issue came back. I yet again switched back to 'null' (the default stablebit API credentials) and the issue went away again... I wonder if there are higher API limits with the Stablebit credentials.

 

Maybe @Christopher (Drashna) can shed some light.

[kird 0]

I don't have my api configured and I have never met that error, maybe these google guys have touched again something related to the API causing the program to fail. Probably because of what you're saying. Looks like the developers are on vacation.

[Jellepepe 4]

This issue appeared for me over 2 weeks ago (yay me) and it seems to be a gradual rollout.

The 500.000 items limit does make sense, as clouddrive stores your files in (up to) 20mb chunks on the provider, and thus the issue should appear somewhere around the 8-10tb mark depending on file duplication settings. 
In this case, the error actually says 'NonRoot', thus they mean any folder, which apparently can now only have a maximum of 500.000 children.

I've actually been in contact with Christopher since over 2 weeks ago about this issue, and he has informed Alex of the issue and they have confirmed they are working on a solution. 
(other providers already has such issues, and thus it will likely revolve around storing chunks in subfolders with ~100.000 chunks per folder, or something similar.)

It is very interesting that you mention reverting to the non-personal api keys resolved it for you, which does suggest that indeed it may be dependent on some (secret) api level limit.
Though, that would not explain why adding something to the folder manually also fails...

@JulesTop Have you tried uploading manually since the issue 'disappeared' ? If that still fails, it would confirm that the clouddrive api keys are what are 'overriding' the limit, and any personal access, including web, is limited.

Either way hopefully there is a definitive fix coming soon, perhaps until then using the default keys is an option.

 

[JulesTop 4]

1 hour ago, Jellepepe said:

This issue appeared for me over 2 weeks ago (yay me) and it seems to be a gradual rollout.

The 500.000 items limit does make sense, as clouddrive stores your files in (up to) 20mb chunks on the provider, and thus the issue should appear somewhere around the 8-10tb mark depending on file duplication settings. 
In this case, the error actually says 'NonRoot', thus they mean any folder, which apparently can now only have a maximum of 500.000 children.

I've actually been in contact with Christopher since over 2 weeks ago about this issue, and he has informed Alex of the issue and they have confirmed they are working on a solution. 
(other providers already has such issues, and thus it will likely revolve around storing chunks in subfolders with ~100.000 chunks per folder, or something similar.)

It is very interesting that you mention reverting to the non-personal api keys resolved it for you, which does suggest that indeed it may be dependent on some (secret) api level limit.
Though, that would not explain why adding something to the folder manually also fails...

@JulesTop Have you tried uploading manually since the issue 'disappeared' ? If that still fails, it would confirm that the cloudflare api keys are what are 'overriding' the limit, and any personal access, including web, is limited.

Either way hopefully there is a definitive fix coming soon, perhaps until then using the default keys is an option.

 

You called it. I just tried uploading a small file to the content folder directly from the Google drive web interface and got an 'upload failure (38)', same as before.

It looks like this restriction rollout is definitely happening. It will be fantastic if @Alex has a solution coming soon! But also good that the default keys are keeping us going for now.

[steffenmand 60]

Maybe this is related to the issues we have with Internal Error etc. if we are already above the 10 TB mark. Then it fails when trying to write new chunks because we are already above the 10 TB mark

[kird 0]

4 hours ago, steffenmand said:

Maybe this is related to the issues we have with Internal Error etc. if we are already above the 10 TB mark. Then it fails when trying to write new chunks because we are already above the 10 TB mark

So far, with google's own api, I haven't had that kind of error and my files take up more than 100 TB

[Jellepepe 4]

4 hours ago, steffenmand said:

Maybe this is related to the issues we have with Internal Error etc. if we are already above the 10 TB mark. Then it fails when trying to write new chunks because we are already above the 10 TB mark

I also doubt this is related to this issue. I have well over 10TB used and i have never had a 'internal error' error. 

The only errors i have seen are user rate limit exceeded (when exceeding the upload or download limits) and since about 2 weeks this exceeded maximum number of children error.