Google Drive: The limit for this folder's number of children (files and folders) has been exceeded

[KiLLeRRaT]

Hi All,

I've recently noticed that my drive has 40GB to be uploaded, and saw there are errors in the top left.  Clicking that showed 

Quote

The limit for this folder's number of children (files and folders) has been exceeded

What's the deal here, and is this going to be an ongoing issue with growing drives?

My drive is a 15TB drive, with 6TB free (so only using 9TB).

Attached is my drive's info screen.

EDIT: Cloud Drive Version: 1.1.5.1249 on Windows Server 2016.

 

Cheers,

[darkly]

I'm noticing that since upgrading to 1316, I'm getting a lot more I/O errors saying there was trouble uploading to google drive. Is there some under the hood setting that changed which would cause this? I've noticed it happen a few times a day now where previously it'd hardly ever happen.

Other than that, not noticing any issues with performance.

EDIT

Here's a screenshot of the error:

[KiLLeRRaT]

I also upgraded to 1315 and it looks like everything is working the way that it should.  I have not had issues that @darkly have reported where gets the insufficient bandwidth errors.

My disk upgrade went instantly, even through I was getting the API limit error before.  I did revert back to standard API keys (not using my own) before upgrading.

I haven't yet tried 1316 yet.

Cheers,

[darkly]

7 minutes ago, KiLLeRRaT said:

I also upgraded to 1315 and it looks like everything is working the way that it should.  I have not had issues that @darkly have reported where gets the insufficient bandwidth errors.

My disk upgrade went instantly, even through I was getting the API limit error before.  I did revert back to standard API keys (not using my own) before upgrading.

I haven't yet tried 1316 yet.

Cheers,

I should probably mention I'm using my own API keys, though I don't see how that should affect this in this way (I was using my own API keys before the upgrade too). I'm also on a gigabit fiber connection and nothing about that has changed since the upgrade. As far as I can tell, this feels like an issue with CD.

[srcrist]

I haven't had any errors with 1316 either. It might have been a localized network issue.

[darkly]

On 7/20/2020 at 10:49 PM, srcrist said:

I haven't had any errors with 1316 either. It might have been a localized network issue.

still going on. Happens several times a day consistently since I upgraded to 1316. Never had the error before unless I was actually having a connection issue with my internet.

[srcrist]

16 hours ago, darkly said:

still going on. Happens several times a day consistently since I upgraded to 1316. Never had the error before unless I was actually having a connection issue with my internet.

I'm still not seeing an issue here. Might want to open a ticket and submit a troubleshooter and see if Christopher and Alex can see a problem. It doesn't seem to be universal for this release.

[darkly]

On 7/22/2020 at 12:07 PM, srcrist said:

I'm still not seeing an issue here. Might want to open a ticket and submit a troubleshooter and see if Christopher and Alex can see a problem. It doesn't seem to be universal for this release.

Might do that if it keeps happening. Out of curiosity, can you share your I/O Performance settings?

[srcrist]

14 minutes ago, darkly said:

Might do that if it keeps happening. Out of curiosity, can you share your I/O Performance settings?

Sure. Presently using:

• 12 down threads, 5 up threads
• Background I/O Disabled
• Upload throttling at 70mbits/s
• Upload threshold at 1MB or 5mins
• 20MB Minimum download size
• Prefetching enabled with:
  • 8MB trigger
  • 150MB forward
  • and a 15 second window

[darkly]

Just now, srcrist said:

Sure. Presently using:

• 12 down threads, 5 up threads
• Background I/O Disabled
• Upload throttling at 70mbits/s
• Upload threshold at 1MB or 5mins
• 20MB Minimum download size
• Prefetching enabled with:
  • 8MB trigger
  • 150MB forward
  • and a 15 second window

Are you using this for plex or something similar as well? I've been uploading several hundred gigabytes per day over the last few days and that's when I'm seeing the error come up. It doesn't seem to affect performance or anything. It usually continues just fine, but that error pops up 1-3 times, sometimes more. My settings are just about the same with some slight differences in the Prefetcher section. I've seen a handful of conflicting suggestions when it comes to that here. This is what I have right now: 
Don't think that should cause the issues I'm seeing though... Are you also using your own API keys?

[srcrist]

I use CloudDrive for a few things, but the largest drive, which is the one that I just gave you settings for, is for media, yes. It's constantly uploading as well, basically. I am not currently using my own API keys for that drive. There isn't really an "optimal" prefetcher setting--it just depends on your needs. I shouldn't really impact this.

[StealthyDruid]

Same, I also just started seeing this problem happening and I've been using Stablebit for Plex for years. It seems to have started yesterday. Tonight I updated to the latest Stablebit version (1.1.6.1318) and I'm still getting the errors on my Google drives (on a GSuite account). I logged into the GSuite admin console and I don't see any errors regarding limits, everything says 'unlimited'. I viewed the actual folders and they don't show errors either.

This can't be a coincidence that suddenly a bunch of us are seeing these errors.... any thoughts/ideas/suggestions would be very welcome at this point!

[Tlewishood]

I have begun to move my content to a new CloudDrive that I created. Seems that is going well. I am moving it to a different account, one of the 5 that I have to get unlimited in GSuite. 

[Homineslupus]

I have too, it seems there is a new limit by google, max 500k items in one folder. (files and folders)

[superka]

what now? create another drive ?

[wildcard8493]

I also just started experiencing this issue in the last few days. The timing of the error with the re-brand from GSuite to Google Workspace is a bit sus.

I have 8 clouddrives pooled together with DrivePool.  Each drive is tied to a different user under a common G Suite account (so > 5 users so unlimited space).  The pool has a deduplication on and set to 3.  The pool is used for media

The error is on the upload side.  When working on the To Upload data backlog, the chunks are uploaded but as soon as they reach 100% they error out and the drive retries it looks like.  As such, the drive uploads indefinitely but the amount of data is never reduced.

Here is a screenshot: https://pasteboard.co/JuQuHdE.png

CloudDrive 1.1.6.1318

Windows Server 2019 1809

[JoeyD]

5 hours ago, superka said:

what now? create another drive ?

Managed to fix this today as my client was giving errors also. 

Install Beta version from here: http://dl.covecube.com/CloudDriveWindows/beta/download/ (I used 1344)

Reboot.

Don't start CloudDrive and/or service.

Add the below to this file: C:\ProgramData\StableBit CloudDrive\Service\Settings.json

   "GoogleDrive_ForceUpgradeChunkOrganization": {
    "Default": true,
    "Override": true
  } 

Start Service & CloudDrive. Should kick in straight away. I have 42TB in GDrive and it went through immediately. Back to uploading as usual now. 

Hope this helps. 

[superka]

4 hours ago, JoeyD said:

Managed to fix this today as my client was giving errors also. 

Install Beta version from here: http://dl.covecube.com/CloudDriveWindows/beta/download/ (I used 1344)

Reboot.

Don't start CloudDrive and/or service.

Add the below to this file: C:\ProgramData\StableBit CloudDrive\Service\Settings.json

   "GoogleDrive_ForceUpgradeChunkOrganization": {
    "Default": true,
    "Override": true
  } 

Start Service & CloudDrive. Should kick in straight away. I have 42TB in GDrive and it went through immediately. Back to uploading as usual now. 

Hope this helps. 

I'm afraid of doing this.

 

Last time i did a rollback my computer entered in a bluescreen reboot mode all the time.

@srcrist can we have some info about this? i'm afraid

[superka]

Just now, superka said:

I'm afraid of doing this.

 

Last time i did a rollback my computer entered in a bluescreen reboot mode all the time.

@srcrist can we have some info about this? i'm afraid of lossing my data.

 

[Gijs]

3 hours ago, superka said:

 

I'm having this same exact problem since this morning. Before trying the above solution I want to make sure it's safe as well. Currently my drive is stuck and I paused uploading. If anyone has any more info about this please let us know. My drive has unmounted a few times already... @Christopher (Drashna)

[jaynew]

Yeah.... Same here - two out of my 11 cloud drives (all on one GSuite account) are both producing the same "limit for this folder's number of children has been exceeded". Stuck uploading and can only pause to stop it. Version 1318 & Windows Server 2016 1607.

[otravers]

16 hours ago, jaynew said:

Yeah.... Same here - two out of my 11 cloud drives (all on one GSuite account) are both producing the same "limit for this folder's number of children has been exceeded". Stuck uploading and can only pause to stop it.

Same here for the past couple of days, v1318 on Windows 10 Pro.

[StealthyDruid]

3 hours ago, Homineslupus said:

I have installed the last beta because if you read the changelog, several modifications for this in beta since several versions.

I not forced manually like JoeyD but all cloud drive with problems are forced upgrade at unlock, it's indicated and fast.

(maybe hybrid Flat/Hierarchical, because explanations in changelog)

 

More details in changelog in http://dl.covecube.com/CloudDriveWindows/beta/download/changes.txt

In .1305 .1307 ...

 

And after upload is good for me.

Thanks! I was hesitant to do the beta with the edits that JoeyD mentioned, but after your post, I also installed the latest Beta (1.2.0.1344) and the drives started uploading again! I didn't have to do anything other than reboot after the Beta installed!

All is good for me too now!

[otravers]

All good on my end too with the 1344 beta! Here's hoping we do retain the ability have an infinite Gdrive.

[srcrist]

20 hours ago, superka said:

Last time i did a rollback my computer entered in a bluescreen reboot mode all the time.

@srcrist can we have some info about this? i'm afraid

I don't have any info on this other than to say that I am not experiencing these issues, and that I haven't experienced any blue screens related to those settings. That user isn't suggesting a rollback, they're suggesting that you edit the advanced settings to force your drive to convert to the newer hierarchical format.

I should also note that I do not work for Covecube--so aside from a lot of technical experience with the product, I'm probably not the person to consult about new issues. I think we might need to wait on Christopher here. My understanding, though, was that those errors were fixed with release .1314. It presumes that existing data is fine as-is, and begins using a hierarchical structure for any NEW data that you add to the drive. That should solve the problem. So make sure that you're on .1314 or later for sure.

Relevant changelog:

.1314
* [Issue #28415] Created a new chunk organization for Google Drive called Hierarchical Pure.
    - All new drives will be Hierarchical Pure.
    - Flat upgraded drives will be Hierarchical, which is now a hybrid Flat / Hierarchical mode.
    - Upgrading from Flat -> Hierarchical is very fast and involves no file moves.
* Tweaked Web UI object synchronization throttling rules.
.1312
* Added the drive status bar to the Web UI.
.1310
* Tuned statistics reporting intervals to enable additional statistics in the StableBit Cloud.
.1307
* Added detailed logging to the Google Drive migration process that is enabled by default.
* Redesigned the Google Drive migration process to be quicker in most cases:
    - For drives that have not run into the 500,000 files per folder limit, the upgrade will be nearly instantaneous.
    - Is able to resume from where the old migration left off.

 

[superka]

On 10/10/2020 at 6:34 PM, srcrist said:

I don't have any info on this other than to say that I am not experiencing these issues, and that I haven't experienced any blue screens related to those settings. That user isn't suggesting a rollback, they're suggesting that you edit the advanced settings to force your drive to convert to the newer hierarchical format.

I should also note that I do not work for Covecube--so aside from a lot of technical experience with the product, I'm probably not the person to consult about new issues. I think we might need to wait on Christopher here. My understanding, though, was that those errors were fixed with release .1314. It presumes that existing data is fine as-is, and begins using a hierarchical structure for any NEW data that you add to the drive. That should solve the problem. So make sure that you're on .1314 or later for sure.

Relevant changelog:

.1314
* [Issue #28415] Created a new chunk organization for Google Drive called Hierarchical Pure.
    - All new drives will be Hierarchical Pure.
    - Flat upgraded drives will be Hierarchical, which is now a hybrid Flat / Hierarchical mode.
    - Upgrading from Flat -> Hierarchical is very fast and involves no file moves.
* Tweaked Web UI object synchronization throttling rules.
.1312
* Added the drive status bar to the Web UI.
.1310
* Tuned statistics reporting intervals to enable additional statistics in the StableBit Cloud.
.1307
* Added detailed logging to the Google Drive migration process that is enabled by default.
* Redesigned the Google Drive migration process to be quicker in most cases:
    - For drives that have not run into the 500,000 files per folder limit, the upgrade will be nearly instantaneous.
    - Is able to resume from where the old migration left off.

 

Ok, upgraded to las beta version crossing my fingers and everythig is working fine, no more errors. Thanks.