Jump to content
  • 1

Google Drive: The limit for this folder's number of children (files and folders) has been exceeded


KiLLeRRaT

Question

Hi All,

I've recently noticed that my drive has 40GB to be uploaded, and saw there are errors in the top left.  Clicking that showed 

Quote

The limit for this folder's number of children (files and folders) has been exceeded

What's the deal here, and is this going to be an ongoing issue with growing drives?

My drive is a 15TB drive, with 6TB free (so only using 9TB).

Attached is my drive's info screen.

EDIT: Cloud Drive Version: 1.1.5.1249 on Windows Server 2016.

 

Cheers,

2020-06-18 11_42_07-mRemoteNG - confCons.xml - oscar.gouws.org.png

Link to comment
Share on other sites

206 answers to this question

Recommended Posts

  • 0
5 minutes ago, srcrist said:

Just a note for those of you concerned with a lack of drive activity via the web interface: I don't believe Google reports file *movement* in the activity panel. Only file creation and modification. If the file is simply moved from folder A to folder B, it shouldn't show up. To confirm that the migration is working correctly, I would note the most recent directory structure and compare it some time later and see if new folders have been created or files have been moved to the folders that exist.

Theoretically, this migration should produce little, if any, "activity" in the web interface activity panel.

This is not correct, the web UI shows file movement.

1.PNG

Link to comment
Share on other sites

  • 0
1 hour ago, srcrist said:

 

OK. My mistake, then. I haven't started this process, I just thought I remembered this fact from some rClone work I've done previously. I'll remove that comment.

1306 doesnt show any files being moved though like 1305... so dont know if it works or not :-P

Link to comment
Share on other sites

  • 0
3 hours ago, steffenmand said:

1306 doesnt show any files being moved though like 1305... so dont know if it works or not :-P

Yea, that's basically my issue at this point - other than the percentage increasing, there's no evidence anything is actually happening anymore

Link to comment
Share on other sites

  • 0

Unintentional Guinea Pig Diaries.

Day 8 - Entry 1

I spent the rest of yesterday licking my wounds and contemplating a future without my data. I could probably write a horror movie script on those thoughts but it would be too dark for the people in this world. I must learn from my transgressions. In an act of self punishment and an effort to see the world from a different angle I slept in the dogs bed last night. He never sleeps there anyways but he would never have done the things I did. For that I have decided he holds more wisdom than his human. This must have pleased the Data God's because this morning I awoke with back pain and two of my drives mounted and functioning. The original drive which had completed the upgrade, had been working, and then went into "drive initializing"...is now working again. The drive that I had tried to mount and said it was upgrading with no % given has mounted (FYI 15TB Drive with 500GB on it). However my largest drive is still sitting at "Drive queued to perform recovery".

Maybe one more night in the dogs bed will complete the offering required to the Data God's

End Diary entry.

 

(P.S. Just in case you wondered...that spoiled dog has "The Big One" Lovesac as a dog bed.. In a pretty ironic fashion their website is down. #Offering) ps-8dm-ms006-l.jpg

Link to comment
Share on other sites

  • 0

I had the wisdom 2 years ago when one of the ssds in my server corrupted and took almost a full 50TB of data in a clouddrive with it, that i would mirror all my data over multiple clouddrives and multiple google accounts, i am very happy with my past self at this stage.

I will start a staged rollout of this process on my drives and keep you updated if i find any issues.

Link to comment
Share on other sites

  • 0
1 hour ago, Chase said:

...

this morning I awoke with back pain and two of my drives mounted and functioning. The original drive which had completed the upgrade, had been working, and then went into "drive initializing"...is now working again. The drive that I had tried to mount and said it was upgrading with no % given has mounted (FYI 15TB Drive with 500GB on it). However my largest drive is still sitting at "Drive queued to perform recovery".

Did you use your own API keys or the default ones?

Link to comment
Share on other sites

  • 0
2 hours ago, Chase said:

Unintentional Guinea Pig Diaries.

Day 8 - Entry 1

I spent the rest of yesterday licking my wounds and contemplating a future without my data. I could probably write a horror movie script on those thoughts but it would be too dark for the people in this world. I must learn from my transgressions. In an act of self punishment and an effort to see the world from a different angle I slept in the dogs bed last night. He never sleeps there anyways but he would never have done the things I did. For that I have decided he holds more wisdom than his human. This must have pleased the Data God's because this morning I awoke with back pain and two of my drives mounted and functioning. The original drive which had completed the upgrade, had been working, and then went into "drive initializing"...is now working again. The drive that I had tried to mount and said it was upgrading with no % given has mounted (FYI 15TB Drive with 500GB on it). However my largest drive is still sitting at "Drive queued to perform recovery".

Maybe one more night in the dogs bed will complete the offering required to the Data God's

End Diary entry.

 

(P.S. Just in case you wondered...that spoiled dog has "The Big One" Lovesac as a dog bed.. In a pretty ironic fashion their website is down. #Offering) ps-8dm-ms006-l.jpg

Are you using your own api key or the api key from stablebit?

Link to comment
Share on other sites

  • 0
1 hour ago, ezek1el3000 said:

Did you use your own API keys or the default ones?

Uh. Yes.

When this process first started I was on the default API. At that time I had two drives going through the update. In the middle of that process I added my own API but since you aren't able to reauthorize during the update it didn't actually switch to my API. When my first drive finished I was able to reauth. My second drive that is still having issues has never been able to reauth so my understanding is that it is still on the default API. The third drive that I mounted and said it was updating without showing a % was after I added my own API and therefore authorized on my API.

Link to comment
Share on other sites

  • 0

I'm still holding off patiently on the conversion, it sounds like it works, but waiting to get a better idea at the of the time it takes by drive data size.

 

I've noticed that without any changed settings these past few days I've gotten a couple yellow I/O error warnings about user upload rate limit exceeded (which otherwise haven't been problems), and I've noticed gdrive side upload throttling at a lower than normal concurrency, only 4 workers at 70mbit.

 

I'm guessing some of these rate limit errors people may be seeing in converting are transient from gdrive being under high load.

Link to comment
Share on other sites

  • 0

I may have fixed my drive that is stuck in "queued for recovery". But damn did it make me nervous and I'm far from out of the woods. In my understanding of the system the "cache" is just a local file of your pinned data and a COPY of the last #GB of data that was received or transferred between your machine and the cloud drives. It also seems that it holds the information for the CloudDrive program to say...yep this is a drive that we have mounted. So I shut down the windows service for CloudDrive and deleted the cache file for my problem drive. After restarting my computer when CloudDrive opened up it only showed the two working drives as mounted and the problem drive was available to mount. Sweet! Mount that sucker.

Seems to be working. It now says that the drive is upgrading and I get a percentage actually shown. Albeit it is way slower compared to last time I went through this stage. Probably because this latest Beta has less I/O interactions. It did NOT give me the option to reauth so I assume this drive is still using the StableBit API.

At this rate it should complete by Monday-ish? 

Link to comment
Share on other sites

  • 0
6 minutes ago, Chase said:

...

and deleted the cache file for my problem drive

nice find. what is that file called? and where can i find it?

Right now 2/6 drives are queued to perform recovery and the rest is in "Drive cannot be mounted" mode :wacko:

Edith: did you refer to the cache files stored at the designated CD cache drive? If so, there are over 100 files. How did you know which one to delete?

Edited by ezek1el3000
reasons
Link to comment
Share on other sites

  • 0

Decided to try for the beta as well, now that I have holiday to babysit the process.

2 drives started doing the upgrade (Drive is upgrading), 2 are stuck in "Drive initializing" and the 5 others seemed to work fine.

Due to Windows updates, I restarted the server after checking this thread and after seeing some weird issues (the mounted ones started saying it can't find the file when looking in performance settings, I forgot the phrasing of it) and found a bit of a grizzly sight.

The 5 drives that were "fine" are now unable to be mounted and my cache drive is completely filled to the brim (44k free). This could be due to some automation filling the earlier mounted drives, but I have never seen it be filled below the 5 GB limit. Now, they most likely cannot be mounted, hopefully, due to no space left on the cache and thus not possible to write to it, but as I don't have a functioning drive or any means to actually empty the cache drive, I'm stuck in a bit of a place where I am unsure if everything is lost or I just have to be patient.

Log says: [CloudDrives] Cannot mount cloud part xyz. It was force unmounted.

I'll try for patience for now.

So, be careful, out there.

Link to comment
Share on other sites

  • 0
1 hour ago, Chase said:

In my understanding of the system the "cache" is just a local file of your pinned data and a COPY of the last #GB of data that was received or transferred between your machine and the cloud drives.

The cache also includes information that has been modified but not yet uploaded. Everyone should be *very* careful before simply deleting the local cache. Note that any modifications that have not yet been uploaded to the cloud provider will be permanently lost, and you could potentially corrupt your drive as a result. I believe that you are theoretically correct as long as none of the information in the cache has yet to be uploaded, but extreme caution should be used before following this example. Anyone who deletes their cache with data in the "to upload" state will, A), definitely lose that data for good and, B), potentially corrupt their drive depending on what that data is (read: file system data, for example).

Edited by srcrist
Link to comment
Share on other sites

  • 0
4 minutes ago, trillex said:

The 5 drives that were "fine" are now unable to be mounted and my cache drive is completely filled to the brim (44k free). This could be due to some automation filling the earlier mounted drives, but I have never seen it be filled below the 5 GB limit.

It sounds like we might be stumbling on some buggy cache code, between this and the previous notes from Chase about deleting the cache. Make sure you guys are submitting actual tickets and troubleshooter dumps as well, so Alex and Christopher can take a look at the related code and your logs.

Link to comment
Share on other sites

  • 0
1 hour ago, ezek1el3000 said:

nice find. what is that file called? and where can i find it?

Right now 2/6 drives are queued to perform recovery and the rest is in "Drive cannot be mounted" mode :wacko:

Edith: did you refer to the cache files stored at the designated CD cache drive? If so, there are over 100 files. How did you know which one to delete?

To answer your question..I deleted the main Cache file which for me is stored on my C:/ Drive. It starts with Clouddrive and is followed by your drive UID.

 

as SRCRIST said....it is very risky and my drive may be totally corrupt when it finishes. That was a risk I was okay with taking. You make your own decision.

 

I did get a message back from Christopher on the Covecube team. They looked at my path of issues and whats happened plus troubleshooter data and his message was this...

"We have found a bug in the migration code, that may be the problem here.We should have a fix out soon, that should be much faster (or even instantaneous) and handle failure better. "

 
Link to comment
Share on other sites

  • 0

New beta looks to have major improvements to the migration process, make sure you're on it before reporting any additional bugs or doing something crazy like delete local cache.

.1307
* Added detailed logging to the Google Drive migration process that is enabled by default.
* Redesigned the Google Drive migration process to be quicker in most cases:
    - For drives that have not run into the 500,000 files per folder limit, the upgrade will be nearly instantaneous.
    - Is able to resume from where the old migration left off.
* [Issue #28410] Output a descriptive Warning to the log when a storage provider's data organization upgrade fails.

 

Link to comment
Share on other sites

  • 0
2 hours ago, Firerouge said:

New beta looks to have major improvements to the migration process, make sure you're on it before reporting any additional bugs or doing something crazy like delete local cache.


.1307
* Added detailed logging to the Google Drive migration process that is enabled by default.
* Redesigned the Google Drive migration process to be quicker in most cases:
    - For drives that have not run into the 500,000 files per folder limit, the upgrade will be nearly instantaneous.
    - Is able to resume from where the old migration left off.
* [Issue #28410] Output a descriptive Warning to the log when a storage provider's data organization upgrade fails.

 

Upgraded to this version and my drive instantly mounted. If I understand the logs correctly, the new version simply doesn't bother moving chunks if the drive is under the 490k limit? I'm assuming upgrading part way through the prior version moving the files hasn't broken anything but I am entirely not sure.

Link to comment
Share on other sites

  • 0

Upgraded to the latest beta this morning. That drive I was having issues with and had deleted the cache was showing about 15% completed at the time of upgrade. The percent started over but it only took about 35 minutes to complete and mount my drive. (56TB with 20TB of data). Everything works on all drives except that annoying error when I try to change the I/O settings I talked about previously. Again...the changes take it just pops up with an error that the drive isn't mounted.
 

I've got family in for the next several days. Be safe guys. I'm very happy with the latest beta fix. All of my drives are functional.

 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...