Jump to content
Covecube Inc.
  • 0
seanvree

PLEX scanning TV shows on Cloud Drive takes A LONG TIME

Question

OS:  Windows 10E x64 1709 FCU

ST CD V: 1.0.2963 BETA

Provider:  MS OneDrive for Business

 

I have about 1TB of TV shows on my Cloud Drive.  I"ve noticed when PLEX does it's normal scanning of the TV shows that are stored on the drive it takes HOURS to complete scanning.  CD shows it's downloading data at 6+ MB/s, so I"m not sure why this would take so long to simply scan the TV shows, about 30 min for a 30 min episode.  Playing content from the drive works just fine.  No errors in PLEX logs or CD,  just takes a long damn time. 

Settings in the attachment. 

 

SBCD.thumb.PNG.972797cc53dcef2868d13b26d5121281.PNG

 

 

 

 

 

 

 

Share this post


Link to post
Share on other sites

12 answers to this question

Recommended Posts

  • 0

Can you be more explicit about what sort of scanning plex is doing? What are the other settings for CloudDrive? What is your chunk size? 

I use Google Drive, not OneDrive, but scanning for changes takes no time at all. I once lost my library though, and re-adding all of the content took over a day--so if that's what it's doing, that's normal. But give some more details.

Share this post


Link to post
Share on other sites
  • 0
2 hours ago, srcrist said:

Can you be more explicit about what sort of scanning plex is doing? What are the other settings for CloudDrive? What is your chunk size? 

I use Google Drive, not OneDrive, but scanning for changes takes no time at all. I once lost my library though, and re-adding all of the content took over a day--so if that's what it's doing, that's normal. But give some more details.

 

@srcrist thanks for the response Sir. 

ANY type of scanning for the library (which is stored on the CD)  will trigger this.  So, if a show is added, or during regular "scheduled tasks". 

Cache size is set to 30GB. 

I also compiled a quick 30-second screen grab of what's going on.  You'll see in the top left frame shows that PLEX is scanning the "TV SHOWS" library (which is where the CD is attached to).  However, in the top right frame shows PLEX is not logging activity (which I think is normal in cases where it's a "scheduled task").  Bottom left frame you'll see provider I/O, and the bottom right frame shows that there is actual data being downloaded from the provider.   And it stays like this for several hours.  

 

My question is...how could it be downloading THIS much data?  What is it downloading?  I don't get it. 

Plex_SB_Scan.gif

Share this post


Link to post
Share on other sites
  • 0
6 hours ago, Christopher (Drashna) said:

Plex is not just scanning the file name. it's reading the file's header.  So that is a couple of MB from each video.  So, it's jumping around a LOT.  

 

enabling prefetching may help with this, as may doubling the number of threads that you're using. 

Chris, 

 

I actually have prefetching disabled ATM as suggested by you.  Which has helped the disconnecting issue A LOT

 

 

 

Share this post


Link to post
Share on other sites
  • 0

I have a feeling that a caching scheme where we could opt to store xMB of each file at minimum for files over yMB treshold work. That way if there were 10 000 files in the Plex Library, we could store for example 2MB from the start of each file (total 20GB) to make the scan work as fast as on local drives.

Share this post


Link to post
Share on other sites
  • 0
On 12/19/2017 at 9:44 PM, Kraevin said:

I have a feeling this has something to do with One Drive, as Google, Box and Dropbox i have not seen this issue with.

It may be.  OneDrive for Business is backed by sharepoint, which is known for performance issues. 

On 12/20/2017 at 8:49 AM, red said:

I have a feeling that a caching scheme where we could opt to store xMB of each file at minimum for files over yMB treshold work. That way if there were 10 000 files in the Plex Library, we could store for example 2MB from the start of each file (total 20GB) to make the scan work as fast as on local drives.

It's been requested in th the past, aind it's a feature request. 

As for an ETA, I can't even begin to comment on that. 

Share this post


Link to post
Share on other sites
  • 0

@Christopher (Drashna)  

 

I wanted to follow up on this a bit. As it's still a big problem for me.  

When I'm downloading data from the drive that's mounted to my MS OneDrive account, the activity fluctuates anywhere from 5MB/s then back down to 0, will hang at zero for a few seconds, then go back up.  Nothing else is happening on the server, or on the LAN.  

UL pipe is 30mb/s

DL is 250 mb/s

 

Settings are in the attached screenshot.

Here's a small GIF to show you what's going on. Pay attention to the bottom right corner as that's the file copy process:

 

https://drive.google.com/file/d/1omSwQsDaY8ivDq9_LNlhxeDRdB5o-zqE/view?usp=sharing

 

SB_CloudDriveSettings.PNG

Share this post


Link to post
Share on other sites
  • 0

Unfortunately, the prefetcher may help in this case.

That said, could you do both of these:
http://wiki.covecube.com/StableBit_CloudDrive_Drive_Tracing
http://wiki.covecube.com/StableBit_CloudDrive_Web_Logs

 

And could you open a ticket at https://stablebit.com/Contact about this?

Share this post


Link to post
Share on other sites
  • 0

I'll chime in here, as I've been fiddling around a lot with settings to find what's fastest for Plex library scans. I have 250MBit downstream and 50MBbit up.

Provider: GDrive (Gsuite account)
Cache size: I keep changing this based on other activities as I'm limited by quite a small SSD, 60GB for now.
Minimum download size: 5MB - We're running with high thread count small minimum DL size so that we have many threads for scanning that die fast when they've done their job.
Download threads: 15 - this lets Plex read a lot of files concurrently.
Prefetch trigger: 50MB - I want Plex to prefetch whenever it's doing a deep analyze of a file or when I'm watching something, but I don't want partial scans triggering prefetch.
Prefetch forward: 500MB
Prefetch time window: 30s (I don't really know the function of this setting, though)

For me, prefetch on / off doesn't really affect scans at all, scans don't trigger it for me at all at current settings - it kicks in when playing files from the cloud though, and I've been wondering if it would be a good idea to put it even higher as most of my files are around a lot larger than 500MB.

Share this post


Link to post
Share on other sites
  • 0
5 hours ago, red said:

Prefetch trigger: 50MB - I want Plex to prefetch whenever it's doing a deep analyze of a file or when I'm watching something, but I don't want partial scans triggering prefetch.
Prefetch forward: 500MB
Prefetch time window: 30s (I don't really know the function of this setting, though)

 

The prefetch trigger is how much data has to be read (sequentially) in the prefetch time window amount of time.  Then, it grabs the next prefetch forward amount of data. 

So, the larger the window, the longer the window for the trigger.  And the larger the trigger, the harder it is to trigger. 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×