Jump to content
Covecube Inc.
  • 0
seanvree

PLEX scanning TV shows on Cloud Drive takes A LONG TIME

Question

OS:  Windows 10E x64 1709 FCU

ST CD V: 1.0.2963 BETA

Provider:  MS OneDrive for Business

 

I have about 1TB of TV shows on my Cloud Drive.  I"ve noticed when PLEX does it's normal scanning of the TV shows that are stored on the drive it takes HOURS to complete scanning.  CD shows it's downloading data at 6+ MB/s, so I"m not sure why this would take so long to simply scan the TV shows, about 30 min for a 30 min episode.  Playing content from the drive works just fine.  No errors in PLEX logs or CD,  just takes a long damn time. 

Settings in the attachment. 

 

SBCD.thumb.PNG.972797cc53dcef2868d13b26d5121281.PNG

 

 

 

 

 

 

 

Share this post


Link to post
Share on other sites

15 answers to this question

Recommended Posts

  • 0

Can you be more explicit about what sort of scanning plex is doing? What are the other settings for CloudDrive? What is your chunk size? 

I use Google Drive, not OneDrive, but scanning for changes takes no time at all. I once lost my library though, and re-adding all of the content took over a day--so if that's what it's doing, that's normal. But give some more details.

Share this post


Link to post
Share on other sites
  • 0
2 hours ago, srcrist said:

Can you be more explicit about what sort of scanning plex is doing? What are the other settings for CloudDrive? What is your chunk size? 

I use Google Drive, not OneDrive, but scanning for changes takes no time at all. I once lost my library though, and re-adding all of the content took over a day--so if that's what it's doing, that's normal. But give some more details.

 

@srcrist thanks for the response Sir. 

ANY type of scanning for the library (which is stored on the CD)  will trigger this.  So, if a show is added, or during regular "scheduled tasks". 

Cache size is set to 30GB. 

I also compiled a quick 30-second screen grab of what's going on.  You'll see in the top left frame shows that PLEX is scanning the "TV SHOWS" library (which is where the CD is attached to).  However, in the top right frame shows PLEX is not logging activity (which I think is normal in cases where it's a "scheduled task").  Bottom left frame you'll see provider I/O, and the bottom right frame shows that there is actual data being downloaded from the provider.   And it stays like this for several hours.  

 

My question is...how could it be downloading THIS much data?  What is it downloading?  I don't get it. 

Plex_SB_Scan.gif

Share this post


Link to post
Share on other sites
  • 0

Plex is not just scanning the file name. it's reading the file's header.  So that is a couple of MB from each video.  So, it's jumping around a LOT.  

 

enabling prefetching may help with this, as may doubling the number of threads that you're using. 

Share this post


Link to post
Share on other sites
  • 0
6 hours ago, Christopher (Drashna) said:

Plex is not just scanning the file name. it's reading the file's header.  So that is a couple of MB from each video.  So, it's jumping around a LOT.  

 

enabling prefetching may help with this, as may doubling the number of threads that you're using. 

Chris, 

 

I actually have prefetching disabled ATM as suggested by you.  Which has helped the disconnecting issue A LOT

 

 

 

Share this post


Link to post
Share on other sites
  • 0

I have a feeling that a caching scheme where we could opt to store xMB of each file at minimum for files over yMB treshold work. That way if there were 10 000 files in the Plex Library, we could store for example 2MB from the start of each file (total 20GB) to make the scan work as fast as on local drives.

Share this post


Link to post
Share on other sites
  • 0
On 12/19/2017 at 9:44 PM, Kraevin said:

I have a feeling this has something to do with One Drive, as Google, Box and Dropbox i have not seen this issue with.

It may be.  OneDrive for Business is backed by sharepoint, which is known for performance issues. 

On 12/20/2017 at 8:49 AM, red said:

I have a feeling that a caching scheme where we could opt to store xMB of each file at minimum for files over yMB treshold work. That way if there were 10 000 files in the Plex Library, we could store for example 2MB from the start of each file (total 20GB) to make the scan work as fast as on local drives.

It's been requested in th the past, aind it's a feature request. 

As for an ETA, I can't even begin to comment on that. 

Share this post


Link to post
Share on other sites
  • 0

@Christopher (Drashna)  

 

I wanted to follow up on this a bit. As it's still a big problem for me.  

When I'm downloading data from the drive that's mounted to my MS OneDrive account, the activity fluctuates anywhere from 5MB/s then back down to 0, will hang at zero for a few seconds, then go back up.  Nothing else is happening on the server, or on the LAN.  

UL pipe is 30mb/s

DL is 250 mb/s

 

Settings are in the attached screenshot.

Here's a small GIF to show you what's going on. Pay attention to the bottom right corner as that's the file copy process:

 

https://drive.google.com/file/d/1omSwQsDaY8ivDq9_LNlhxeDRdB5o-zqE/view?usp=sharing

 

SB_CloudDriveSettings.PNG

Share this post


Link to post
Share on other sites
  • 0

Unfortunately, the prefetcher may help in this case.

That said, could you do both of these:
http://wiki.covecube.com/StableBit_CloudDrive_Drive_Tracing
http://wiki.covecube.com/StableBit_CloudDrive_Web_Logs

 

And could you open a ticket at https://stablebit.com/Contact about this?

Share this post


Link to post
Share on other sites
  • 0

I'll chime in here, as I've been fiddling around a lot with settings to find what's fastest for Plex library scans. I have 250MBit downstream and 50MBbit up.

Provider: GDrive (Gsuite account)
Cache size: I keep changing this based on other activities as I'm limited by quite a small SSD, 60GB for now.
Minimum download size: 5MB - We're running with high thread count small minimum DL size so that we have many threads for scanning that die fast when they've done their job.
Download threads: 15 - this lets Plex read a lot of files concurrently.
Prefetch trigger: 50MB - I want Plex to prefetch whenever it's doing a deep analyze of a file or when I'm watching something, but I don't want partial scans triggering prefetch.
Prefetch forward: 500MB
Prefetch time window: 30s (I don't really know the function of this setting, though)

For me, prefetch on / off doesn't really affect scans at all, scans don't trigger it for me at all at current settings - it kicks in when playing files from the cloud though, and I've been wondering if it would be a good idea to put it even higher as most of my files are around a lot larger than 500MB.

Share this post


Link to post
Share on other sites
  • 0
5 hours ago, red said:

Prefetch trigger: 50MB - I want Plex to prefetch whenever it's doing a deep analyze of a file or when I'm watching something, but I don't want partial scans triggering prefetch.
Prefetch forward: 500MB
Prefetch time window: 30s (I don't really know the function of this setting, though)

 

The prefetch trigger is how much data has to be read (sequentially) in the prefetch time window amount of time.  Then, it grabs the next prefetch forward amount of data. 

So, the larger the window, the longer the window for the trigger.  And the larger the trigger, the harder it is to trigger. 

Share this post


Link to post
Share on other sites
  • 0

I have a problem : i just uploaded my collection to stablebit cloud, with a gsuite account.

Well, the problem is that, when plex discovers the new movie, i have to wait until , i believe, it downloads the whole movie in cash, so it could identify it . I see stablebit starting to download alot, then it stops and plex can succesfully analyze the movie.

This is absurd and totally impossible for a large collection....

Solution ?

Thanks

Share this post


Link to post
Share on other sites
  • 0
On 12/21/2017 at 9:48 PM, Christopher (Drashna) said:

It may be.  OneDrive for Business is backed by sharepoint, which is known for performance issues. 

It's been requested in th the past, aind it's a feature request. 

As for an ETA, I can't even begin to comment on that. 

I actually like this feature request - being able to cache headers of your files up to X GB. It would for certain improve a lot of stuff in various usecases

Share this post


Link to post
Share on other sites
  • 0
11 hours ago, kye said:

Well, the problem is that, when plex discovers the new movie, i have to wait until , i believe, it downloads the whole movie in cash, so it could identify it . I see stablebit starting to download alot, then it stops and plex can succesfully analyze the movie.

This should not be the case. Check to make sure that this isn't a Plex configuration issue. If you have Plex set to automatically scan on a filesystem change, it should discover additions within minutes at the most. I have noticed that this feature does not seem to function correctly unless a full scan has been completed since the Plex sever software was last started, though. I think it's just buggy, but it's a Plex issue.

In any case, though, this is not a CloudDrive issue and it doesn't actually have to download the whole file to the cache in order for the file to be visible to applications or for Plex to identify it.

If it isn't a library scanning issue, it may be that you have Plex set to generate video thumbnails or something when new media is added, in which case the analysis won't be "complete" until it has done so, which requires that it read the entire file to generate those thumbnails. I personally recommend only using chapter thumbnails, at the most, if you want to store your media in the cloud. This feature can be turned on or off and adjusted in the Plex settings.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...