4k Streaming on Plex - Playback issues - Detailed Post

[NMonky]

I have tried to do as much troubleshooting as possible, but cannot get 4k movies to play properly. My setup is as follows:

1000/500mbps Connection

Drivepool 'Cloud' H: - Plex picks up all the media from H: which consists of:

1. Drivepool 'Cloud SSD' - 2 x 4TB HDDs pooled together - data goes here first before being moved to the 'Cloud Drives'
2. Drivepool 'Cloud Drives' - 3 x 100TB Cloud Drives - Only filled to around 5TB each

Settings on the Cloud Drives are all as follows:

Cache Drives are all 100GB Expandable on an SSD that is not performing any other functions.

The issue is that 4k movies play fine for a few minutes and then have buffering issues, sometimes just for around 1 to 2 seconds. Sometimes, the video stutters and becomes pixelated. I am playing the videos on a Samsung Smart TV.

Troubleshooting I have already done:

- Tested video files, network, and TV. When played off either the 'Cloud SSD' (which is actually 2 x HDD) or my 1TB SSD the files play perfectly on my Samsung TV via Plex - it is not the files, nor the hard drives, nor my network.

- Tested Plex. I get the same results from Emby and when playing the video directly on my computer - it is not Plex

- Check the files are direct playing - they are

- Run speedtest - I am getting the full bandwidth from my provider

- Disabled all other programs so that my system and SSD have nothing else to deal with, including disabling all Plex scans - It is not a lack of system resources

Things I have noticed:

- My system seems to be prefetching enough data - it doesn't seem to go below 300mb, sometimes going to 600mb

- I rarely, if ever, get more than 1 new connection to the CloudDrive show up when playing movies. When uploading I get more.

- My download speed seems to be quite slow in general - around 5MB/s with peaks and troughs when transferring data. My upload speed is around 10 MB/s normally, so I'm not hitting anywhere near what my bandwidth is capable of.

- 1080p almost always works perfectly

 

Please let me know if there is any more information I can give to help. I really want to get this working as it should.

Thanks

[srcrist]

Some of this may seem counter-intuitive, but try this: Drop your download threads to ten, drop your prefetch window to 150-175MB, raise your prefetch trigger to 15MB and drop the window to 10 seconds. Try that, test it, and report back.

Also, what is your drive structure? Most importantly, what is the chunk size that you specified at drive creation?

Also, to rule out one more factor, have you tried playing the files off of the CloudDrive with a local media player like VLC?

[srcrist]

1 minute ago, NMonky said:

Will test the above. Quick question; what exactly do you mean by drive structure? 

Chunk size was the highest allowed. I think that was 20.

I mostly meant the chunk size, but drive structure would be things like chunk size, cluster size, sector size, file system type. The things you chose when you created the drive. But really only the chunk size is relevant here. 

[NMonky]

1 minute ago, srcrist said:

Some of this may seem counter-intuitive, but try this: Drop your download threads to ten, drop your prefetch window to 150-175MB, raise your prefetch trigger to 15MB and drop the window to 10 seconds. Try that, test it, and report back.

Also, what is your drive structure? Most importantly, what is the chunk size that you specified at drive creation?

Will test the above. Quick question; what exactly do you mean by drive structure? 

Chunk size was the highest allowed. I think that was 20.

[NMonky]

56 minutes ago, srcrist said:

I mostly meant the chunk size, but drive structure would be things like chunk size, cluster size, sector size, file system type. The things you chose when you created the drive. But really only the chunk size is relevant here. 

Got it. Yeah, can confirm chunk size is 20.

I'm going to try and test it tonight, will definitely be getting back to you ASAP. 

Thanks for taking the time to help out, appreciate it. 

[NMonky]

1 hour ago, srcrist said:

Some of this may seem counter-intuitive, but try this: Drop your download threads to ten, drop your prefetch window to 150-175MB, raise your prefetch trigger to 15MB and drop the window to 10 seconds. Try that, test it, and report back.

Also, what is your drive structure? Most importantly, what is the chunk size that you specified at drive creation?

Also, to rule out one more factor, have you tried playing the files off of the CloudDrive with a local media player like VLC?

Just to add, regarding VLC, yes I have. It is marginally better, but still has similar issues. Again, playing off HDD or SSD it has no issues. Playing off Plex Media Player for Windows is slightly less marginally better. 

[NMonky]

19 hours ago, srcrist said:

Some of this may seem counter-intuitive, but try this: Drop your download threads to ten, drop your prefetch window to 150-175MB, raise your prefetch trigger to 15MB and drop the window to 10 seconds. Try that, test it, and report back

Okay, I did some quick tests last night. This has definitely moved us in the right direction. There are still some issues with pixelation and a little bit of buffering, but they are definitely less pronounced. Are there any setting that you would recommend tweaking further?

Also, I noticed that one of my three Cloud Drives is set to 10mb chunks instead of 20. Is there any quicker and easier way to fix this without having to download all the files and reupload them?

[srcrist]

10 hours ago, NMonky said:

Okay, I did some quick tests last night. This has definitely moved us in the right direction. There are still some issues with pixelation and a little bit of buffering, but they are definitely less pronounced. Are there any setting that you would recommend tweaking further?

Also, I noticed that one of my three Cloud Drives is set to 10mb chunks instead of 20. Is there any quicker and easier way to fix this without having to download all the files and reupload them?

So, to be clear, there isn't anything that CloudDrive can do that should ever really cause pixelation. That just isn't how digital video works. Buffering, maybe. But not pixelation. That would be a Plex thing, and a result of how Plex is encoding or decoding the video. The drive throughput is either going to work or not. It's going to be fast enough to keep up, or not. It won't pixelate the video, it just won't work. I suppose Plex, as a client, might display pixelation if it isn't able to get data fast enough, as a sort of stop-gap to keep the stream going--so that might be why you're seeing that. 

The settings I gave you were the ones I use, and am able to stream 4K video with at original quality. So that's effectively all the settings we can tweak within CloudDrive--at least in the short term. There are some tweaks we might be able to make to optimize for you later, but those should be working and they're a good starting point for your connection speed.

At this point, it's probably best to start looking at the network, if you've ruled out the possibility that there is some issue with the file. Play with your DNS servers and see if you're being sent to a non-optimal Google server, and check your network settings across the board. You should easily be able to hit 400mbps or so with the CloudDrive settings I gave you. If you aren't, there is something wrong.

As far as the maximum chunk size goes, there isn't any way to change that without recreating the drive. Though you may want to consider that anyway, as you cannot use chkdsk on volume sizes of 100TB, and it's a lot less optimal to use 3 separate CloudDrive drives than 1 drive partitioned into multiple volumes. In fact, if all three of those drives are all caching from the same SSD, that might be the cause of all of your issues here, as well. The cache can be very disk usage heavy, and I don't generally recommend any more than a single virtual drive caching to a single physical drive. I had problems with that myself. Note that a single 1PB CloudDrive drive can be divided up into many volumes, and all can share a single cache. It's probably better to make these changes now, while there is still relatively little on the drives, than later, when it can take months. Each volume should be smaller than 60TB to work with Volume Shadow Copy and CHKDSK. 

[NMonky]

11 hours ago, srcrist said:

 I suppose Plex, as a client, might display pixelation if it isn't able to get data fast enough, as a sort of stop-gap to keep the stream going--so that might be why you're seeing that.

I believe this is the case. My description wasn't very good, the video stutters, as if it is not quite receiving the data quick enough.

 

11 hours ago, srcrist said:

At this point, it's probably best to start looking at the network, if you've ruled out the possibility that there is some issue with the file. Play with your DNS servers and see if you're being sent to a non-optimal Google server, and check your network settings across the board. You should easily be able to hit 400mbps or so with the CloudDrive settings I gave you. If you aren't, there is something wrong.

Okay, I am currently using Cloudflare's 1.1.1.1, I will switch it to Google's DNS service and see if that helps at all.

 

11 hours ago, srcrist said:

As far as the maximum chunk size goes, there isn't any way to change that without recreating the drive. Though you may want to consider that anyway, as you cannot use chkdsk on volume sizes of 100TB, and it's a lot less optimal to use 3 separate CloudDrive drives than 1 drive partitioned into multiple volumes. In fact, if all three of those drives are all caching from the same SSD, that might be the cause of all of your issues here, as well.

Understood. I have reduced the size of the volumes to 30TB and I have begun the process of consolidating all the drives together. My plan is just to let the DrivePool balancers take care of it. It will take a few weeks I guess, but hey-ho, such is life. I am looking for long term stability, so a few weeks is fine. Or, is there a more efficient method you would recommend?

My plan now is to wait for a few weeks for it all to consolidate onto one CloudDrive, use the other CloudDrives for duplication in case one fails, switch to Google DNS, and once all this is done get back to you and let you know if it is all working or not. If not, it will be time to start looking at my network. Are you happy for me just to comment on this thread again in a few weeks time?

Also, most importantly, thanks again for your help with all this. If you're ever in Kuala Lumpur let me know, I owe you a beer.

 

[srcrist]

6 hours ago, NMonky said:

Okay, I am currently using Cloudflare's 1.1.1.1, I will switch it to Google's DNS service and see if that helps at all.

 

Sure thing. And also do not be scared to simply use the DNS servers provided by your ISP to test them. They're likely to be the most local servers for you, and the most likely to provide you with efficient routing to Google--despite the other flaws with using ISP provided DNS.

 

6 hours ago, NMonky said:

Or, is there a more efficient method you would recommend?

No, I think that's fine. I used DrivePool to consolidate when I recognized this problem myself. You can always click the little arrows next to the status bar in the DrivePool UI to give the transfer a higher priority, if you're doing other things on the drives. That may help speed up the process somewhat.

 

6 hours ago, NMonky said:

My plan now is to wait for a few weeks for it all to consolidate onto one CloudDrive, use the other CloudDrives for duplication in case one fails, switch to Google DNS, and once all this is done get back to you and let you know if it is all working or not.

Just make sure that they aren't all caching on the same physical drive. Move the other two to another cache drive, even if you're only using them for duplication.

 

6 hours ago, NMonky said:

Are you happy for me just to comment on this thread again in a few weeks time?

That's fine.

 

6 hours ago, NMonky said:

Also, most importantly, thanks again for your help with all this. If you're ever in Kuala Lumpur let me know, I owe you a beer.

 

Any time.

[JulesTop]

Hi, I have a similar situation...

My OS is on my SSD, and I am using the same SSD as the cache for my 3 clouddrives. However, two out of the three are used for duplication. Is there a way to select one of them as the drive that the data gets read from all the time... because right now, although it looks like the data is always pulled from the same drive, I have no way of guaranteeing it (as far as I can tell).

[srcrist]

8 hours ago, JulesTop said:

Hi, I have a similar situation...

My OS is on my SSD, and I am using the same SSD as the cache for my 3 clouddrives. However, two out of the three are used for duplication. Is there a way to select one of them as the drive that the data gets read from all the time... because right now, although it looks like the data is always pulled from the same drive, I have no way of guaranteeing it (as far as I can tell).

That's a good question, and one probably better asked in the DrivePool forum, because it's really a DrivePool question. I'm not sure if there is any way to tell DrivePool to prefer reads off of a specific drive. I know that it's smart enough to prefer local drives over cloud drives if the same data exists on both, but I'm not aware, off the top of my head, of a way to tell it to prefer a specific drive that falls within the same tier of the built-in hierarchy. That doesn't mean it doesn't exist, though.

One work-around that I can think of would be to simply point your applications at the hidden folder on the drive, rather than pointing them at the pool itself. That would work.