Jump to content
Covecube Inc.
  • 0
superka

[Suggestion] Dual cache HDD.

Question

Hi everybody!

 

I've been using this fantastic software for 3 months ago and i really love it.

 

the only thing I miss is being able to configure the download cache different from the upload

Why?

With ftth 600/600 even the SSD Samsung pro 950 gets bottleneck if i copy big size files and the software is downloading and uploading at same time (upload verification)

 

It would be wonderful if you could use a mechanical hard drive for uploads and a solid hard drive for downloads. (also longer life for the SSD).

 

What do you think?

 

Javi,

 

 

Share this post


Link to post
Share on other sites

14 answers to this question

Recommended Posts

  • 0

How large are the files you are talking about?  Small enough to fit (mostly) in memory temporarily?  If so a RAM cache might help your downloading/uploading.  I use a 20GB L1 cache (using 3rd party software) with a variable write time delay, to help avoid bottlenecks like that.

The notion of specifying up/down caches for Clouddrive is interesting (I do like the idea), just not sure they could expose & split that feature between two separate physical drives.

Or...  perhaps it's time for a NVMe?   You could even use the NVMe drive as a L2 cache in front of your Samsung SSD, to speed up all read/write ops and even reduce writes with built-in trimming.

P.S.  I'm officially jealous of your transfer speeds!    :P   Still stuck on conventional 400/20 cable broadband here.  Fiber has been "coming" for a decade now, still not in sight!

 

Share this post


Link to post
Share on other sites
  • 0
 

How large are the files you are talking about?  Small enough to fit (mostly) in memory temporarily?  If so a RAM cache might help your downloading/uploading.  I use a 20GB L1 cache (using 3rd party software) with a variable write time delay, to help avoid bottlenecks like that.

The notion of specifying up/down caches for Clouddrive is interesting (I do like the idea), just not sure they could expose & split that feature between two separate physical drives.

Or...  perhaps it's time for a NVMe?   You could even use the NVMe drive as a L2 cache in front of your Samsung SSD, to speed up all read/write ops and even reduce writes with built-in trimming.

P.S.  I'm officially jealous of your transfer speeds!    :P   Still stuck on conventional 400/20 cable broadband here.  Fiber has been "coming" for a decade now, still not in sight!

 

 

i do have an NVMe as primary disk, im using the samsung 950 pro as only cache,

 

In 2 months i have over 10tb's of data writen (i lots of gb's of video 4k)

 

Mechanical disk is not fast enought to use all my bandwitch with data upload verification activated(and this is important for me)

 

Nope i don't mean this.

 

Everytime i copy something to my gdrive it just copies to the SSD, then it upload to the cloud. At the same time it beging to download in the same disk (to do the verification)

 

If some1 else of my network start to streaming the video, it just buffers and transfers speed to cloud decrease.

 

Thats why if we could use one hardrive to upload and another to download would be wonderful.

Share this post


Link to post
Share on other sites
  • 0
 

i do have an NVMe as primary disk, im using the samsung 950 pro as only cache,

In 2 months i have over 10tb's of data writen (i lots of gb's of video 4k)

Mechanical disk is not fast enought to use all my bandwitch with data upload verification activated(and this is important for me)

Well, a 64GB or 128GB RAM cache would probably solve the bottleneck issue.  But that's a bunch of money into RAM, and the motherboard to support it.

 

 

I think he means the ability to set the cache folder separately for reads and writes.  i.e. all reads would be cached on drive X, all writes would be cached on drive Y.  He's trying to increase throughput at the drive level since it's his current bottleneck for transfers.  Apparently the SSD just can't keep up with up/down at the same time.   :wacko:

Share this post


Link to post
Share on other sites
  • 0
 

Well, a 64GB or 128GB RAM cache would probably solve the bottleneck issue.  But that's a bunch of money into RAM, and the motherboard to support it.

 

I think he means the ability to set the cache folder separately for reads and writes.  i.e. all reads would be cached on drive X, all writes would be cached on drive Y.  He's trying to increase throughput at the drive level since it's his current bottleneck for transfers.  Apparently the SSD just can't keep up with up/down at the same time.   :wacko:

exactly.

 

2bb505906073e1dc66aaea7c2e61e4ca.png

 

 

 

485b6bb43417e59f757ec026fbfb8d6e.png

Share this post


Link to post
Share on other sites
  • 0

I think you are in a unique situation and no one else has the problem.  :huh:

Usually a network connection isn't going to outpace a SSD, and if it does there are other options to fall back on.  I use a 20GB L1 RAM cache in my system to avoid bottle-necking my drives (even a Samsung 850 Pro) and speed up all disk operations, out of 64GB total RAM.  In your case, if you have (or can install) 64GB or 128GB of memory and dedicate ~80% of that to a L1 RAM cache, you might avoid most of the issues you are seeing.  A RAM cache will go a long way to help eliminate unnecessary writes (live trimming), redundant reads, reads against written data, etc.

You might also consider limiting up/down transfer speeds with Clouddrive's bandwidth limiter in the meantime, so you don't choke the drive(s) with 100% activity.

Share this post


Link to post
Share on other sites
  • 0
 

I think you are in a unique situation and no one else has the problem.  :huh:

Usually a network connection isn't going to outpace a SSD, and if it does there are other options to fall back on.  I use a 20GB L1 RAM cache in my system to avoid bottle-necking my drives (even a Samsung 850 Pro) and speed up all disk operations, out of 64GB total RAM.  In your case, if you have (or can install) 64GB or 128GB of memory and dedicate ~80% of that to a L1 RAM cache, you might avoid most of the issues you are seeing.  A RAM cache will go a long way to help eliminate unnecessary writes (live trimming), redundant reads, reads against written data, etc.

You might also consider limiting up/down transfer speeds with Clouddrive's bandwidth limiter in the meantime, so you don't choke the drive(s) with 100% activity.

64GB of ram nowdays its really expensive for me , this would be the best option but is out of my budget.

 

I can also limit the up and down transfers... but its sad to do this when a software implementation can help the transfers.

 

What about who own a 1gbit simetric network? the problem will persist.

Share this post


Link to post
Share on other sites
  • 0
 

Everytime i copy something to my gdrive it just copies to the SSD, then it upload to the cloud. At the same time it beging to download in the same disk (to do the verification)

 

If some1 else of my network start to streaming the video, it just buffers and transfers speed to cloud decrease.

 

Then it sounds like the two issues that you're having is capacity, and IOPS.   A good SSD (and better, an NVMe drive) will fix the second issue.  But the capacity issue is something I can't really help with. But this also sounds like a case of "throw more money at the problem". 

Share this post


Link to post
Share on other sites
  • 0
 

Then it sounds like the two issues that you're having is capacity, and IOPS.   A good SSD (and better, an NVMe drive) will fix the second issue.  But the capacity issue is something I can't really help with. But this also sounds like a case of "throw more money at the problem". 

Samgun 970 Evo 500gb sounds good? or best to go with pro version?

Share this post


Link to post
Share on other sites
  • 0

Pro has much higher endurance, and would be the recommended one for higher usage.  Samsung SSD/NVMe Pro editions are top shelf.

But don't you already have a 950 Pro that's getting bottlenecked?  Or is the 970 you are considering purchasing a NVMe?

This site can be helpful when comparing NVMe's and SSDs for speed and pricing.

Share this post


Link to post
Share on other sites
  • 0
 

Pro has much higher endurance, and would be the recommended one for higher usage.  Samsung SSD/NVMe Pro editions are top shelf.

But don't you already have a 950 Pro that's getting bottlenecked?  Or is the 970 you are considering purchasing a NVMe?

This site can be helpful when comparing NVMe's and SSDs for speed and pricing.

850 pro (the sata one)

 

I've got 2 M.2 sockets, one runing pci 3.0 x4 and other one runing pci 2.0 x4.

Share this post


Link to post
Share on other sites
  • 0

Ah, nice.  Then you can put a decent NVMe in your second socket.  Given your circumstances, I'd go for a nice Samsung NVMe, whatever you can afford.  It should fix the bottleneck problem if it's large enough and on the faster socket.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×