Jump to content
Covecube Inc.

zeroibis

Members
  • Content count

    39
  • Joined

  • Last visited

  • Days Won

    1

zeroibis last won the day on October 12

zeroibis had the most liked content!

About zeroibis

  • Rank
    Advanced Member
  1. zeroibis

    Correct way to apply PrimoCache write cache

    Oh BAM I found what was off. I knew based on the CPU usage and SSD usage that they were not being loaded heavy and figured it was something with the network. Changed the nic to enable jumbo frames and bam dropped the transfer down to ~9min. I do not remember turning them on before but maybe they were enabled when the ati card was connected. I do know that for some reason the ati gpu drives would screw around with the network connections in general (for example with the ati gpu it would take an extra 45 seconds to establish a network connection and this issue never occurs when an nvidia card is connected instead). Very strange behavior but at least now I am back to 9min to transfer 94GB which is what I am looking for. Especially given previous best case test SSD to SSD was getting 8.5min.
  2. zeroibis

    Correct way to apply PrimoCache write cache

    No that was just the transfer time from the 960 EVO (1tb) to a 970 evo (250gb). The transfer directly to an nvme SSD represents the fastest possible transfer time. Actually recreating that test on my current system it takes over 10min now and I am not sure why that is. Only setting changed since then was enabling sata hot plug in bios and swapping a PCIe x16 ATI card (running pcie2.0 x4) for a x1 (3.0) nvidia card. Oh it could be that the NVME has now settled to its true performance because I wrote over 11TB to it earlier this week. (that shit is warn in now)
  3. zeroibis

    Correct way to apply PrimoCache write cache

    I realized that I never posted the SSD to SSD result here it is:
  4. zeroibis

    New Pool: Duplicate before adding data or after?

    I run quite a few VMs and generally best practice is that you want one VM drive that stores your OS and additional ones to store your files. In this way it is simple to spin up another VM of your OS and load your files over there. If you are not already doing so I would highly recommend splitting your OS vms from your data storage vm files. The risk with a VM lies in data corruption. In a normal system data corruption may cause a file to be lost. In a VM system file corruption may cause the VM which contains everything to be lost. This is a major risk and is why they are generally used on ZFS volumes and things like ECC ram are used. This is not to say you can not pace them on NTFS as the latest versions actually do a way better job of managing data corruption than in the past but this is where things like raid, snapraid and drivepool etc come in. These things are not just used to help mitigate risk from drive failure but from data corruption as well.
  5. Ah, I understand now that the best candidates are files such as virtual disks that literally contain duplicate files, makes sense. Thanks for the info!
  6. Interesting, also thanks for updating with the solution you found. Is there any material you would recommend to read up on the deduplication service. It appears this is a great solution if you have a lot of files that are only accessed but not often modified, such as large sets of videos and photos. I would still imagine that there is a heavy processing and memory cost to this that my present system likely can not pay but it is something to read up on and look into for future upgrades.
  7. Ah I understand and yes I had never heard of deduplication´╗┐ before and assumed it was a block level duplication service via storage spaces. I understand now that it is a form of compression. I would presume that the decompression would have a decent performance penalty but it sounds pretty great if you do not need a lot of read performance on old random files. So you have two disks each with deduplication´╗┐ and then both of these same disks added to the same drive pool with real time duplication enabled. I am guessing that MS has changed some APIs with regards to the way you access data from the volume that is effecting drivepools ability to maintain duplicates. A work around could be to temporarily real time duplication and instead have data flow to one drive and then to the second as an archive.
  8. I am a bit confused as to what your I/O stack looks like, can you draw a little map in paint or something so we know what is going on? Also a bit confused as to why your using MS deduplication ´╗┐instead of the duplication from DrivePool. One of the main benefits of DrivePool is the duplication feature which eliminates problems such as the exact one your having.
  9. zeroibis

    New Pool: Duplicate before adding data or after?

    You basically have the right idea. In essence all the files stored in the VMs virtual drive is in reality stored in a single vhdx/vmdk file which sits on whatever volume it is on. The issue this creates is that just like any single large file the vhdx/vmdk file can not exceed the size limit of the partition it is stored on.
  10. zeroibis

    New Pool: Duplicate before adding data or after?

    I think part of his problem was that he did not have any empty drives or only has a single drive that is currently empty. Yes copying to the x2 pool with real time duplication would be the fastest it would assume that he has already two blank drives which to my understanding he does not. If you do not have extra space you might need to do this task with duplication off and then after you clear the space by deleting the old VM files you can turn it back on. So you would need to setup the new machine running your ESXi VM without duplication, if your not going to have enough space, and then transfer the files over. Remember that as your VM's are individual files they can only rest on a single drive in a normal pool for example if you had 2x 2tb drives in a non-duplication pool you would have 4tb of free space but your vm can not exceed 2TB. This is different than a raid 0 array where your 2x 2tb drives would give you a 4tb limit for the size of your vm. Understanding this limitation is very important. When using drivepool no individual file (in your case the file that contains a VM drive) can never exceed the size of the physical disk it is placed on. You can get around this by mounting additional VM drives in your environment and splitting up the data. If you need vmdk files that individually exceed the size of your largest HDD then the only solution that is going to work for you is hardware or software based RAID. Note that you can take things like storage spaces and pool those. For example you could create two raid 0 arrays in storage spaces and then pool those to be a duplication pool. This makes recovery less complicated than a RAID 10 storage spaces implementation. Also keep in mind that duplication and RAID is NOT A BACKUP. Please ensure that you have backups before preceding. IF YOU DO NOT HAVE A BACKUP SOLUTION STOP NOW AND GET ONE.
  11. zeroibis

    New Pool: Duplicate before adding data or after?

    The data copy via duplication when you add a second drive to the pool is extremely fast. Nothing will transfer your data to a second drive faster than this method. Just be sure to enable duplication as soon as you add the second drive to the pool so that it will start copying the files rather than moving them to balance right away. Then you can also press the >> button so that it goes faster. From my own experience the transfer rates were very good and I was about to duplicate a few TB of data in half a day.
  12. zeroibis

    Correct way to apply PrimoCache write cache

    I have ordered a second 970 EVO and a x1 gpu so I will have those in later this week. I will re-test with that installed and I will split up the drives between the two caches as so to avoid the double read/write penalty. I am hoping that by using the ssd that will be using the PCH PCI 2.0 x4 connection as the cache for the HDDs that are also on the same PCH it will avoid any bandwidth issues. My understanding is that the PCH to CPU connection is at 32Gb/s. The x4 slot is at 20Gb/s and I also assume my integrated 10Gb/s nick is on there as well. So even with them all saturated I should not run into a PCH bottleneck on that side if I am lucky. Not running over-provisioned but I will also look into the benefits of that as well. Note though that I was using only a 111GB partition for the tests.
  13. zeroibis

    Correct way to apply PrimoCache write cache

    Here are the test results. ssd to raid 0: ssd to raid 0 cache 8kb buffer 60: ssd to raid 0 cache 8kb normal 60: ssd to raid 0 cache 32kb normal 60: ssd to raid 0 cache 128kb normal 60: ssd to raid 0 cache 512kb normal 60: ssd to raid 0 mirror cache 8kb normal 60: ssd to raid 0 mirror cache 512kb normal 60: ssd to raid 0 mirror cache 512kb normal 300:
  14. zeroibis

    Correct way to apply PrimoCache write cache

    I think it could be that my real limiting factor is that I could be using a faster SSD, from what I can see the larger sizes do have faster to 2x the read/write speeds compared to the 250gb 970 evo. I have a 960 EVO 1TB that I could toss in there and run all the benchmarks again. I will try that later and see how it goes. In the mean time I have gotten it down to 14min transfer times (3min improvement over no cache) using 512k blocks.
  15. zeroibis

    Correct way to apply PrimoCache write cache

    Actually it looks like that may not be the case; however, I have started testing and using it on a single raid 0 it is great it takes a 17min transfer down to 9min. I also found that normal was the fastest and that the size had no real impact on transfer times. Unfortunately, the double W/R penalty of using a single cache on a mirrored raid 0 was a disaster. You had everything working against it. First doubling the W/R rate killed most of the performance benefit of the SSD and then the double space requirement finished it off in the end. I will be posting images later of the benchmarks. Using the write cache allowed for a speed improvement of 1min for a transfer time of 16min vs 17min without the cache. I do believe there is a workaround although the best case workaround can not be achieved on Ryzen Your going to need threadripper. However, I will still attempt this solution in the future as I am sure it is better than nothing. What you need is a minimum number of SSDs equal to the number of duplication sets in drivepool. So if your using 2x real time duplication you need a minimum of 2 SSDs. Then you set each one to independently handle a different set of underline drives so they never get hit with the double penalty. I am still running some other tests and will post up all the results when they are finished. If the SSD cache for drive pool could resolve 1 single issue then it would be the best method. That issue is the size limit that you can not transfer more data than the size of the ssd cache. If that could be fixed then it would blow primocache out of the water. Also you might wonder about the non real time duplication issue when using the drivepool duplication cache but I realized that you can get around that by adding an existing duplication pool with real time duplication instead of the underline drives and do it that way. Basically: Pool with SSD cach -> Pool with Duplication -> underline volumes.
×