Jump to content

fly

Members
  • Posts

    32
  • Joined

  • Last visited

  • Days Won

    3

Reputation Activity

  1. Like
    fly got a reaction from jenesuispasbavard in PowerShell script to move files from SSD to magnetic by first in, first out policy   
    If you'd like to see the genesis of this script, check out my original thread here
    Since I finally got my PowerShell script running, and I thought I'd post it here in case anyone else might find it helpful.  
    SYNOPSIS: Script will move files from one DrivePool to another according to FIFO policy
    REQUIRED INFRASTRUCTURE: The expected layout is a DrivePool consisting of two DrivePools, one magnetic and one solid state. 
    The main variables are pretty obviously documented.  I added the file archive limit for people like me who also run SnapRAID Helper.  That way the script doesn't trip the 'deleted' file limit (I'm assuming moves would trip it, but I didn't actually test it).

    Warning, I've obviously only tested this on my system.  Please test this extensively on your system after you have ensured good backups.  I certainly don't expect anything to go wrong, but that doesn't mean that it can't.
    The code is full of on-screen debugging output.  I'm not a great coder, so if I've done anything wrong, please let me know.  I've posted the code here so that you can't C&P it into a script of your own, since Windows can be annoying about downloaded scripts.  Please let me know if you have any questions.

    Set-StrictMode -Version 1 # Script drivePoolMoves.ps1 <# .SYNOPSIS     Script will move files from one DrivePool to another according to FIFO policy .DESCRIPTION      The script can be set to run as often as desired.  The expected layout is a DrivePool consisting of two DrivePools, one magnetic and one solid state. .NOTES      Author     : fly (Zac) #> # Number of files to move before rechecking SSD space $moveCount = 1 # Path to PoolPart folder on magnetic DrivePool drive $archiveDrive = "E:\PoolPart.xxxxx\Shares\" # Path to PoolPart folder on SSD DrivePool drive $ssdSearchPath = "F:\PoolPart.xxxxx\Shares\" # Minimum SSD drive use percent.  Below this amount, stop archiving files. $ssdMinUsedPercent = 50 # Maximum SSD drive use percent.  Above this amount, start archiving files. $ssdMaxUsedPercent = 80 # Do not move more than this many files $fileArchiveLimit = 200 # Exclude these file/folder names [System.Collections.ArrayList]$excludeList = @('*.covefs*', '*ANYTHING.YOU.WANT*') # Other stuff $ssdDriveLetter = "" $global:ssdCurrentUsedPercent = 0 $fileNames = @() $global:fileCount = 0 $errors = @() Write-Output "Starting script..." function CheckSSDAbove($percent) {     $ssdDriveLetter = $ssdSearchPath.Substring(0, 2)     Get-WmiObject Win32_Volume |          Where-object {$ssdDriveLetter -contains $_.DriveLetter} |         ForEach {             $global:ssdUsedPercent = (($_.Capacity - $_.FreeSpace) * 100) / $_.Capacity             $global:ssdUsedPercent = [math]::Round($ssdUsedPercent, 2)         }          If ($ssdUsedPercent -ge $percent) {         Return $true     } Else {         Return $false     } } function MoveOldestFiles {     $fileNames = Get-ChildItem -Path $ssdSearchPath -Recurse -File -Exclude $excludeList |          Sort-Object CreationTime |         Select-Object -First $moveCount          If (!$fileNames) {         Write-Output "No files found to archive!"         Exit     }          ForEach ($fileName in $fileNames) {         Write-Output "Moving from: "         Write-Output $fileName.FullName         $destFilePath = $fileName.FullName.Replace($ssdSearchPath, $archiveDrive)         Write-Output "Moving to: "         Write-Output $destFilePath         New-Item -ItemType File -Path $destFilePath -Force         Move-Item -Path $fileName.FullName -Destination $destFilePath -Force -ErrorAction SilentlyContinue -ErrorVariable errors         If ($errors) {             ForEach($error in $errors)             {                 if ($error.Exception -ne $null)                 {                     Write-Host -ForegroundColor Red "Exception: $($error.Exception)"                 }                 Write-Host -ForegroundColor Red "Error: An error occurred during move operation."                 Remove-Item -Path $destFilePath -Force                 $excludeList.Add("*$($fileName.Name)")             }         } Else {             Write-Output "Move complete."             $global:fileCount++ # Increment file count, then check if max is hit             If ($global:fileCount -ge $fileArchiveLimit) {                 Write-Output "Archive max file moves limit reached."                 Write-Output "Done."                 Exit             } Else {                 Write-Output "That was file number: $global:fileCount"             }              }         Write-Output "`n"     } } If (CheckSSDAbove($ssdMaxUsedPercent)) {     While (CheckSSDAbove($ssdMinUsedPercent)) {         Write-Output "---------------------------------------"         Write-Output "SSD is at $global:ssdUsedPercent%."         Write-Output "Max is $ssdMaxUsedPercent%."         Write-Output "Archiving files."         MoveOldestFiles         Write-Output "---------------------------------------"     } } Else {     Write-Output "Drive not above max used." } Write-Output "Done." Exit
  2. Like
    fly reacted to Christopher (Drashna) in Drivepool SSD + Archive and SnapRAID?   
    Sorry for the delay!
    Seed it, just like a normal pool:
    http://wiki.covecube.com/StableBit_DrivePool_Q4142489
     
    And the rest looks good
  3. Like
    fly reacted to Jaga in Drivepool SSD + Archive and SnapRAID?   
    Makes sense - it would make the SSD cache drive more like a traditional cache, instead of simply a hot landing zone pool member which is offloaded over time.   +1
  4. Like
    fly got a reaction from Jaga in Drivepool SSD + Archive and SnapRAID?   
    Pretty please? 
    Just to reiterate my use case: My media server has quite a few users.  As new media comes in, it's likely to have a lot of viewers and therefore a lot of I/O.  As files get older, fewer and fewer people access them.  So I'd love to have an SSD landing zone for all new files, which then get archived off to platters as space requires.  It would be fantastic if I could say, once the SSD drive(s) reaches 80% full, archive files down to 50% using FIFO.
  5. Like
    fly reacted to Christopher (Drashna) in Drivepool SSD + Archive and SnapRAID?   
    Mostly just ask.  
     
  6. Like
    fly reacted to Christopher (Drashna) in Drivepool SSD + Archive and SnapRAID?   
    "dpcmd remeasure-pool x:"

  7. Like
    fly got a reaction from Jaga in Drivepool SSD + Archive and SnapRAID?   
    Well then, seems I have a weekend project. 
     
    Will report back with script in case anyone else would like to use it.
  8. Thanks
    fly reacted to Christopher (Drashna) in Drivepool SSD + Archive and SnapRAID?   
    Welcome!
×
×
  • Create New...