In this blog post, I want to share some of the most used best practices when working with storage arrays and VMware ESXi hosts, using PowerCLI. These configurations, are in particular for AFAs on iSCSI.
Is not my intention to explain the benefits and behaviors when these settings are modified (maybe in another blog post), just the PowerCLI script reference and steps. PowerCLI is extremely useful to work with larges environments (clusters). If you are setting a single ESXi, I’d use ESXCLI commands via an SSH Terminal, my personal preference.
Note: These are general recommendations from VMware and most of the storage (AFA) vendors. But you need to take into account that every vendor may suggest a different configuration. Also, are specific scenarios in which these best practices might not apply.
I took some of these scripts around the web and modified most of them. All these best practices could be written and run in a single script, but because of what I mentioned before, I prefer to keep each script separated, so you can test individually and troubleshoot if needed.
PowerCLI: Install and connect
Install PowerCLI
1. Open the Windows PowerShell console.
2. Install all official PowerCLI modules, running the following command.
Install-Module VMware.PowerCLI -Scope CurrentUser
Connect to the Cluster
1. Open the Windows PowerShell console.
2. Replace the bolded text and run the following command to connect to your vCenter Server.
connect-viserver -server Your_vCenter_IP -user Your_User -password Your_Password
Best Practices with PowerCLI
Change the Path Selection Policy to Round Robin
Current iSCSI LUNs
1. Connect to the vCenter Server using PowerCLI.
2. Replace Cluster_Name, copy and run the commands below.
$AllHosts = Get-Cluster Cluster_Name | Get-VMHost | where {($_.ConnectionState -like "Connected" -or $_.ConnectionState -like "Maintenance")}
foreach ($esxhost in $AllHosts) {Get-VMHost $esxhost | Get-ScsiLun -LunType disk | Where-Object {$_.CanonicalName -like "eui.*"} | Set-ScsiLun "RoundRobin" | Select-Object VMHost, CanonicalName, MultipathPolicy}
For new LUN Creation
With this option you will set all ESXi hosts to automatically configure Round Robin to all the new recognized Volumes/LUNs.
1. Connect to the vCenter Server using PowerCLI.
2. Replace Cluster_Name and run the commands below.
$AllHosts = Get-Cluster Cluster_Name | Get-VMHost | where {($_.ConnectionState -like "Connected" -or $_.ConnectionState -like "Maintenance")}
foreach ($esxhost in $AllHosts) {$esxcli = get-esxcli -vmhost $esxhost; $esxcli.storage.nmp.satp.set($null,"VMW_PSP_RR","VMW_SATP_ALUA")}
Round Robin I/O Operations Limit
Change the Round Robin I/O Operation Limit from 1,000 to 1. From the latest VMware Best Practices, VMware considers the default setting is enough to provide good performance.
1. Connect to the vCenter Server using PowerCLI.
2. Replace Cluster_Name and run the commands below.
$AllHosts = Get-Cluster Cluster_Name | Get-VMHost | where {($_.ConnectionState -like "Connected" -or $_.ConnectionState -like "Maintenance")}
foreach ($esxhost in $AllHosts) {Get-VMHost $esxhost | Get-ScsiLun -LunType disk | Where-Object {$_.Multipathpolicy -like "RoundRobin"} | Set-ScsiLun -CommandsToSwitchPath 1 | Select-Object VMHost, CanonicalName, MultipathPolicy, CommandsToSwitchPath}
Changing Disk.MaxIOSize to 4,096KB
1. Connect to the vCenter Server using PowerCLI.
2. Replace Cluster_Name and run the command below.
$AllHosts = Get-Cluster Cluster_Name | Get-VMHost | where {($_.ConnectionState -like "Connected" -or $_.ConnectionState -like "Maintenance")}
3. Set the Disk.DiskMaxIOSize value to 4096 with the command below.
foreach ($esxhost in $AllHosts) {$esxcli = get-esxcli -vmhost $esxhost; $esxhost | get-AdvancedSetting -Name "disk.diskmaxiosize" | set-AdvancedSetting -Value "4096" -Confirm:$false | Select-Object Name, Value}
Set Login Timeout to 30
1. Connect to the vCenter Server using PowerCLI.
2. Replace Cluster_Name and run the commands below.
$AllHosts = Get-Cluster Cluster_Name | Get-VMHost | where {($_.ConnectionState -like "Connected" -or $_.ConnectionState -like "Maintenance")}
$hba = Get-VMHostHba -VMHost $esxhost -Type iScsi | Where-Object {$_.Model -like "iSCSI Software Adapter"}
3. Set the LoginTimeout value of the iSCSI Software Adapter to 30 with the command below.
foreach ($esxhost in $AllHosts) {$esxcli = get-esxcli -vmhost $esxhost; $esxcli.iscsi.adapter.param.set($hba.device,$false,'LoginTimeout','30')}
Disable DelayedAck
1. Connect to the vCenter Server using PowerCLI.
2. Replace Cluster_Name and run the commands below.
$AllHosts = Get-Cluster Cluster_Name | Get-VMHost | where {($_.ConnectionState -like "Connected" -or $_.ConnectionState -like "Maintenance")}
$hba = Get-VMHostHba -VMHost $esxhost -Type iScsi | Where-Object {$_.Model -like "iSCSI Software Adapter"}
3. Disable the DelayedAck value of the iSCSI Software Adapter with the command below.
foreach ($esxhost in $AllHosts) {$esxcli = get-esxcli -vmhost $esxhost; $esxcli.iscsi.adapter.param.set($hba.device,$false,'DelayedAck', '0')}
VAAI
VAAI is enabled by default. You can check the VAAI status, enable and disable VAAI with PowerCLI.
1. Connect to the vCenter Server using PowerCLI.
2. Replace Cluster_Name and run the commands below.
These commands will call the primitives XCOPY, WRITE_SAME, and ATS, respectively.
VAAI status
Get-Cluster Cluster_Name | Get-VMHost | Get-AdvancedSetting -Name DataMover.HardwareAcceleratedMove
Get-Cluster Cluster_Name | Get-VMHost | Get-AdvancedSetting -Name DataMover.HardwareAcceleratedInit
Get-Cluster Cluster_Name | Get-VMHost | Get-AdvancedSetting -Name VMFS3.HardwareAcceleratedLocking
Enable VAAI
Get-Cluster Cluster_Name | Get-VMHost | Get-AdvancedSetting -Name DataMover.HardwareAcceleratedMove | Set-AdvancedSetting -Value 0 -Confirm:$false
Get-Cluster Cluster_Name | Get-VMHost | Get-AdvancedSetting -Name DataMover.HardwareAcceleratedInit | Set-AdvancedSetting -Value 0 -Confirm:$false
Get-Cluster Cluster_Name | Get-VMHost | Get-AdvancedSetting -Name VMFS3.HardwareAcceleratedLocking | Set-AdvancedSetting -Value 0 -Confirm:$false
Disable VAAI
Get-Cluster Cluster_Name | Get-VMHost | Get-AdvancedSetting -Name DataMover.HardwareAcceleratedMove | Set-AdvancedSetting -Value 1 -Confirm:$false
Get-Cluster Cluster_Name | Get-VMHost | Get-AdvancedSetting -Name DataMover.HardwareAcceleratedInit | Set-AdvancedSetting -Value 1 -Confirm:$false
Get-Cluster Cluster_Name | Get-VMHost | Get-AdvancedSetting -Name VMFS3.HardwareAcceleratedLocking | Set-AdvancedSetting -Value 1 -Confirm:$false