So I recently got a Synology NAS which I was intending to use to provide iSCSI LUNs out to my ESXi hosts. Upon initially configuring it I was having terrible performance and was ready to switch over to NFS. Googling this topic resulted in finding lots of posts about how a Synology-based iSCSI implmentation could not perform reasonably and NFS was much better for Synology-based datastores.
After taking some time to test out various different scenarios here is list of best practices I have come up with that seem to result in decent performance of Synology-based iSCSI LUNs.
- On your networking equipment:
- Do NOT use LACP or any form of link aggregation for the ports connected to the Synology NAS. iSCSI uses multipath I/O to take advantage of multiple interfaces on an array. LACP will slow things down because its not optimized for MPIO. As a side note, this same advice actually applies to hosting NFS 4.1 based datastores as well.
- In vSphere:
- On the ESXi host(s) do not use any form of link aggregation on the port groups of vmkernel interfaces communicating with the Synology NAS. If using more than one vmkernel to connect to the Synology NAS make sure each one is in its own dedicated port group that has only 1 Active network adapter uplinked to it (see below).
- In the vSphere Software iSCSI initiator:
- Configure and enable port binding for vmkernel interfaces communicating with the Synology array assuming they are both on the same subnet. If the vmkernels are on different subnets then port binding should not be used.
- Port Binding requires each vmkernel to be on a port group that uses only 1 active adapter.
- Add all Synology LAN interface IP addresses that will communicate with the ESXi host to the “Static Discovery” tab.
- Set the following settings on the “Advanced Options” tab:
- Login Timeout: 60
- Delayed Ack: <UN-CHECKED>
- On the ESXi host configure the pathing policy as “Round-Robin” on each storage device backed by a Synology iSCSI LUN.
- On the ESXi host configure Round-Robin IOPS limit to 1 for each storage device backed by a Synology iSCSI LUN.
- Use the following CLI command on each ESXi host to do this:
- for i in `esxcfg-scsidevs -c |awk ‘{print $1}’ | grep <NAA ID OF LUN>`; do esxcli storage nmp psp roundrobin deviceconfig set –type=iops –iops=1 –device=$i; done
- Example: for i in `esxcfg-scsidevs -c |awk ‘{print $1}’ | grep naa.6001405d8a35569d6f45d497fd8a87d9`; do esxcli storage nmp psp roundrobin deviceconfig set –type=iops –iops=1 –device=$i; done
- Reference Link: VMware KB2069356 – Adjusting Round Robin IOPS limit from default 1000 to 1
- Use the following CLI command on each ESXi host to do this:
- On the ESXi host disable VAAI ATS heartbeat.
- Method 1: Use the following CLI command on each ESXi host to do this:
- esxcli system settings advanced set -i 0 -o /VMFS3/UseATSForHBOnVMFS5
- Method 2: In the web interface on the ESXi host edit the Advanced System Settings. Set the following value:
- Property Name: VMFS3.UseATSForHBOnVMFS5
- Value: 0
- Reference Link: VMware KB2113956 – ESXi host loses connectivity to a VMFS datastore with VAAI ATS
- Reference Link: Synology Knowledge Base – How can I improve the performance or resolve disconnection issues of iSCSI LUNs?
- Method 1: Use the following CLI command on each ESXi host to do this:
- Install the Synology Storage Console for VMware virtual appliance and connect it to the vCenter with hosts using Synology-based iSCSI LUNs.
- In Synology DSM iSCSI Manager:
- Depending on your needs, configure the iSCSI LUN to be thick or thin provisioned. This really comes down to your particular requirements and needs.
- Thin provisioned is required if you wish to use DSM-based snapshots.
- Thick provisioning seems to deliver slightly better performance (~3-5% improvement). Not worth it unless absolute peak performance is super important to you.
- Make sure each iSCSI target has “Allow multiple sessions” checked under the advanced tab of the target.
- Make sure the network bindings tab has all Synology network interfaces that will communicate with the ESXi host checked. Set the I/O Queue Depth in the Settings section to 128.
- If using spinning disks in your Synology array disk pool(s) to host iSCSI LUNs:
- Use a SSD cache for any volumes hosting iSCSI LUNs. Use only one disk pool and one volume on the Synology array if possible for all iSCSI LUNs. The reason to create all Synology iSCSI LUNs in this volume is so they can all benefit from a single SSD cache. This minimizes the number of SSD caches that will be needed on the array overall.
- Depending on your needs, configure the iSCSI LUN to be thick or thin provisioned. This really comes down to your particular requirements and needs.
Notes: After tuning and testing I will admit that NFS performance still seems just a tad faster than iSCSI even after these optimizations In general though the performance was pretty similar and your not likely to notice the difference. Just so you know, I like NFS for ESXi datastores because its so simple to manage but IMHO Synology NFS shares need to be broken out into different volumes for each NFS share if you want the available space to be reported correctly for each NFS datastore in vSphere. Doing that means you need a seperate SSD cache for each volume and can be wasteful of SSD resources. iSCSI LUNs always report their correct storage space available and a single SSD read/write cache for them all seems fine in most cases.)