VMware vFlash Read Cache (VFRC)

VMware vFlash Read Cache was introduced in vSphere 5.5 and continue to be present in vSphere 6.x. It allows to leverage local SSD in each host as a cache. Note that no extra VIBs need to be installed on ESXi host or no other software. All the necessary modules are built-in the ESXi 5.5 and higher. In this post we will look at How to configure and How to Determine VMware vFlash Read Cache Block Size (VFRC).

But VMware is moving forward with new Framework called VAIO. VMware VAIO stands for “vSphere APIs for IO Filtering”. It’s a new API framework technology present in vSphere 6.0 (6.0 U1 more precisely) allowing vendors to present capabilities for caching and replication to individual VMs. It’s a framework, not feature. VAIO can be used not only caching but also for replication. Check further details in my post about VAIO Framework here.

But let's get back to VFRC which is present, it's bundled in vSphere at no extra costs. Let's explain some further details. Each local SSD configured for vFRC is formatted with special formatting with new filesystem called VFFS (Virtual Flash File System). This VFFS spans the cluster and forms a flash resource which can get consumed by VMs as a read cache.

Advantages of using vFRC is pretty obvious but in short it lowers application latency because the read requests instead of going all the way down to the storage, across all the physical network controllers,  storage network, storage controllers, spinning media… they just go to the vFRC to be acknowledged. It’s possible to use PCIe cards or Solid State drives (SSDs) with SATA, SAS and PCI Express interfaces.

vFRC is compatible with vMotion, HA, DRS…

1. Always migrate the cache contents
2. Do not migrate the cache contents

While the activation of the vFRS per host(s) is quite simple, the block size determination isn’t so obvious. And depending of the block size the performance gets better (or not). We will look at both in this post.

You might wonder how to configure a host, or multiple hosts for vFlash Read Cache?

Step 1. First enable the vFlash Read Cache on single host. Select host >Manage Tab > Virtual Flash Read Cache Resource Management > Add capacity

Then select available SSD , check the box and click OK

After few minutes you’ll see that in the the device backing empty space area your SSD will appear, and you’ll see the capacity informations and the file system used – VFFS.

You can check the Flash Read cache resource availability through the Summary Tab.

For multiple hosts you can follow this video from VMware

How to determine the vFRC Block Size?

To actually configure the Block size it’s done on per-VM level (per VMDK actually !!! ) as the workloads runing in each VM are different so the block size is different for each VM as well. You will need a VM with virtual hardware 10 at least (vSphere 5.5).

Quote:

Having the optimal cache block size is critical to overall perform ance of vFRC. Because the metadata structures for vFRC are indexed by cache block size, the metadata footprint size depends on the cache block size. For good performance, vFRC places its metadata in the memory and therefore the cache block size has a direct correlation with memory usage. The higher the cache block size, the lower the amount of metadata is required for indexing those blocks and therefore results in a smaller memory footprint. Consequently, a smaller cache block size consumes a bigger memory footprint.

The best way to choose the best cache block size is to match it according to the I/O size of the workload. VscsiStats may be used to find the I/O size in real time when running the workload.  This utility outputs an IOLength histogram that can be used to find the most dominant I/O size of the workload.  The cache block size of vFRC can be configured to match this value. In general ,
vFRC performs better if the cache block size either matches or is less than the I/O size of workloads. Find out more in the VMware communities message.

Run

/usr/lib/vmware/bin/vscsiStats -l

to get the VMs WorldGroup ID – in our case its 701464

Then start the vSCSI tool

/usr/lib/vmware/bin/vscsiStats -s -w <world_group_id>

shot:

After about 30 minutes vscsiStats will stop running… You can start it once again if you need to monitor during longer period…

/usr/lib/vmware/bin/vscsiStats -p <histo_type> [-c]

Few things to take in mind:

• It’s only write-through mode (read only) at the v 1.0 of vFRC
• The minimum supported block size is 1KB and  the maximum supported block size is 1,024KB.
• The Virtual Flash Read Cache is granularly allocated on a per-VMDK basis.
• The default Virtual Flash Read Cache configurable maximum size is 200GB, but a 400GB total maximum is supported
• To use a 400GB cache working set, the host’s advanced settings values require the modification of VFLASH.MaxCacheFileSizeMB to 409600 and VFLASH.MaxDiskFileSizeGB to 16384
• There is a threshold mechanism that is set to 90 percent of utilization. When the cache fill percentage reaches threshold, the eviction process is triggered

Monitoring via CLI:

esxcli storage vflash cache stats get –m <module> -c <cache file name >

What If I want to configure the host swap to SSD and use the same SSD drive?

Host will be swapping only if it used all the other memory optimization techniques are used already – Transparent page sharing (TPS), Memory ballooning, Memory compression.  But it can still happens if you heavily overcommit your memory. So you can do both, configure the SSD to partly use the space for vFRC and partly for Host cache. This is useful as you can allocate some small amount of SSD to host caching if you have hosts with RAM heavily overcommitted.

Select host > Manage > Settings > Virtual Flash Host Swap Cache configuration.  I hope that this article was helpful and you have found what you have been looking for.