Large Memory Pages settings configuration options in VMware vSphere

Are you thinking to configure large memory pages in VMware vSphere? There are few things that one has to keep in mind. Large memory pages will not be shared because chances of finding 2MB memory pages that are identical are much lower. Transparent page sharing kicks in only when there is a memory contention.

Transparent page sharing (TPS) then scans for identical pages. The scan does it on hosts VMs periodically (60 minutes by default). It reclaims identical 4KB pages (default setting). When there is a match, then those identical pages are not stored in physical memory, but only unique memory pages are.

Large pages enable applications to establish large memory regions. Memory address translations use translation lookaside buffers (TLB) inside the CPU. TLB is a cache that memory management hardware uses for speeding up virtual address translations.

Concerning a configuration of Large Memory Pages VMware vSphere 5.5 performance best practices document shows us that:

In addition to the usual 4KB memory pages, ESXi also provides 2MB memory pages (commonly referred to as “large pages”). ESXi assigns these 2MB machine memory pages to guest operating systems whenever possible; on systems with hardware-assisted MMU virtualization, ESXi does this even if the guest operating system doesn’t request them (though the full benefit of large pages comes only when the guest operating system and applications use them as well). The use of large pages can significantly reduce TLB misses, improving the performance of most workloads, especially those with large active memory working sets.

In addition, large pages can slightly reduce the per-virtual-machine memory space overhead.If an operating system or application can benefit from large pages on a native system, that operating system or application can potentially achieve a similar performance improvement on a virtual machine backed with 2MB machine memory pages. Consult the documentation for your operating system and application to determine how to configure each of them to use large memory pages.

But there is a gotcha too:

Use of large pages can also change page sharing behavior. While ESXi ordinarily uses page sharing regardless of memory demands, it does not share large pages. Therefore with large pages, page sharing might not occur until memory overcommitment is high enough to require the large pages to be broken into small pages. For further information see VMware KB articles 1021095 and 1021896.

Large memory pages are active by default in vSphere:

The Memory Page settings can be done on the advanced settings page. In vSphere web client select Host > Manage > Settings and type LPage into the filter box. The by selecting a particular row you can modify an option, but proceed with caution and know what you're doing.

AFAIK there hasn't been any change since vSphere 5.0 concerning large memory pages.

Other links and resources for Large Memory Pages settings:

• There is also a performance document called Large Page Performance at VMware – technical papers section. Check it out!
• vSphere resource management (PDF) document for ESXi 5.5 and vCenter Server 5.5
• Transparent Page Sharing (TPS) in hardware MMU systems http://kb.vmware.com/kb/1021095
• Use of large pages can cause memory to be fully allocated https://kb.vmware.com/kb/1021896