VMware VDP quiesce problems of Windows 2008 R2 Guest OS

VDP cannot backup Windows 2008 R2. Are you having troubles Backing up Windows 2008R2 with VMware vSphere Data Protection (VDP)? Quiesced snapshots don't work? It's probably that you're using the VMware VSS to quiesce the guest OS.  It's a known issue which can be easily avoided by using the VSS providers which are part of your backup tool, if of course the tool provides that.

Update: The issue has been resolved in the latest release of VDP 5.1.10 – read VMware vSphere 5.1 Update 1

One of those tools, which I'm aware of, and which does good job on it, is Veeam Backup. I thought that everyone was aware of problems of quiescing of Windows 2008 R2 guest OS with built in VMware VSS, and I was not thinking about posting an article about it. But not quite yet.

In the latest Veeam Community Forums Digest E-mail notifications by Anton Gostev, that I'm receiving. I'm also the part of the Veeam community forums, and receiving the newsletter where Anton describes the issue, that I'm aware of it:

it affects all backup solutions which leverage VMware VSS integration instead of providing their own. Veeam B&R has had proprietary VSS integration since version 2 (particularly due to lack of functionality and limitations of native VMware VSS intergration).

So the resolution?

As concerning Veeam, the solution is simple:

be sure to enable application-aware processing enabled in Veeam job settings for it to be used. If instead you enable VMware Tools quiescence in the advanced job settings, you will be affected by the issue as well.

You can get a free trial from Veeam or start with the free “VeeamZIP” in order to test in your environment.

If you're using other backup products, you will need to look at the configuration of the specific backup solution to solve the issue, if you're affected.

Usually the error looks like this:

“Can not create a quiesced snapshot because the create snapshot operation exceeded the time limit for holding off I/O in the frozen virtual machine”.

The VDP Administration Guide PDF (page 52) recommends disabling the disk UUIDs in the VMX config file (setting the value to FALSE).  This fixes the problem with quiescing but apparently this causes the quiescing not being application consistent:

by disabling disk UUIDs you turn off application consistent VSS snapshots, which might be an issue if you require application consistent backups

See this VMware Forum post.

There is one other method that I found (but not tested personaly) that is available through bluemonkey.com blog:

Here is a quick quote from the source:

01. Shutdown your VM, edit the advanced settings and change “disk.enableUUID” to “false” per VMware’s instructions. https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2035736

02. Power the VM back on.

03. Download the Windows Server 2008 SDK and install it on your PC. https://www.microsoft.com/en-us/download/details.aspx?id=11310

04. Included with the SDK is a program called vshadow. There is a version for x86 (vshadow_x86.exe) and x64 (vshadow_x64.exe) installs. Pick the appropriate one for your install and copy it to the C: drive of your VM. More info on vshadow here: https://msdn.microsoft.com/en-us/library/windows/desktop/bb530725(v=vs.85).aspx

05. Navigate to your VMware Tools folder and create a folder called “backupscripts.d”

06. Create a batch file named “freeze.bat”

07. In the batch file, enter a single line “C:\vshadow_xxx.exe c:” (replace xxx with x86 or x64 for which version you are using). You will need to add each drive on which you want to stop I/O to the end of the line, ex: “C:\vshadow_x64.exe c: e: f:”.

08. Create your VDP job and run it!

If you're using other methods, don't hesitate and share it through the comments section.