Horizon Agent: Installation fails at VM_BlastUDPReservePorts

I’m building customized and optimized Windows 10 images for my VDI, removing unnecessary stuff from the Windows installation (to name a few: unneeded drivers, retail/demo files, Hyper-V components). This reduces the disk and memory footprint. Once the customized Golden Image for the Linked Clones is finally installed in a virtual machine and optimized (with the OS Optimization Fling) accordingly, you need to install the VMware Horizon Agent at some point before rolling it out.

All preparations done. So guess what failed after days of optimization and dozens of Windows 10 image customizations? Yep, the VMware Horizon Agent installation…

Horizon Agent: The Rollback.

IMPORTANT: BEFORE you proceed, please notice that this is NOT officially supported by VMware! Do that at your own risk. If unsure, reach out to the support first.

Journey starts…

What does the logs say?

When paying the log file at %temp%\vmmsi.log some attention, you can find following:

[... Hosts entries gets added; PCoIP ports, etc are added to firewall using netsh successfully ...]
MSI (s) (F8:40) [11:41:11:056]: Executing op: CustomActionSchedule(Action=VM_BlastUDPReservePorts,ActionType=3073,Source=BinaryData,Target=VMBlastUDPReservePorts,CustomActionData=;16;VMware Blast Ausnahme UDP-Datenverkehr)
MSI (s) (F8:1C) [11:41:11:056]: Invoking remote custom action. DLL: C:\Windows\Installer\MSI7547.tmp, Entrypoint: VMBlastUDPReservePorts
CustomAction VM_BlastUDPReservePorts returned actual error code 1603 (note this may not be 100% accurate if translation happened inside sandbox)
Aktion beendet um 11:41:11: InstallFinalize. Rückgabewert 3.
MSI (s) (F8:40) [11:41:11:321]: Note: 1: 2265 2:  3: -2147287035 
MSI (s) (F8:40) [11:41:11:321]: User policy value 'DisableRollback' is 0
MSI (s) (F8:40) [11:41:11:321]: Machine policy value 'DisableRollback' is 0
[... Rollback of the whole installation ...]

In %temp%\vminst.log you can find similar like this:

vdmInstUtil: 03/03/20 07:04:00 Begin Logging
vdmInstUtil: 03/03/20 07:04:00 --- CA exec: VMBlastUDPReservePorts
vdmInstUtil: 03/03/20 07:04:00 --- CA exec: GetDynamicUdpPortValues
vdmInstUtil: 03/03/20 07:04:01 INFO: startingPort = 0
vdmInstUtil: 03/03/20 07:04:01 INFO: numPorts = 16384
vdmInstUtil: 03/03/20 07:04:01 Getting Property CustomActionData = ;16;VMware Blast UDP Traffic Exception
vdmInstUtil: 03/03/20 07:04:01 --- CA exec: FindAndReserveUDPPorts
vdmInstUtil: 03/03/20 07:04:01 INFO: CreatePersistentUdpPortReservation(0, 16, 0) returned: 87
vdmInstUtil: 03/03/20 07:04:01 ERROR: CreatePersistentUdpPortReservation() failed: The parameter is incorrect
vdmInstUtil: 03/03/20 07:04:01 ERROR: Failed to find a range of UDP ports to reserve.
vdmInstUtil: 03/03/20 07:04:01 End Logging
vdmInstUtil: 03/03/20 07:04:01 Begin Logging
vdmInstUtil: 03/03/20 07:04:01 --- CA exec: VMBlastUDPReleasePorts
vdmInstUtil: 03/03/20 07:04:01 Getting Property CustomActionData = VMware Blast UDP Traffic Exception
vdmInstUtil: 03/03/20 07:04:01 WARNING: missing blast udp port values. Exiting
vdmInstUtil: 03/03/20 07:04:01 End Logging
vdmInstUtil: 03/03/20 07:04:01 Begin Logging

What happens here?

During the installation of the Horizon Agent several firewall rules are added to the Windows Firewall. As we see above, apparently adding the firewall rule for required UDP ports of the Blast communication fails.

The error of interest is following:

CreatePersistentUdpPortReservation(0, 16, 0)

According to the Windows documentation:

IPHLPAPI_DLL_LINKAGE ULONG CreatePersistentUdpPortReservation(
  USHORT   StartPort,
  USHORT   NumberOfPorts,
  PULONG64 Token
);

So the installation wizard tries to create a firewall rule allowing UDP communication for total of 16 dynamic ports and starting with 0, so speaking 0-16. As you might know, port 0 is not valid and therefore this commands fails for obvious reasons.

When we check dynamic ports using netsh int ipv4 show dynamicport udp:

There we can see the starting port is 49152. When we apply the above logic, the reserved ports should be in fact 49152 – 49168. What you might also notice are the two first lines when running the above command:

The following helper DLL cannot be loaded: IPMONTR.DLL.
The following helper DLL cannot be loaded: IPPROMON.DLL.

Due to the image customization, some DLL libraries appears to be missing which are somehow required in the netsh command by default (Even the fact I haven’t noticed any kind of impact).

The issue & fix

The reason for this behavior relies in relevant parts which parses the StartPort of netsh during the installation: The thrown warning "The following helper DLL cannot be loaded" irritates the parsing part. That’s why the starting port defaults to 0 and the installation fails.

I believe this error is pretty rare… however in case you stumble over the same issue, I hope it helps!

There are two ways solving this situation:

1. Removing helper DLLs
You can simply remove the helper DLLs requirement from netsh to not show the warning and let the parsing succeed. However checking with Microsoft recommended before doing so.
Open a cmd.exe prompt as Administrator privileges and use following commands:

netsh delete helper ipmontr.dll
netsh delete helper ippromon.dll

2. Specifying start port
Alternatively there’s a way of specifying the StartPort when running the Horizon-Agent.exe installation. To do this, you have to check the starting port manually using netsh int ipv4 show dynamicport udp command and then start the installation, for instance, as shown below:

VMware-Horizon-Agent.exe /l /vBLAST_UDP_STARTING_PORT=49152