vCenter 7.0, Lifecycle Manager fails downloading depot metadata & proxy

Recently I’ve seen more cases where the Lifecycle Manager in vCenter 7 fails syncing updates and pulling depot metadata. In this case you see an error like: An error occurred while downloading depot metadata from https://hostupdate.vmware.com/software/VUM/PRODUCTION/addon-main/vmw-depot-index.xml..

Just to note, the usual disclaimer: This is provided as-is and must not be supported. If you want an official response, these instructions do not help or you require urgent help, raise an official case with VMware GS.

What you should expect:

• This post is intended for some quick checks.
• Most of the content is HTTP-PROXY-related.
• You are using vCenter 7.0 GA or vCenter 7.0 U1.

First, connect to the vCenter appliance via SSH (Enable SSH via VAMI if required, go into shell)

A few scenarios

Scenario 1

This scenario assumes following:

• Your environment requires a HTTP proxy to communicate to the outside world.
• Your proxy server is HTTP-only
• Your proxy needs no authentication

In vCenter 7.0 GA and 7.0 Update 1 the VAMI (vCenter Server Appliance Management Interface) does not allow you setting a HTTP-only-proxy for your HTTPS proxy:

But as the metadata is pulled from https://hostupdate.vmware.com (so HTTPS), the synchronization can fail and result in a timeout.

To verify:

1. This should work: curl -I http://hostupdate.vmware.com
2. This should fail: curl -I https://hostupdate.vmware.com
3. This should work as well: HTTPS_PROXY="http://your-proxy:8132/" curl -I https://hostupdate.vmware.com

The workaround is following:

1. Edit /etc/sysconfig/proxy via vi /etc/sysconfig/proxy
2. Modify HTTPS_PROXY to match HTTP_PROXY
3. Reboot the appliance so that all services pick up the new settings

Scenario 2

This scenario assumes following:

• Your environment requires a HTTP proxy to communicate to the outside world.
• Your HTTP and HTTPS proxy settings are correct and verified working
• Tests using curl and wget are both working as intended

In this case try following:

1. Output proxy configuration via cat /etc/sysconfig/proxy
2. Check if HTTP_PROXY or HTTPS_PROXY are having an leading slash. Examples:
   • Wrong: HTTPS_PROXY="http://your-proxy:8132"
   • Good: HTTPS_PROXY="http://your-proxy:8132/"
3. Reboot VCSA and verify if vLCM can pull updates now

Scenario 3

This scenario assumes following:

• Your environment requires a HTTP proxy to communicate to the outside world.
• Your proxy server is HTTPS-only
• Your proxy server requires authentication

For this scenario, please see more information in KB 80838 here

Scenario 4

This scenario assumes following:

• Your environment requires a HTTP proxy to communicate to the outside world.
• Proxy settings are configured.
• OVA deployment works when no proxy is configured
• OVA deployment fails with Invalid response code: 504, note that HTTP/s proxy is configured for the transfer.

In this case please raise a case with VMware GS.

Tips

Here we’re going to check basic network connectivity to the depot links…

While you can change proxy configuration and reboot after each change, this can be quite time-consuming and a challenge in a production environment. Luckily there’s an easier way by simply overriding environment variables of individual commands.

Check current proxy configuration file

Use cat /etc/sysconfig/proxy

Check with current settings in place of the VCSA:

When everything works, it should look like:

$ curl -I http://hostupdate.vmware.com
[...]
$ curl -I https://hostupdate.vmware.com
HTTP/1.1 404 Not Found
Accept-Ranges: bytes
Content-Length: 10
Server: AkamaiNetStorage
Date: Sat, 28 Nov 2020 02:08:39 GMT
Connection: keep-alive

Does it work when specifying a proxy manually?

Can we access the VMware-Host-Depot links using a proxy?

HTTP_PROXY="http://your-proxy:3128/" curl -I http://hostupdate.vmware.com
HTTPS_PROXY="http://your-proxy:3128/" curl -I https://hostupdate.vmware.com

Certain domains must not use proxy

In certain environments several services are not available via proxy but on the local network, thus no proxy settings should be used. In this case you can edit setting NO_PROXY in /etc/sysconfig/proxy.

Tip: Add FQDN and IP addresses, just to make sure. To apply changes, reboot.