Quick Tip: NSX-T 3.0: Removing VIBs manually from ESXi host

Article automatically read by Amazon's Text-To-Speech Engine:

Due to some experiments with NSX-T 3.0 in my lab with ESXi 7.0 hosts I was in need removing the VIB files manually from the host, as the NSX Manager failed doing so.

Just to be very clear: This is NOT officially supported by VMware! Do that at your own risk. If unsure, reach out to the support first.

Should you ever be in similar situation, here a few takeaways and lessons learned so far:

1. If you think this is going to be a good idea… it’s not. As there are various resources configured during NSX-T installation and its configuration (like the VDS being configured/migrated in 7.0+ to a NSX Switch), there are dependencies of installed NSX components so that the ESXi network stack is able to load properly.
   • Without proper removal of NSX configuration from the host, you will end up in a ESXi installation without a working netstack – in other words: No network at all.
   • For example in /var/log/boot you can see failed to deserialize config data:
     
2. If it’s already late too and you ended up in a half-broken ESXi, you might want to consider clean installation. Otherwise there might be still one chance left: Get the NSX VIB package as the previous installed NSX-T version on that specific host, install the VIB and reboot.
   • You can either load the ESXi VIB before removal on a local datastore,
   • or craft a custom ISO file with the NSX-T module ZIP to mount over iDRAC/iLO/IPMI (to mount ISO on ESXi this might be helpful)

Still not scared enough? Here’s a very good read for removing NSX-T components I found very helpful: https://blog.zuthof.nl/2020/01/29/manually-removing-faulty-nsx-t-transport-nodes/

However, while above reference is still based on NSX-T 2.x, some VIBs have changed in NSX-T 3.0. Therefore you might want use the following order when removing the VIBs (due to the dependencies between each VIB packages):

esxcli software vib remove -n=nsx-adf
esxcli software vib remove -n=nsx-context-mux
esxcli software vib remove -n=nsx-exporter
esxcli software vib remove -n=nsx-host
esxcli software vib remove -n=nsx-monitoring
esxcli software vib remove -n=nsx-netopa
esxcli software vib remove -n=nsx-opsagent
esxcli software vib remove -n=nsx-proxy
esxcli software vib remove -n=nsx-python-logging
esxcli software vib remove -n=nsx-python-utils
esxcli software vib remove -n=nsxcli
esxcli software vib remove -n=nsx-sfhc
esxcli software vib remove -n=nsx-platform-client
esxcli software vib remove -n=nsx-cfgagent
esxcli software vib remove -n=nsx-mpa
esxcli software vib remove -n=nsx-nestdb
esxcli software vib remove -n=nsx-python-gevent
esxcli software vib remove -n=nsx-python-greenlet
esxcli software vib remove -n=nsx-python-protobuf
esxcli software vib remove -n=nsx-vdpi
esxcli software vib remove -n=nsx-idps
esxcli software vib remove -n=nsx-esx-datapath --no-live-install # <- required here
esxcli software vib remove -n=vsipfwlib
esxcli software vib remove -n=nsx-cpp-libs
esxcli software vib remove -n=nsx-proto2-libs
esxcli software vib remove -n=nsx-shared-libs

After removal it should look like: (some VIBs requires reboot, therefore still present)

[root@esxi04:~] esxcli software vib list | grep -Ei "nsx|vsipfwlib"
nsx-cpp-libs                   3.0.1.1.0-7.0.16556498               VMware   VMwareCertified   2020-07-22
nsx-esx-datapath               3.0.1.1.0-7.0.16556498               VMware   VMwareCertified   2020-07-22
nsx-proto2-libs                3.0.1.1.0-7.0.16556498               VMware   VMwareCertified   2020-07-22
nsx-shared-libs                3.0.1.1.0-7.0.16556498               VMware   VMwareCertified   2020-07-22

Then simply reboot the host and your NSX-T VIBs are gone. If something’s broken… I told you!