I often get asked if it's better to upgrade an ESX/ESXi 4.x host or just do a fresh install of ESXi 5.0. Before I answer this question lets review the differences between an upgraded host and a freshly installed host.
1. Boot disk partition format: When upgrading, the boot disk retains the older MBR (Master Boot Record) partition format where on a new install the boot disk is formatted as GPT (GUID Partition Table). The key here being that the new GPT format enables you to use LUNs larger than 2TB and up to 64TB. The larger LUN size support is typically not an issue for ESXi boot disks.
2. VMFS volume version: When you upgrade, the existing boot disk VMFS-3 volume is preserved. On a fresh ESXi 5.0 install a new VMFS-5 volume is created. Keep in mind that you can always upgrade the VMFS-3 volume to VMFS-5 after the host upgrade. To understand the implications of upgrading a VMFS-3 volume to VMFS-5 check out this blog.
The above two issues apply to both ESX and ESXi hosts. ESX hosts have one additional difference to consider:
3. Location of the ESXi scratch partition: When you do a fresh ESXi install a 4GB VFAT partition is created for the scratch partition (assuming you are installing to a local or SAN disk). However, when you upgrade, instead of a dedicated disk partition a scratch directory is created on the VMFS datastore. While the location of /scratch is different, there is no operational significance to using a VMFS directory compared to a disk partition. Also, keep in mind that you can always change the location of /scratch at any time.
With these considerations in mind, I don't see any disadvantages to upgrading a host compared to doing a fresh install. What's more, I think upgrading has advantages over doing a fresh install: (1) the upgrade preserves the host configuration eliminating the need to manually reconfigure each host, and (2) the data on the VMFS datastore is preserved eliminating the need to manually migrate data off the boot disk or having to rely on backup and restore tools. This not only helps to speed up the upgrade process, but also help reduce the risks of losing data or running into configuration errors while reconfiguring the hosts.
As such, I personally tend to recommend doing in-place upgrades, especially if you have a lot of ESX/ESXi hosts to upgrade. However, if you only have a small number of hosts and are comfortable doing the manual reconfiguration, doing a fresh install works as well. It's largely an issue of personal preference and which approach is convenient for you.
