We've released a new version of ESX 4.0 Update 1 that resolves the issue with the ESX 4.0 Update 1 install failing, timing out and resulting in the host entering an usable state.
The new version is called ESX 4.0 Update 1A.
You can find the new version on VMware Downloads.
For more details, see Due to a possible dead lock on rpmpd, upgrading ESX 4.0 to ESX 4.0 Update 1 can fail or time out and leave the host in an unusable state (1016070).
