VMTN
Blogs
Twitter
Facebook
YoutubeScratch partition best practices for USB/SD booted ESXi?
Last week I received some questions around the best practice for sizing the scratch partition with USB/SD booted ESXi hosts. As the installable uses 4GB as a default value the assumption was made that this was the minimal required amount of diskspace for a host. As you can imagine when having 32 nodes in a cluster, but even with 10, using a default value of 4GB per host for a shared volume (VMFS/NFS) can become fairly expensive as SAN storage will need to be sacrificed for it.
Lets first explain where the 4GB is coming from, the partition type used for a normal install for the scratch partition is vfat. The maximum size of that partition type is 4GB and not entirely coincidentally the installer uses the same value. Main reason of course being to have room for future versions. This also implies that in many scenarios 4GB is more than likely not required.
How much would you need to reserve for a shared volume? We recommend to ensure alarms on this datastore are enabled and to monitor it closely so that it never fills up. We recommend using a 20GB volume, even in the case of a 32 node cluster this will suffice and in a cluster smaller than 15 nodes it will give room for future expansions without the need to change the size of the volume. Ensure each server has its own directory and set the scratch partition advanced setting accordingly.
The following KB Article provides multiple mechanisms for setting the location of scratch partition.
Posted by Duncan Epping on April 27, 2011 | Permalink |
Also to note that if the scratch partition has to go ramdisk, only a 512MB partition (and not 4GB) will be created. 512MB taken out of the server’s available physical memory.
Cheers,
Didier
.
Posted by: PiroNet | 04/28/2011 at 04:40 AM
Although this looks shiny/creamy/hi-tech - Running ESXi on a USB or SD cards, but in Production, we have seen many issues.
This filesystem is generally created on a RAM disk (a file on your USB drive, where your ESXi OS is running) for logging. Usually fills up when your HOSTD is backtracing.
I came across a KB from VMware although not in detail, you might want to refer:
http://kb.vmware.com/kb/1026028
To cleanup, you might want to delete the messages.X files and hostdstats file. And migrate the scratch location from default to a new folder created on your local/SAN VMFS datastore.
There were many budget reasons as well in bringing the ESXi with USB or SD. But altogether the setup was quite unstable. I would still recommend to dedicate disks for running OS.
Posted by: RAMESH GEDDAM | 04/28/2011 at 07:07 AM