One of the things to take into considerations when planning, designing or implementing an ESXi environment is what to do with the log files, how to audit ESXi and if you should allow people to log in.
Logging
Log files are the type of files that you will rarely need, but if you need them they better be available. ESXi is no different than any other solution out their, well maybe it is slightly different. Depending on the type of device you used for the installation the log files might not be retained after a reboot. Hopefully everyone has read my article about the scratch partition and created it, if the installer hadn't done that for you.
The log file structure for ESXi when compared to ESX is slightly different. Due to the fact that there is no Service Console there is also no need to have the same collection of files. With ESXi the following log files are used:
Path + Log File
Description
/var/log/messages
This log file includes the VMkernel, vmkwarning, and hostd logs
/var/log/vmware/hostd.log
Host Management service (hostd = Host daemon) log
/var/log/sysboot.log
System boot log
/var/log/vmware/aam/vmware_hostname-xxx.log
VMware HA log file
Of course ESXi offer syslogging capabilities and this is what we generally recommend using. Using a syslog server will make troubleshooting a lot easier as log files are stored in a central location.
Another thing that is possible, which many seem to forget, is to specify a location for the local logfiles. Now in this case the local logfiles don't need to be as local as it sounds. A best practice for environments without a syslog server would be to specify a remote VMFS datastore to ensure that if anything happens to the local storage the log files will still be available and there is at least a possibility you can do a root cause analysis. This is fairly straightforward and well explained in this KB article but I have copied it for your convenience:
To configure syslog for an ESXi host via GUI:
In the vSphere Client inventory, left-click the host.
Click the Configuration tab.
Click Advanced Settings under Software.
Select Syslog in the tree control.
In the Syslog.Local.DatastorePath text box, enter the datastore path to the file where syslog will log messages. If no path is specified, the default path is /var/log/messages. In addition if pointing at a datastore, ensure that the directory has been previously created.
The datastore path format is /vmfs/volumes/<datastore>/<folder>/filename
Note: You may have to reboot the host for the changes to take affect. We would also like to recommend to include the server name in the "folder" name.
Of course you can use the vMA as a syslog repository as described by Simon Long in this article. Other alternatives are for instance Splunk which will enable you to do event correlation between multiple "devices" where vMA is more aimed at ESXi only. Again, setting this up is fairly straight forward:
In the vSphere Client inventory, left-click the host.
Click the Configuration tab.
Click Advanced Settings under Software.
Select Syslog in the tree control.
In the Syslog.Remote.Hostname text box, enter the name of the remote host where syslog data will be forwarded. If no value is specified, no data is forwarded.
In the Syslog.Remote.Port text box, enter the port on the remote host where syslog data will be forwarded. By default Syslog.Remote.Port is set to 514, the default UDP port used by syslog. Changes to Syslog.Remote.Port only take effect if Syslog.Remote.Hostname is configured.
Click OK.
I cannot make the decision what type of tool, or even to use a tool, for you but I do recommend to ensure log files are stored outside the hypervisor itself.
Login and auditing?
ESXi is not designed for Shell access. Yes, there are ways of getting into the shell. But we generally recommend against loggin in for day-to-day operations. ESXi has been designed as an in-memory hypervisor that should be treated as an appliance.
The two main arguments for restraining yourself for loggin in to the ESXi shell are:
Memory Filesystem
Auditing
The first one is the obvious one. As ESXi lives in memory it requires memory to operate. Running scripts or custom binaries could mean that this has an impact on the in-memory filesystem. Who hasn't experienced this with ESX Classic where a roque script would fill up the filesystem? Believe me when I say that the result was never pretty.
The second one is less obvious but think about it for a second. ESXi is currently not setup for having non-root local users, but yes AD works indeed. (Check Harley Stagner's article on how to set it up) But what does that mean with regards to having an audit trail? Well unless you are using AD authentication for your ESXi environment you are limited. A better solution would be to use the vMA as it is designed to function as a management appliance. Of course alternatives are the vCLI and PowerCLI. All three provide auditing mechanisms and in the case of the vMA it can also be used to gather log-files which has been described in this excellent article by Simon Long.
