This is part 2 of this post. Part one can be found here.
As we discussed in the previous blog post, and white papers networking latency is significantly lower than typical flash device latency, however there are some new exception cases that have come up as the Virtual SAN platform has been extended.
Client Cache – In Virtual SAN, there have always existed small DRAM based read ahead cache containers that were relatively small. In 6.2 these caches were extended and consolidated to use .4% up to 1GB of DRAM per host. This cache uses system memory which has incredibly low latency ~15ns which is significantly lower than modern low latency ethernet ~350ns. In this case the cache is stored local to the virtual machines current execution. Given the small sizes of these caches re-warming them after vMotions is a low impact (it is signifigantly less memory than most virtual machines contain, and orders of magnitude smaller than the size of the virtual machines full active set) this does not carry the same negative performance consistency problems of migrating the entire working set.
Stretched Clusters – In a stretched cluster the second site can be up to 5ms from the primary site. As this network latency can impact read performance all reads will come only from the local site. For stretched clusters with very low latency capability that need to run as active/active or leverage both sites cache this behavior can be disabled. Two node systems are effectively a small form of stretched cluster. Depending on the network latency between the nodes, local reads may or may not be beneficial. Jase explains how we can optimize this for low latency, or for higher network latency.
Virtual SAN leverages data locality when the latency impact of the network would be important and prioritizes consistency of performance when it does not. Virtual SAN continues to improve and change to meet the needs of customers.
