Caching is an overloaded term, with two working definitions. One definition describes a cache as a place to stash data. In this way, a cache is the implementation of a datastore that serves as the system of record. The second definition describes a cache as a datastore that increases the performance of an app's data requests by holding copies of needed data. The retrieval of those copies from the cache is fast, while the original data remains in a slower datastore, typically a disk-based storage system.
Tanzu GemFire is an in-memory, key-value store that can be used as a system of record or as an accelerator to a slower datastore. It handles both cases with grace, ease, and speed. This post introduces caching, and focuses on the implementation of the cache-aside pattern, where the app is responsible for all communication with both the cache and the system of record.
The cache as a stash
Consider the humble squirrel. Into the fall, the squirrel gathers nuts and stashes them for winter eating by burying them nearby. (Yes, lots of squirrels remember where they've buried their nuts.) So we could also say that the squirrel caches its nuts.
The squirrel's caching of nuts is like our software systems' caching of data. A cache may simply be a place to store data (think: a datastore or a database).
Tanzu GemFire has all the features needed to act as a system of record. It's a key-value store that is designed to be highly available. The quantity of redundant copies is configurable, making a cluster more resilient in the face of failures, and data can be persisted to disk when needed. You can treat it as a database by doing queries using a query language similar to SQL called Object Query Language (OQL). Tanzu GemFire also has an API to implement transactions. Better still, it's fast and it's scalable, and doesn’t lose performance when scaled.
Cache basics
A cache that holds copies of app data improves the performance of modern apps that need data held in a centralized datastore. Often, that datastore is a relational database.
Without a cache, as the quantity of app queries to the database increases, database response time becomes a bottleneck. To maintain its ACID promises on its tables of data, the relational database must respond to the many concurrent queries from the apps in a serial order. This leads to increased latency, causing performance issues for the many apps waiting for their query results.
Caching data can dramatically increase performance. A cache holds copies of data that originate in a centralized datastore (such as a database), which acts as the system of record for that data. If the cache has a copy of a piece of data, and if the cache can respond to the app's data access requests faster than the database, then performance increases. Caches are designed to be fast, so the only remaining issue is to find a way to populate the cache with the data that the app needs. Caches take advantage of a data access pattern called temporal locality. Temporal locality of data occurs when an app reuses the same data, generally within a relatively short window of time. Most apps exhibit a high degree of temporal locality in their data accesses. Given this, populating the cache is as simple as placing in the cache any data that the app requests.
There are two common ways of structuring a system that includes a cache. The two ways differ in their responsibilities for communications between the app, the cache, and the datastore. One way of structuring is known as cache-aside; the other is known as inline.
The cache-aside pattern makes the app responsible for communication with both the cache and the database.
An inline cache makes the cache responsible for communication with the database.
Modern apps do CRUD (create, read, update, delete) operations on the data. Caches do reads (also called a lookup) and writes. A read operation is the only one of the four CRUD operations that does not modify data. The create, update, and delete operations do change data; they are cache write operations.
With either of these caching patterns, requested data might be
-
only in the cache
-
only in the database
-
in both the cache and the database
If a particular value changes in the cache, that value may not be the same as the value in the database. This potential for inconsistency has implications for how a cache needs to handle operations.
Cache-aside read operations
Consider an app's read request for a single piece of data, identified by a key. The app sends its request to the cache-aside cache. The cache does a lookup to determine if that key and its data are currently in the cache. If they are in the cache, it’s called a cache hit. The cache then returns the data to the app, completing the request. The database is not involved in the request, which saves tens or hundreds of milliseconds of app wait time.
If the key and its data are not currently in the cache, it’s called a cache miss. The app needs to fetch the necessary data from the database, which is considerably slower than the cache in responding to requests. The app places the data fetched as a result of a cache miss into the cache to take advantage of the temporal locality of data accesses. The next time the app requests that cached data, there will be a cache hit, and the data will be quickly returned to the app.
Cache-aside write operations
Create, update, and delete are write operations; they each modify data in some way. Write operations create a potential consistency issue. If data can be written to the cache while the unchanged value remains in the database, the two values are inconsistent. A cache-aside implementation must ensure that the completion of a write operation makes the data consistent.
One of these two implementations will be used to maintain consistency in a cache-aside cache, such that any copies of values in the cache are consistent with the values in the database. Both implementations ensure that the database, as the system of record, writes the data.
With the first implementation, the data is updated, created, or deleted from the database and the operation also goes to the cache; this is known as write through. Consistency is maintained, as the cache and the database will both have—or will have deleted—the same piece of data when the write operation completes.
With the second, the data is updated, created, or deleted from the database, and any copy of the data currently in the cache is invalidated, such that the cache no longer has a copy of the data. Only the database, as the system of record, has the data when the write operation completes.
Data consistency for write operations with multiple apps
Consider the use case in which more than one app accesses a single cache in a cache-aside implementation.
The apps execute code independent of each other, but may access the same data. If both apps issue concurrent writes to the same data, a consistency issue appears. It is as if two painters are attempting to paint the same boat hull at a marina that mandates that each hull must be painted one solid color. Our painters are unaware of each other, as they happen to start on opposite sides of the boat, and each painter works their way to the right with their own color—one beige, and the other green. The perception of the boat’s color depends on the observation location and at what point in the painting process the two painters are. As each painter finishes coating the hull once around, they both believe that they have completed their respective painting task. Unfortunately, the boat hull now has two colors.
Each app is like a boat hull painter, with app data as the paint. For the cache-aside pattern that implements write through, sending the data both to the database and to the cache, there is a consistency issue. If more than one app writes to the same data item concurrently, the timing of the writes to the database and the cache may overlap, causing them to interfere with each other, just as the painters might. The way around this multiple concurrent writers (painters) issue is to have all apps with the potential for inconsistent data use a transactional API to embed sets of write operations into transactions. Tanzu GemFire has an API for this. It acts much in the same way as a wet paint sign on the gate that leads to the boat that will be painted. The first painter that arrives doesn't see a wet paint sign, so puts their sign up. When the second painter arrives and sees the sign, they know that the boat is in the process of being painted, so they know that they cannot start painting the boat while the wet paint sign is posted. Problem solved.
Caching with Tanzu GemFire
While Tanzu GemFire can do much more than what’s outlined in this post, it’s the following design features that make it an ideal cache:
-
Data is kept in memory, which is substantially faster than disk. Of course, data can be persisted to disk, but in-memory storage makes data accesses fast.
-
It can handle lots of concurrent read requests, and without a degradation in performance, as might be seen with a relational database.
-
Its servers hold data, and you can increase in-memory capacity by adding more servers. A scaled-up cluster with more servers will be just as fast as the cluster with fewer servers.
To learn more about its features, the VMware Tanzu GemFire Developer Guide offers guides and tutorials that help developers quickly get up to speed. And the Basic Cache guide for Spring Boot for Apache Geode uses the cache-aside pattern for read requests with the bike incident app. It further demonstrates the performance gains of using a cache, as it prints the latency for each read access.
The app acquires and lists data about bike incidents within an area designated by a zip code, so the app user begins by entering a zip code as a key. As the app starts, there isn’t any bike incident data in the cache, so the first zip code entered causes a cache miss. The app sends the request to the service at bikewise.org and places the data that’s returned into the cache. Subsequent reads of the same data become cache hits, as shown in the image below.
While each access or use of the app will generate distinct response time numbers, any use will show a dramatic performance improvement for repeated zip codes because those repeated reads hit in the cache. The cache in this example is a Tanzu GemFire for VMs service instance. In this sample run, accesses to the service's datastore range from one to 10 seconds, while accesses that hit in the Tanzu GemFire cache are in the tens of milliseconds range. (Two orders of magnitude is a significant improvement in latency.) As a side benefit, those cache hit read requests reduce the quantity of queries to the service's data store.
Tanzu GemFire: an ideal cache
Tanzu GemFire works well as a system of record; it’s also an ideal cache. And as seen in the VMware Tanzu GemFire Developer Guide, the cache-aside pattern is easy to implement. Caching with Tanzu GemFire dramatically improves performance.
