During my tenure at Pivotal, I’ve spent countless hours implementing caching solutions for customers. A cache is a remarkably versatile piece of tech. Caching is often used to boost scalability and stability, even when the underlying infrastructure is ephemeral and performance constrained. If the cache is designed for horizontal scalability and high availability (Pivotal Cloud Cache is one such product), then performance and availability deficiencies in the underlying infrastructure can be masked by the cache. Caching is also a huge enabler of microservices architectures. Here, a cache can be used locally, and act as a serving layer exclusive to a microservice. This is a popular option for modern development teams, as it sidesteps the complexities related to sharing databases across microservices and teams.
In addition to the fortification of the infrastructure, there are several scenarios in which caching can lead to savings, including savings related to API access. APIs are generating revenue for organizations that share data and applications using easily accessible standards and platforms and their adoption has exploded. In fact, Programmable Web, an API directory, now has more than 22K APIs—this represents a 1000 percent growth rate over the last decade. There is no sign of slowing down—the API management market is expected to grow at more than 30 percent per year for the next several years. Companies are envisioning whole new lines of business centered around their API monetization strategy.
So there’s lots of money changing hands, as is common with a sea change in development patterns. This shift is a tremendous boost to developers who can focus on generating value, rather than re-creating functionality that is now broadly available for reuse.
This benefit comes with a cautionary note. Costs related to metered APIs can escalate quickly, especially with frequent and repeated API access for the same information. As someone who professionally works with caching, I instantly thought of how we can use caching to mitigate the costs on the paying side. Many enterprises are thinking about this as well. So let’s review how caching can help trim your cloud bill while empowering your developers.
I was working with a client where they had an external billing API that would cost them per invocation. A single page impression could end up calling that billing API several times as it rendered different aspects of the bill. The business recognized that this was an area where they could save money. The approach? Call the billing service once-per-customer, then instruct the cache to invalidate the entry on a periodic basis. This way, the system would regularly refresh the data, boosting accuracy.
There are two common patterns for this type of use case:
1. Look-Aside Cache: The application is in control of the cache and the API we are looking to cache.
2. Inline Caching: The cache is in control of the API we are looking to cache.
You can find a discussion of these caching patterns here.
After going through some discovery and framing the problem, we decided to explore the look-aside pattern. This is a natural first step since the application was already calling the billing API. So the look-aside option was low-risk; we’d simply add some logic to check the cache before calling the billing API. We were excited to build our caching-enabled MVP!
As we paired to evolve their heritage application to use caches we found a simple place to add the look-aside pattern. Here’s a bit of pseudocode to anonymize the scenario.(This particular application was written in .NET, but the same principle applies to Java, Node, and any other framework.)
Old Application:
billingInfo = getBillingInfo(customerId);
Look-Aside Pattern:
billingInfo = cache.get(customerid);
if(billingInfo == null){
//No billing info cached so call billing API
billingInfo = getBillingInfo(customerId);
//Let’s cache that billing info for next call.
cache.put(customerId);
}
The astute reader might raise the question: I see how the cached result gets into the cache, but how is it being removed?
There are a couple of options. First, we could have the application remove the entry when it's not needed, i.e. “cache.remove(customerId)”. That puts more burden on the application. Instead, we opted to have Cloud Cache remove the entry after a period of time. We call this time-based eviction.
In addition to time-based eviction, you can use Cloud Cache to configure custom eviction policies. Here, Cloud Cache will inspect the data for the right eviction strategy. For this particular client, we opted for `entry-time-to-live` since we knew that there was an update cycle for the billing. We just needed the time to live to be shorter than the billing cycle. In fact, we set the `entry-time-to-live` to just a couple of minutes, knowing that in the worst case we would just call the billing API an extra time.
Here’s an example of how we configured eviction for the billing cache from the Cloud Cache CLI. Note, in Cloud Cache a region is a way to logically group data, somewhat akin to a table in a relational database:
create region --name=billing --type=PARTITION --enable-statistics=true --entry-time-to-live-expiration=120 --entry-time-to-live-expiration-action=DESTROY
Based on how we configured the region, we ask Cloud Cache to create a region in the cache for billing. Further, we set up Cloud Cache to partition the data without a backup. Note this is a basic cache example, that doesn’t require the full power of Cloud Cache. For example, we created the region without high availability properties because we could just call the billing service and cache that result. Cloud Cache is often used as persistent storage, but in this case we only needed a transient cache. And we don’t need Cloud Cache’s capability to safely store the data in a redundant manner. (You will definitely need these capabilities for other enterprise scenarios.)
We can also see that we set the `entry-time-to-live` to 120 seconds and to `DESTROY` that entry once the time has elapsed. Any further accesses to that `customerId` would result in a cache miss, so the application would have to call the billing API.
The client was extremely pleased with the cost savings adding caching to their design. As an added bonus, the application was faster since the results from the third party APIs were cached locally to their Pivotal Platform foundation.
With Cloud Cache, we’ve been heavily focused on a rich set of APIs to facilitate easy integration. In a recent blog post, we discussed and showed how you would go about integrating Cloud Cache into a Java/Spring application. With our .NET client support, we have made this simple for your .NET applications as well, as you can see from the video below:
Next Steps
Reducing the cost of accessing data because of repeated API calls is just one of many ways Cloud Cache adds value to modern architectures. There’s lots more for you to explore here.
For a more immersive experience, the upcoming SpringOne Platform 2019 conference is a great way for you to explore new use cases and best practices in topics at levels that range from beginner to advanced. Here are some highlights:
-
Pivotal Cloud Cache is powered by Apache® Geode™. So, the upcoming Apache® Geode™ Summit is a great opportunity for you to learn from experienced practitioners, developers and committers to the Geode project.
-
The PCC Developer Workshop explores the use of caching for fast and highly available access to application data. Using a combination of presentation and hands-on exercises, you can go deeper into a number of related topics.
-
There are several Geode breakout sessions at the main SpringOne platform conference, in addition to the Geode Summit mentioned above. You can get a full list of Geode sessions at the conference sessions page by using the Geode tag.
You can register now, and receive a $200 discount by using the following code: S1P200_JMirani. Note that there is a Monday only registration option for those who only want to attend the Geode Summit.
