If you’re serious about building modern software, you’re using microservices in some portion of your app portfolio. You’re also hungry for tools and innovation that make running microservices easier. These two observations explain the popularity of Istio, an open-source project that just released version 1.0.0.
We’re big fans of this tech. Pivotal has contributed to Istio in meaningful ways, and we’re making Istio a core component in a number of open-source projects, as well as our flagship commercial products.
The 1.0.0 milestone is an important one, and it calls for some reflection. Here’s a closer look at Istio, the problems it solves, and how Pivotal is bringing Istio to the Forbes Global 2000.
Microservices: The Cause of, and Solution to, All of Life’s Problems
Microservices are a complex pattern. Yet, they are wildly popular. Why? Microservices deliver what engineering teams crave – speed-to-market, scalability, and flexibility. The added complexity from microservices is often worth it. (Just make sure you microservice responsibly.)
The open-source community has also helped the situation a great deal. There are hundreds of projects that promise to make life easier for microservices adopters. Istio – a project to simplify how microservices are connected, managed, and secured – is one of the most prominent.
And it’s easy to see why. Networking is often the thorniest part of microservices. After all, the respective architecture of IP-based networks and microservices are at odds with each other. To close the gap, developers must implement service discovery, circuit breakers, and other patterns on their own. They have to add health checks, tackle internal security, and implement policy controls. Sure, performing these tasks has gotten easier in Java since the launch of NetflixOSS in 2011. (More recently, Steeltoe does the same for .NET.) But there are extra dependencies to look after. And what about other languages, like Node.js and Go?
This is where Istio shines. It handles most of these functions with out-of-process proxies. You don’t need to change your application code. Plus, Istio is polyglot. Any language can benefit from this service mesh. Let’s dive into how it works.
Why Use a Service Mesh? Because We Can’t Change the Network
So today’s networks complicate microservices implementation. (This is why Spring Cloud Netflix and other tools are so useful.) Istio takes an alternative approach to simplifying microservices: it uses sidecar proxies to push higher level logic to the network. Use Istio, and let the proxies and the network do the work for you!
Three simple pictures tell this story.
The Status Quo: Direct Communication
Here’s the communications model for microservices today. Services talk to each other directly, with help from microservices tooling.
Source: Connecting All Abstractions with Istio
Enter the Sidecar Proxy
Now the fun starts. We introduce a sidecar proxy (SC), which “tricks” your services into thinking they are talking directly to each other. But in reality, they are talking to a proxy. This way, the network can take on security, service discovery, and other tasks the developer had to worry about in the direct model. This network of proxies is your new data plane.
Source: Connecting All Abstractions with Istio
Istio, The Control Plane, Manages the Proxies and Your Business Logic
Now we have all these proxies running alongside our services. You manage them all with a control plane. That’s Istio.
Source: Connecting All Abstractions with Istio
Want to learn more about the past, present, and future of Istio? Watch this talk from Pivotal’s Ramiro Salas and Google’s Laurent Demailly:
What Istio Does for You
Earlier, we mentioned that Istio is an open way to connect, manage, and secure microservices. Our friends at Google explain how Istio helps:
Istio for Monitoring
With features like tracing, monitoring, and logging, Istio gives you visibility into the performance of all your services and how it affects other processes, enabling you to detect and solve issues quickly. It also provides the deep logging and visibility you need for auditing and compliance.
Istio for Connectivity
Istio lets you manage and control how traffic and API calls flow between services, while providing visibility into your traffic. This means you can identify and fix issues before they become problems, making calls more reliable, and your network more robust, no matter what conditions you face.
Istio for Security
Istio starts off by providing strong authentication based on non-replayable identities to protect against replay attacks from a compromised service. Istio provides flexible authorization policies for enabling micro-segmentation based on identity or any request an attribute like IP address. Further, Istio enables security administrators to enforce encryption in transit using policies without any changes to the application. Thus, with Istio, security administrators can enforce strong security while developers can focus on application logic.
Sounds good, right? So how will you work with Istio in your day-to-day? Platform operators will spend time managing the control plane. But if we at Pivotal are doing our job right, developers won’t have to know much about Istio, or even know it’s there.
Istio can help you realize better business outcomes with software. But it should be an implementation detail you don’t have to care about. As Casey West will tell you, your CEO is not going to say, “Great job using a service mesh!”
Slides from my talk at #oscon London, "Said no CEO ever: Things that don't matter in the cloud" https://t.co/9WOwoQob8k
— Casey West (@caseywest) October 17, 2016
Pivotal is Adding Istio to the Abstractions that Matter
Technology is at its best when it’s invisible. This is the power of abstractions. In the case of Istio and a service mesh, you just want your microservices to work with as little grunt work as possible. It follows, then, that the abstractions you use for microservices should embed Istio where it’s practical to do so. This way, you can reap the benefits of a service mesh without having to fiddle with the plumbing.
And that’s exactly what Pivotal is doing. We’re making Istio core to key open-source projects and our flagship commercial products. And we’re doing it for all the abstractions that matter to a modern enterprise: app platforms and Kubernetes and functions.
Istio in Cloud Foundry
We have four tracks that highlight how we’re embedding Istio in your app platform.
Track 1: Envoy on App Instances for mTLS
This was completed in Cloud Foundry (CF) in February, and added to Pivotal Cloud Foundry shortly thereafter. Shannon Coen, PM for routing in Cloud Foundry, describes the output:
Shannon goes on to explain why this matters:
Increased security: Gorouter will encrypt traffic to application containers via TLS.
Increased resiliency: Gorouter will ignore the TTL of app routes, keeping your apps available during failures in the routing control plane.
Increased consistency: Gorouter will use the certificate presented in the TLS handshake to validate the identity of application instances before forwarding HTTP requests. Optimizing for availability increases the risk of misrouting, as a healthy Diego will continue recreating containers to keep your apps running and the probability of port reuse is statistically significant; this mechanism increases guarantees against misrouting.
All this without any additional burden on application developers!
My favorite part is that last bit. Istio, like other platform tech, should just work.
If you want more technical details, don’t miss Eric Malm's excellent post that takes you inside the guts of the implementation:
Had a ton of questions recently about how @PivotalCF uses @EnvoyProxy for transparent TLS all the way to the app container, so I wrote up a blog post about it: https://t.co/xZVaVSkgkk pic.twitter.com/hXYSdyIbYp
— Eric Malm (@emalminator) April 5, 2018
Track 2: Ingress (North-South Traffic)
Track 1 is done. Now, the CF routing team is enhancing ingress routing (i.e. North-South traffic) in Cloud Foundry with Istio and Envoy. The work will help us address several customer needs. (Check out the OSS proposal for a full run-down).
One of the first features you’ll see as a result of this work is weighted routing. Here, a developer can easily control the percentage of requests for a route sent to a “mapped” application. Use weighted routing to ease the rollout of new versions, and test the traffic for a new version. This is some next-level control for A/B testing and blue-green deployments. Expect to see these capabilities in open-source Cloud Foundry in the coming months.
Watch this CF Summit talk from Pivot Aaron Hurley for more on this scenario:
Track 2 will also feature the use of Istio Pilot for route updates. By now, you’re probably wondering if Istio and Envoy will succeed the existing routing tier in Cloud Foundry. Yes indeed!
Over time, Istio and Envoy will eventually replace existing routing components Gorouter, NATS, TCP Router, and the Routing API. But in the near future, the new Istio-based approach and the incumbent mechanisms will coexist.
Track 3: East-West Traffic
East-West traffic can benefit from Istio, too. App-to-app and app-to-service traffic will use Envoy with mTLS. We'll also use Istio and the Envoy sidecars to offer platform-managed, client-side load balancing. This model eliminates the need for app devs to hassle with libraries in their code (like Ribbon) for these functions.
Track 4: Application Security Policies
When you think of security policies, a few things come to mind. Simple IP-based rules. Identity and access management (IAM). PCF helps you manage network policies with application security groups and container networking. The Single Sign-On service locks down application access for your users. But what if there is a way to think bigger?
Istio offers up tantalizing possibilities, whereby policies could be something far more meaningful. With Istio, our definition of policies could be broadened to govern access based on custom, complex business logic. For example, you could easily restrict access to certain applications for a certain set of employees for a given window of time. Or, you could implement method-level security. In this case, you’d control who could perform GET, PUT, POST, and DELETE actions against a RESTful API.
Istio offers up the promise of policy system pluggability. But as the saying goes, with great power comes great responsibility. So Pivotal’s task is to expose “just enough” policy syntax within Istio to solve enterprise scenarios, without adding undue complexity. We’re actively researching how best to do this. But make no mistake: Istio will be at the core of how Pivotal enforces security policies across microservices and the platform itself. Stay tuned!
In the meantime, check out this talk from Sree Tummidi on how Istio could be used in the Cloud Foundry user account and authentication module:
Pivotal & Our Contributions to Istio
With all the work we’ve done integrating Istio with Cloud Foundry, Pivotal saw opportunities to make significant contributions. Two areas jump out.
The Gateway Resource
Gateway enables you to configure an edge gateway router when your requirements are different than from the aforementioned sidecar scenario. This was a concept that the Istio team was already considering, and the CF Routing team simply accelerated the delivery of this capability.
The Mesh Control Protocol (MCP) API for Istio Pilot – coming soon!
The MCP API for Pilot is all about ease of configuration. This API will make it easier to roll out configuration across Istio components. Further, the MCP API makes it easier to configure Pilot with alternatives to the Kubernetes API. So integration between CF and Istio is faster and simpler. In fact, Cloud Foundry committers can make updates to the MCP-compatible integration component, without requiring changes to Pilot. The MCP will work alongside Galley.
How did this come about? The Istio community wanted to use Galley for configuration. They noticed that CF committers had contributed a bunch of code to Pilot for needs specific to Cloud Foundry. The Istio team decided an API we could both use was the right strategy. The CF Routing team celebrated the idea, and offered to help with implementation.
The Mesh Control Protocol (MCP) will be a standard API that Cloud Foundry, Kubernetes, and Istio teams can all use. MCP will manage multiple configuration profiles for multiple components across multiple platforms. Anytime you can use the same pattern across abstractions, it’s a huge win for the community and enterprise developers. (Case in point: the Open Service Broker API.)
Looking Ahead: Istio in Kubernetes & Knative
Kubernetes & Pivotal Container Service
Developers may naturally be tempted to take on the task of using Istio and Kubernetes together, but this additional operational burden can come at a cost. Unless your core business is building and selling a platform, it’s a waste of time and money. Your best and brightest engineers should be adding unique business value, not wiring up open-source components. (That’s why we caution against the DIY path.)
Pivotal, of course, is in the business of building and selling platforms. And we’ve made Kubernetes clusters Istio-ready in Pivotal Container Service, Pivotal’s commercial distribution of the Cloud Foundry Container Runtime. Here’s how.
Pivotal has baked the installation of Istio into its CI/CD for PKS. Each new version of PKS (1.1.1, 1.1.2, etc.) will help you create “Istio-ready” Kubernetes clusters. This is a foundational capability, one that will help PKS users embrace more Istio capabilities over time.
Knative, riff & Pivotal Function Service
Hot off the press! Knative is a new open source project started by engineers from Google, Pivotal, and other industry leaders. It’s a collection of components that extend Kubernetes and Istio. The project aims to simplify how functions are run on any cloud.
This project was just announced at Google Cloud Next (read all about it here and here). Check out this useful graphic courtesy of Mark Chmarny at Google:
Source: Build, deploy, manage modern serverless workloads using Knative on Kubernetes.
On the commercial side, the forthcoming Pivotal Function Service (PFS) will feature Knative and riff.
Some of riff’s components move into Knative, others stay independent. Read the post for more details.
These open source and commercial efforts are both in their early days. But it’s easy to see how the power of Istio is abstracted away from the developer in each case. The engineer can blissfully focus on their business logic and let Knative, riff, and PFS do the rest.
Outcomes, Outcomes, Outcomes
The tech used by distributed systems in 2018 is different from the tech that underpinned those in 2008. The task for enterprise leaders is to work with the right strategic partners to help you navigate the dynamic world of cloud computing. You want superior business results, even as the underlying tech evolves and changes to offer higher abstractions over time.
That’s what Pivotal offers its customers, and that’s why we’re so bullish on Istio and Envoy. We’re passionate about making it core to our products so you can focus on writing great software.
What should you do now?
Help us define the future of Istio in Cloud Foundry. Start by checking out the istio-release (BOSH release). Then review and comment on our design docs:
Join us at SpringOne Platform in Austin, TX, October 7 to 10. Register now! Use discount code S1P200_JRuckle for $200 off the registration fee.
Learn more:
SpringOne Platform 2018 sessions
