TUTORIAL: Protect Your External Service Credentials with Service Instance Sharing and the CredHub Service Broker. We Explain How.

You may be familiar with the CredHub Service Broker, a service that can store sensitive external service credentials encrypted at rest. It also obscures these secrets from app devs or anyone else with the ability to inspect app metadata. You may also have heard about the new Service Instance Sharing feature (now GA in Pivotal Cloud Foundry 2.3). This feature allows for services created in one org and space to be shared with (and revoked from) other orgs and spaces across the platform.

But did you know that these two features, when used together, can enable a simple, powerful new mode of interaction between service owners and app devs? With the CredHub Service Broker and Service Instance Sharing, platform owners can drastically reduce the complexity of managing external service credentials throughout the platform. Better still, this dynamic duo ensures that platform credentials aren't exposed to unauthorized users.

Here’s How You (Probably) Manage Credentials Today

There’s a good chance your credential provisioning process looks something like this:

A service administrator takes requests from app dev teams who want to interact with said service. (The "service administrator" in this case is whoever is responsible for providing service credentials for apps to use. This could be a DBA, a member of the InfoSec team, or a platform engineer or owner.) The admin does whatever provisioning they need to do—​create a new DB, generate a new user, etc. — and then they provide those credentials to a trusted member of the dev team. Sometimes a trusted third party, like the platform owner, gets the creds. The dev team (or the platform owner) provides those credentials to the application in any number of ways. The creds can go directly into app code or the app manifest. Some savvy teams may even create a custom user provided service (CUPS) in their org and space, and bind that service to the app.

This may work well for your first or second app. But what about your twentieth? Your fiftieth?  As the number of apps on your platform expands and intertwine, this flow can quickly grow unwieldy.

For instance, consider the case where multiple teams need access to the same database or message queue. The service administrator now has to track which credentials are where, who would be affected by planned downtime—or even a security breach. And since that admin is handing out creds directly, there's a possibility that their reach has spread further than the admin themselves are even aware!

One Way to Reduce Your Risk from Leaked Credentials

It doesn't have to be like this. Thanks to the CredHub Service Broker and Service Instance Sharing, service administrators can retain full control of how their credentials are used on the platform. They can add and revoke access to credentials with a single command, all without exposing their value to anyone but the apps themselves.

Here's how it works: when a dev team requests credentials to an external service, the service admin provisions and generates credentials as usual. But instead of providing those values to app devs, they interface with the platform directly. In a Cloud Foundry® org and space (that only they control), the administrator creates a new instance of the CredHub Service Broker service. The newly-generated credentials are stored within that service. Then, using Service Instance Sharing, they share the new service to the org and space of the app devs.

Now the app devs can bind their app to the service, and their apps on the platform can consume the credentials as usual. But if the devs try to inspect app metadata using Apps Manager or the "cf env" command, the value of the credential will be hidden from them, thanks to the CredHub Service Broker! If another team wants those credentials, the same exact admin-controlled service instance can be shared to their org and space as well, leaving in place an object trail of who is using which set of credentials. Pretty slick huh?

Let’s take a closer look.

How It Works

A service admin who is trusted with credentials to an external service creates a credhub service instance:

Now the credentials are stored securely and encrypted in CredHub. But the app teams who want this cred don't push their apps into the svcadmin-space—they have their own spaces where they run their apps. So, the svcadminuser uses Service Instance Sharing to provide those credentials to the relevant dev teams:

Now, app devs who push apps to spaces app-space-1 and app-space-2 can bind the app-pdb service to their apps:

At this point, apps are happily using the credentials provided by the service admin, even though app devs are unable to see the values. Also, importantly, the service values themselves are based on a single source of truth: the service instance that the service admin controls, in their own org and space.

What if the service admin determines that these credentials need to change? Perhaps they have reason to believe the creds are compromised, or maybe it's just been too long since their last rotation. In the past, the service admins would need to:

1. Take stock of everyone who is using the credentials.

2. Generate replacement credentials.

3. Distribute the replacement credentials to every app dev team that uses the service.

4. Follow up with every dev team to get confirmation that they have swapped out the old credential (or set a hard deadline and risk impacting business-critical workloads).

5. Revoke the old credential.

With the CredHub Service Broker and Service Instance Sharing, the process is as simple as this:

1. Generate replacement credentials

2. Run this command:

At this point, if your workloads can tolerate a few seconds of downtime, just revoke the old creds! In the worst-case scenario, the apps will lose connectivity to the external service, restart, and pick up the new credentials as if nothing happened. If you are sensitive to this level of downtime, then just make sure app devs restart their apps before revoking the old credentials. Either way, once you run the command above, apps will start using the new creds on their next restart, without a single keystroke from an app dev.

Do the Right Thing

At Pivotal, we work hard to ensure that the right choices are also the ones that are easiest for our users. Historically, many teams opted to provide plaintext service credentials to trusted members of dev teams. This workflow is easier than the "right" (i.e. more secure) choice of having service administrators individually put credentials into the relevant orgs and spaces. Trusting developers is crucial. But when it comes to credentials, it's always a good policy to expose them to as few people as possible.

With the CredHub Service Broker and Service Instance Sharing, we’ve made it easier to do the right thing. PCF 2.3 brings with it a simple, native process for sharing, tracking, and revoking app access to your external services. So platform owners, Do the Right Thing, and use these features to improve your security posture!