Using Microsoft Configuration Extensions with Steeltoe

When you design a modern application, so many new “doors” are opened as you. One topic that you should ponder: configuration. It’s not a very exciting topic like say, Kubernetes. But configuration has tremendous long term effects on the success of your application. The difference between a successful, rapidly improving app and an unmaintainable system can come down to how your app’s configuration is managed.

How do you avoid a messy jumble of configuration management? First, you can rule out holding settings internally to the app that's definitely not an option. Hard coding values in a compiled artifact is so ‘90s, let's move on. Attaching a .config file to each instance of the application is also not reasonable. Yes, you externalized config values from the artifact. But you’re no better off with managing and updating things.

So you’re left with two options: environment variables and external configuration servers. Both are best practices when managing app settings in the cloud.

But which one is the best for your use case? How do you choose one over the other? Let's talk about that.

Environment Variables: A Solid Choice, but Not at Scale

Environment variables are easy to manage, but it lacks flexibility at scale. Variables can be globalized to each environment the app is promoted through (dev, QA, load-test, prod), but not much further. And what about refreshing the values, and keeping track of those changes? Depending on your choice of runtime, your app might require a redeployment (or re-containerization) for each update in config values. Even for the most advanced cloud platforms, an app restart is needed.

Environment variables can be the right choice. Just realize that as instances grow, so must the app’s supporting systems.

External Configuration Server: More Complexity, but Usually Worth the Effort

With an external configuration server, you can enjoy greater accessibility and flexibility. There are other benefits too, namely zero downtime updates to values, and the separation of duties.

So what’s the catch? More dependencies, and more complexity. For example, most systems won’t have a single application using the configuration; it will be shared with multiple apps.

To help you evaluate these options, let’s consider an example. Just for fun, we can add another twist: the freedom to use your own framework. There’s no reason in these modern times that teams developing for different runtimes can’t share the same configurations.

Using Steeltoe and Microsoft.Extensions for Your Config

Steeltoe offers superb options for .NET applications looking to consume values from an (external) config server, namely Spring Cloud Config. Now, you might be thinking about “framework lock-in”, or “being at the mercy of that framework’s pace of .NET adoption.” It’s only natural to be cautious.

But here’s the exciting thing: Steeltoe simply extends all the rich features of the .NET Core runtime, not implementing everything from scratch! Under the covers, Steeltoe is native C# goodness. Remember, .NET Core is a runtime for many use cases, not just  cloud-native apps. (Of course, Steeltoe adds in a little cloud-native opinion on top, to make it much easier to follow best practices for microservices management and security.)

Here’s a fun fact. Microsoft.Extensions can be implemented in many different types of projects: MVC, Web Api, Forms, WCF, and others. The net effect for you, the developer: you can implement configuration management for all these different types apps exactly the same way! That means sharing configuration models across your mobile app, API, desktop app, and console batch jobs. You can confidently promote your code through testing environments and to production in configuration harmony.

Let Microsoft.Extensions.Configuration Handle The Hard Work

Consider a traditional .NET app. You likely have configuration coming from multiple directions. You are probably used to dealing with web.config, web.dev.config, web.test.config, environment variables, possibly a database holding values, command line arguments, and (even worse) a flat .txt file.

You can’t just simply retrieve a value from each of these config sources. You have to manage its availability, provide default values, parse the values into something structured, and (possibly the most complex) manage precedence of who overwrites who. If that’s not toil, I don’t know what is!

Microsoft.Extensions.Configuration takes a lot of that overhead away with its “different source providers” feature. Let’s say you have different configuration values coming from appsettings.json and environment variables, with a rule that the environment variables should overwrite a matching value from both.

To accomplish this, simply tell your .NET Core app to look in both places by placing the call to AddEnvironmentVariables() after the call to AddJsonFile(), and voila! You’ve established precedence.

public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
 WebHost.CreateDefaultBuilder(args)
   .ConfigureAppConfiguration((hostingContext, config) => {
     config.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
     config.AddEnvironmentVariables();
   })
   .UseStartup<Startup>();

To retrieve these values, you now have one simple, structured place to reference – the Configuration Interface. Let's look at an example of retrieving a config value in a Controller. Using the dependency injection feature of .NET Core, simply inject the IConfiguration package in the constructor, then reference the config as a key/value collection.

public class HomeController : Controller {
 private IConfiguration _config;
 private string _myValue;

 public HomeController(IConfiguration config) {
    _config = config;

    _myValue = _config["some-value"];
 }

...
}

You can read further about Configuration, the options within, and more notably the different types supported in the .NET Core docs.

Now that you know how configuration is abstracted in modern .NET, let's look at how Steeltoe uses these concepts to make  cloud-native design even easier.

Steeltoe Manages the External Config Server

You may have heard of The 12 Factor app, the design techniques that every cloud native application should follow. The third factor “Config” states that “A litmus test for whether an app has all config correctly factored out of the code is whether the codebase could be made open source at any moment, without compromising any credentials.” To extend that test, what if another development team using a different runtime needed to share configuration with your app? How easily could that happen?

With an external config server, like Spring Cloud Config Server, it’s easy to create these types of tests. Simply create a separate Git repo that holds a config file (yml, or equivalent name/value pairs). When it changes, the app automatically sees it.

But what does “app automatically sees it” mean? Well, that’s how the Steeltoe.Configuration component implements Microsoft.Extensions.Configuration. A developer can use an external Spring Cloud Config Server with Steeltoe.

The relevant snippet of Program.cs looks like this:

public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
 WebHost.CreateDefaultBuilder(args)

   .UseCloudFoundryHosting()
   .AddConfigServer()
   .UseStartup<Startup>();

public class MyConfiguration {
   public string Property1 { get; set; }
   public string Property2 { get; set; }
}

Then modify Startup.cs…

public void ConfigureServices(IServiceCollection services) {
 // Setup Options framework with DI
 services.AddOptions();

 // Configure IOptions<MyConfiguration>
 services.Configure<MyConfiguration>(Configuration);
 ...
}

The fundamentals are still following the configuration principles discussed in the earlier “Let Microsoft.Extensions.Configuration Handle The Hard Work” section. We’ve introduced a new concept known as the “options pattern.”  The idea here is to extend configuration capabilities by using classes to represent a group of configuration values. It is implemented via the Microsoft.Extensions.Options.ConfigurationExtensions package.

In the above example, UseCloudFoundryHosting() retrieved the connection properties provided by the platform for connecting to its Spring Cloud Config Server instance via a custom configuration provider. Then AddConfigServer() used those config values to retrieve the properties from the config server. Because we gave the app further direction to load our property values into a custom object [services.Configure<MyConfiguration>], we now have external config values for consumption.

There’s a lot going on with a little amount of code. Below is the result—the ability to reference config values in a simple, structured way.

public class HomeController : Controller {
   public HomeController(IOptions<MyConfiguration> myOptions) {
       MyOptions = myOptions.Value;
   }

   MyConfiguration MyOptions { get; private set; }

   // GET: /<controller>/
   public IActionResult Index() {
       ViewData["property1"] = MyOptions.Property1;
       ViewData["property2"] = MyOptions.Property2;
       return View();
   }
}

As a bonus, you can introduce a kind of blue/green deployment when the values change. Spin up a new instance of the app. Your nifty new code will retrieve the latest values, and monitor the logs. If everything looks good, you can then stop the original app instance. Congratulations, you have refreshed a config value with no downtime!

Stateless Applications Working with its Platform

Environment variables are a key component of an app running on a modern platform. It’s not just about configuration; it’s also a way for the platform to “talk” to the app. Why should a stateless, portable app need to care about its platform? Or its environment for that matter? It doesn’t!

It may seem a bit counter-intuitive, but the platform needs to “tell” the app what services have been bound to it, and if any startup customizations are expected. These are communicated in the form of environment variables.

Take Cloud Foundry for example. It offers two environment variables to every app instance running as VCAP_APPLICATION and VCAP_SERVICES. The value of these VARS is a json formatted string with details about how that specific app instance has been deployed. For VCAP_APPLICATION, this covers bound routes, app name, and space (environment) name are just a few. Meanwhile, the VCAP_SERVICES var has (you guessed it) information about what services have been bound to the application. It’ll cover things like service name, plan, tags, and all the connection details.

Having all that rich information is very valuable. But who wants to deal with parsing the json, managing exceptions, and other gruntwork? You want to be the best developer you can be, and create business value by working on the important stuff, like new features and bug fixes.

This is where configuration components come in. The Steeltoe team saw how .NET Core configuration providers made consuming environment variables so easy. So they used that with the structure of the VCAP_* values to create the Steeltoe Cloud Foundry configuration provider.

This is quite similar to how values are retrieved from an external Spring Cloud Config server. The Cloud Foundry config provider offers a simple, structured object containing the VCAP values. And it’s all thanks to .NET Core’s custom configuration provider.

Configuration Default Values and Dynamic Placeholders

You have many different ways to feed configuration values to your app. Two challenges come to mind. First, how and where will a value get defaulted. Second, the local development environment and the production environment need to be as similar as possible – especially with how configuration values are evaluated.

Of course Microsoft’s Configuration provider along with the Steeltoe Framework have a simple answer to these challenges. We can start with a look at the ordering of the providers when you build the WebHost.

In Program.cs…

public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
 WebHost.CreateDefaultBuilder(args)

   .ConfigureAppConfiguration((hostingContext, config) => {

     config.AddJsonFile("appsettings.json", optional: false, reloadOnChange: false);

     config.AddCommandLine(args)

     config.AddEnvironmentVariables()

   }

   .UseStartup<Startup>();

Now, consider providing the same labeled value first in appsettings.json as “123” and then as  “456” in environment variables. No values are provided as arguments when the app starts up. In the end, the app is going to see the value as “456” because that was the last provider to be parsed.

There’s another piece of the puzzle that isn’t so obvious. Notice our example uses CreateDefaultBuilder to create the WebHost, as opposed to creating a new instance of WebHostBuilder and manual configuration. We can look to the WebHost.CreateDefaultBuilder docs to see what is loaded and in what order.

Here’s the sequence:

• use Kestrel as the web server and configure it using the application's configuration providers

• set the ContentRootPath to the result of GetCurrentDirectory()

• load IConfiguration from 'appsettings.json' and 'appsettings.[EnvironmentName].json'

• load IConfiguration from User Secrets when EnvironmentName is 'Development' using the entry assembly

• load IConfiguration from environment variables

• load IConfiguration from supplied command line args

• configure the ILoggerFactory to log to the console and debug output

• enable IIS integration.

Look at that! IConfiguration already loaded appsettings.json and its derivatives, plus environment variables and command line args. So why did we specifically call out appsettings.json, if it’s a default of CreateDefaultBuilder? Because the app needed to load the Configuration Providers in a different order, thus giving the app a different way to set default values. This is often a nice option when switching environments.

What about a middle ground between relying on environment variables for values, and implementing a configuration server? One may not offer enough flexibility; the other may be overkill for the app. Steeltoe introduces a provider called Placeholder. The idea here is to enable an app to define configuration values as placeholders. Then they can be  resolved to real values at runtime during configuration access. A placeholder takes the form of ${key:subkey1:subkey2?default_value} where key:subkey1:subkey2 represents another key in the configuration.

Lets look at an implementation of Placeholders. First set values in appsettings.json…

// appsettings.json
{
 "Logging": {
   "LogLevel": {
     "Default": "Warning"
   }
 },
 "AllowedHosts": "*",
 "ResolvedPlaceholderFromEnvVariables": "${PATH?NotFound}",
 "UnresolvedPlaceholder": "${SomKeyNotFound?NotFound}",
 "ResolvedPlaceholderFromJson": "${Logging:LogLevel:System?${Logging:LogLevel:Default?NotFound}}"
}


// appsettings.Development.json
{
 "Logging": {
   "LogLevel": {
     "Default": "Debug",
     "System": "Information",
     "Microsoft": "Information"
   }
 }
}

Is this just more toil I’m asking you to take on? Of course not, I wouldn’t do that to you. Steeltoe takes care of all the hard work. Just implement the PlaceHolder provider and get on with building great software!

Getting Started

Now it’s time for you to put all these features to the test. Fortunately, there are quite a few options to get started.

Sean McKenna gave a nice rundown of Cloud Foundry on Azure Friday, and the Azure Marketplace offers a really easy way to get going with Cloud Foundry. Grab the sample apps in the Steeltoe Github Samples repo along with the docs on the Steeltoe site, and the next stop is cloud-native .NET.

If you’d like to work locally, the Steeltoe team has published some Docker Files to help get started. Combine this with the sample apps in the Steeltoe Github repo and you can get a local .NET Core app instance running with Steeltoe in no time.

If you prefer a pre-configured platform, head on over to Pivotal Web Services and create a free account. This will give you access to a fully productionized Cloud Foundry platform. Grab the sample apps in the Steeltoe Github Samples repo along with the docs on the Steeltoe site, and you are off to the races.

Cloud native .NET here we come!

Learn more

• Webinar: Learn Cloud-Native .NET: Core Configuration Fundamentals with Steeltoe 
• Training: Pivotal Cloud-Native .NET Developer (with Steeltoe)
• eBook: Modernizing .NET Applications