Configuration Management With Consul

We use configurations for feature flagging, easily recovering, easier collaboration between teams and easier data change operations. If we have distributed applications in large-scale, we should deep dive about how to manage them.

Let’s have a look at which alternatives we can choose;

1- Store the data in application settings

2- Store the data in database

3- Store the data in Key/Value store

Problems that we can face with:

If we store config data in application settings, we don’t lose performance but we should commit the changes on project and restart our application for using new values of config

If we store data in database, we may lose performance and have more latency and high response time. In this time we can cache data in memory cache for performance but we should restart our application for invalidating cache

Solution

  • dynamic configuration

Using Key/Value store with pub/sub mechanism is a good idea. We can reload changes in application runtime. So we don’t have to restart or deploy for cache invalidation. When the data change in store, application gets the new values and bind the data into related configs

We may use RollingUpdate as deployment strategy. We can think deployment cons doesn’t matter so much for now but with this solution, your application will never be unavailable for config operations

Some popular key-value stores are:

  • Aerospike
  • Apache Cassandra
  • Berkeley DB
  • Couchbase Server
  • Redis
  • Riak
  • Consul

Note

Consul isn’t just a key value store, it provides features about service discovery, secure service communication, multi datacenter etc

Expectations

1- Dynamic configuration

2- Exception Handling

  • On Load
  • On Watch

3- Fallback Scenario

Client Libraries & SDKs

You can use Consul with C#, Go, Java, Python, Php, Swift, Dart etc

https://www.consul.io/api-docs/libraries-and-sdks

Consul has own endpoints that we can do crud operations https://www.consul.io/api/kv. You can choose either writing your client library or using popular client libraries

I will explain usage with .Net 5 and Winton.Extensions.Configuration.Consul client library

Implementation

First of all, we need to have a consul server which our application will connect to

The command will create consul instance with Docker

Follow http://localhost:8500/ui/dc1/services url and visit consul ui as below, now we have a key/value store

Create your first configuration data in Key/Value section

Application

Create .Net 5 Web API

dotnet new webapi -n ConsulExApplication

Add consul client library

dotnet add package Winton.Extensions.Configuration.Consul

Configure Consul in your application

ReloadOnChange- Source will be reloaded if the data in consul changesConsulConfigurationOptions- Allows the default config options for Consul to be overridenPollWaitTime- Maximum amount of time to wait for changes to a key if ReloadOnChange is setOnLoadException- Action that is invoked when an exception is raised during config loadOnWatchException- Func that is invoked when an exception is raised whilst watching

By our consul settings, we will be poll consul data every 5 seconds. If we can’t get config data on application start, we will catch exception in OnLoadException func so that our application will not crush and continue with appsettings data in the project.

Additionally, we can improve log mechanism and monitor exceptions in monitoring tools like Kibana. We can define rule and send alerts for actions

If we can’t get config data while application is running, application will handle request with binded data before from Consul. With the rule that we defined in OnWatchException, application will try to reconnect and get latest data from Consul every 2 seconds

Read Config Value

Let’s call the endpoint and see the result :

1- When Consul worked successfully on load

2-When Consul didn’t work successfully on load and consul still gives error (bc of not running or wrong data format) -Fallback-

3- When Consul didn’t work successfully on load and consul is up again

***** Data will be changed from Consul

From application
From Consul

4- When Consul worked successfully on load but consul is down after (bc of not running or wrong data format)

Last taken data from Consul before consul is down

References

https://www.consul.io/

Software Engineer @hepsiburada