Making applications configurable is a best practice and allows the same binary to be used in different environments and scenarios. Common examples are database (DB) connection strings and credentials to access other services. More information about the general concept can be found on The Twelve-Factor App. In this post, I will give a quick overview of the approaches I often use in Go and give some guidelines on when to use which.

To me, application configuration is a feature like any other with the important aspect that it targets the person or group responsible for running the application. For example, in the case of a web server, an ops team might be responsible for running the application and would therefore be the group targeted by the configuration feature while other features of the server, e.g. the website that it serves, are targeting the end users of the business. For a CLI-based static site generator like Hugo on the other hand, application configuration is targeting the same group of users as its main features, because they want to be able to take control of the site generation with configuration properties.

Thinking about configuration like any other feature helps me determine what’s really needed and what isn’t. At my current job for example, I’m mostly responsible for deploying and running the services I work on. I know exactly how I will provide the configuration to the application. In this particular case I use environment variables. Therefore I already know that I don’t need to support other approaches like config files or command line flags.

Some frameworks, especially those in other programming languages, encourage a specific way to deal with application configuration. In Go on the other hand, application configuration is completely independent from other components of the application like database, message broker or web router (or framework). For example, when using jackc/pgx (a popular driver for PostgreSQL), the only thing it requires is a connection string that is passed as a function parameter when connecting to the DB. It’s entirely up to the developer to decide where that connection string comes from.

Getting started

Every time I start working on a new Go application, I follow roughly the same approach for the main() function. A simplified version is down below. I use a run() error function that is invoked from main(). That allows me to apply idiomatic error handling in run(), i.e. returning early on errors. It also allows using defer to clean up resources and shut down gracefully.

One of the first things I do in run() is loading all configuration values. Depending on the number of values the application needs, I might have a config package that is responsible for loading the configuration into a struct type that holds all the values. In that step, only basic validation is done, e.g. checking that required values have been provided or that number values are actually numbers.

After that, the various components of the application are bootstrapped. That includes database connections, 3rd-party SDK’s and application services.

Finally, the application is started. That could involve starting a web server and listening for requests or in case of a CLI, executing the business logic with the provided configuration.

package main

import (
	"fmt"
	"log"

	"github.com/mheck136/demo/config"
)

func main() {
	if err := run(); err != nil {
		log.Fatalf("application exited with error: %v", err)
	}
}

func run() error {
	if cfg, err := config.Load(); err != nil {
		return fmt.Errorf("failed to load config: %w", err)
	}
	// bootstrap components
	// start the server / run application logic, etc.
	return nil
}

Using the standard library

When I’m working on an application that only requires a few configuration parameters, e.g. only a DB connection string or a hostname, I try to stick with what Go’s standard library offers. It’s easy to use and often completely sufficient and avoids adding another 3rd-party library.

I find that environment variables often are a perfect way to provide configuration information to applications like web servers and they are very well supported. For local development, I use direnv to set up the configuration, keep my terminal profile clean and have different environment variable configurations for different projects.

The os package

The os package of Go’s standard library has two helpful functions to work with environment variables:

  • func Getenv(key string) string 
  • func LookupEnv(key string) (string, bool)

If it’s necessary to distinguish between an empty value and a unset value, LookupEnv is the right choice, otherwise Getenv is sufficient.

The flag package

For CLIs, the built-in flag package is a good place to start. It’s used by the go tool itself and should be good to handle most cases as long as the supported flag syntax is sufficient.

Advanced use cases

When an application grows, the number of configuration values it requires often grows with it. For most of the server applications I work on, I use sethvargo/go-envconfig. It allows loading values from environment variables into a struct that is annotated with variable names. Here’s an example:

type MyConfig struct {
  Database *DatabaseConfig
}

type DatabaseConfig struct {
  Port     int    `env:"PORT"`
  Username string `env:"USERNAME"`
}

It allows nesting structs which helps with grouping values that belong to the same domain, e.g. values to connect to the database. It also allows setting additional configuration for config values, e.g. to mark them as required or to set a default in case the value has not been set in the environment. Here’s an example of how the values are loaded:

var c MyConfig
if err := envconfig.Process(ctx, &c); err != nil {
  return err
}

Another advantage of this specific package is that it allows configuring how environment variables are looked up. That way, instead of using actual environment variables, a map can be provided which is perfect for testing purposes.

This setup has worked pretty well for me so far. If the configuration should instead be provided as a file, e.g. *.yaml or *.toml, then there are plenty of 3rd-party packages for deserializing all kinds of file formats available.

Supporting multiple configuration methods

When it’s important to support multiple ways to configure an application, e.g. because the software is distributed to customers who then run it in different kinds of environments, a more sophisticated system might be needed. This will allow the application to be configured in various ways, so that it can be used on a broader scale.

For these scenarios, spf13/viper is one of the most popular solutions. It’s widely adopted, well-maintained and very versatile. It can easily be combined with spf13/cobra, which helps with building sophisticated CLIs and has been adopted by Kubernetes, Docker, Hugo and others.

Viper helps with supporting various configuration mechanisms, retrieving configuration from various sources and combining the values based on a configurable priority order. It’s well-documented and has plenty of examples that can be used as a starting point.

There are of course many more excellent libraries that can be used to implement highly-flexible application configuration and I would recommend doing a bit of research before deciding on a solution.

Summary

Configuring an application in Go can range from simple to sophisticated, depending on the needs of the user and environment. For smaller applications or local development, sticking to Go’s standard library with environment variables or command-line flags often suffices. As applications grow and require more complex configuration, third-party packages like go-envconfig help manage and organize variables efficiently. For large-scale, flexible applications that need to support multiple configuration methods, tools like Viper provide powerful, versatile solutions. Ultimately, the right approach depends on who will be configuring the application and the level of flexibility required.