Overview
Introduction
The EnvironmentConfiguration class provides injectable environment configuration from programming language agnostic hierarchical property files.
Typically, a server application will be run as one or more separate processes spread across a group of machines. Each running process will be configured according to a specified environment. In order to configure the application differently for each environment, the EnvironmentConfiguration class provides a convenient way of binding externally sourced hierarchical environment properties into the injector, ready for injection into dependent classes as typed named values and named composite EnvironmentProperties instances.
The EnvironmentConfiguration class works with hierarchical property files (see the property parser chapter for more information on defining hierarchical property files). A URI referencing a properties file is specified as a constructor argument of the EnvironmentConfiguration class or an implementing class of the EnvironmentConfiguration class. The referenced properties file is parsed and appropriate bindings are created when the environment configuration instance is called by the instantiating injector.
In addition to string value properties, the environment configuration framework supports typed configuration properties via type specifications. The use of type specification files allows a single set of hierarchical environment configuration type specification files to be used across multiple applications written in multiple programming languages.
Type specifications may also have default values attached to them. This allows sensible defaults that apply to all environments to be specified in a single location, preventing the need for complex environment configuration property files and consequential logical coupling across multiple environments and applications.
Type specification files and property value files are conceptually similar to classes and instances. A type specification file provides a hierarchical typed contract. A property value file provides hierarchical instance values that fulfill a type specification contract. This analogy is not exact, as type specification files can provide default values for instantiation and property value files can provide values that default to string types when no matching type specification is provided. The hierarchical configuration design thus provides a looser contract than the class-instance contract.
The Balau library provides a C++ implementation of the environment configuration and environment properties support classes. Java based environment configuration classes are also planned, to support Guice and Spring based applications.
Usage patterns
There are three ways to use the environment configuration class:
- derive from the EnvironmentConfiguration class and hard wire hierarchical property type specifications inside the configure() method of the derived class;
- instantiate an EnvironmentConfiguration instance directly, by specifying one or more hierarchical property type specification sources in addition to the input environment properties sources.
- derive from the EnvironmentConfiguration class, hard wire hierarchical property type specification declarations inside the configure() method of the derived class, and specify one or more hierarchical property type specification sources in addition to the input environment properties source.
The first approach places the hierarchical property type specifications directly within the C++ code. An advantage of this approach is that any type that has a corresponding fromString function can be used for value property types parsed by the environment configuration, without the need for explicit registration before injector creation.
The second approach places the hierarchical type specifications within type specification definition files. The advantage of the second approach is that the environment configuration type specifications are defined within an IDL (which is itself also in the hierarchical property format). Environment configuration type specification files may thus be defined once and used for multiple software applications written in multiple languages, without needing to redefine the type specifications or the environment configuration files.
The disadvantage of the second approach is that all custom types (i.e. types not pre-registered in the Balau library) referenced in the type specification files must be registered with the consuming C++ applications before creating the injector. This can be achieved via the following function calls.
- EnvironmentConfiguration::registerValueType<T>
- EnvironmentConfiguration::registerUniqueType<T>
Such registration of custom property types will not be necessary in other languages such as Java, where reflection can be used to resolve string to type mappings.
The third approach mixes the first two approaches together. This involves deriving from the EnvironmentConfiguration class and also passing one or more type specification properties files to the base class constructor, in addition to implementing the configure method.
With the first and third approaches, additional validation logic can be placed in the configure method if required. This places validation logic specific to specific environment configuration inside the same class that generates the injector bindings for it.
With all three approaches, type specifications are optional. If a value property is present in a specified property file and there is no corresponding type specification, a std::string property will be created (std::string being the default property value type).
Quick start
#include <Balau/Application/EnvironmentConfiguration.hpp>
Properties
An example hierarchical property file used for environment configuration looks similar to the following.
http.server.worker.count = 8
file.serve {
location = /
document.root = file:src/doc
cache.ttl = 3600
}
The above is an example from the Balau HTTP server tests.
Hard wired specifications
Hard wiring the environment configuration type specifications is best limited to applications that either:
- do not share their environment configuration with other applications; or
- share their environment configuration with other C++ applications via a shared C++ library.
The creation of an environment configuration class with hard wired type specifications consists of deriving from the EnvironmentConfiguration base class, and implementing the configure method. Environment configuration type specifications that correspond to the properties in the referenced property file are placed within the configure method.
The following is an example environment configuration class implementation for the above example property file.
// Environment configuration for the example HTTP file server.
class EnvConfig : public EnvironmentConfiguration {
public: EnvConfig(const Resource::Uri & input) : EnvironmentConfiguration(input) {}
public: void configure() const override {
value<int>("http.server.worker.count");
group("file.serve"
, value<std::string>("location")
, unique<Resource::Uri>("document.root")
, value<int>("cache.ttl")
);
}
};
IDL based specifications
IDL based environment configuration type specifications are best used for applications that:
- are written in multiple programming languages (and thus require language agnostic type specifications);
- are maintained by multiple teams each with a different release schedule.
The IDL based method for specifying environment configuration involves the external creation of environment configuration type specification files, referenced via URI.
The creation of a type specification file consists of creating a hierarchical property file that is similar to the environment configuration property file. Instead of specifying property data values, the type annotations are specified.
Direct instantiation of the EnvironmentConfiguration class involves passing one or more environment configuration type specification files, referenced via URI to the constructor of the EnvironmentConfiguration class. The type specification files are cascaded together, and the resulting hierarchical type specifications are used as if they were specified within the configure method.
The following properties file is an example environment configuration type specification for the previous properties data file.
http.server.worker.count : int
file.serve {
location : string
document.root : uri
cache.ttl : int
}
In this example, the : separator has been used in order to accentuate that the property values are type annotations. Alternative = or whitespace separators can also be used if preferred.
Property type specification files look very similar to property data files, as they have a similar hierarchical structure.
In order to use the type specification file, the EnvironmentConfiguration class is instantiated, specifying the type specification file and the property data file to the constructor.
// Direct instantiation of the EnvironmentConfiguration class.
auto envProps = Resource::File("path/to/env/env1.hconf");
auto specs1 = Resource::File("path/to/type/specs1.thconf");
auto specs2 = Resource::File("path/to/type/specs2.thconf");
auto envConf = EnvironmentConfiguration(envProps, specs1, specs2);
The EnvironmentConfiguration class' constructor is variadic, thus multiple type specification URIs may be specified.
Mixed specifications
Hard wired and IDL based environment configuration type specifications can be mixed in an environment configuration derived class.
If a mixed type specification is used in an environment configuration, it is possible that duplicate type specifications are provided for a property (one in the hard wired configuration and another in the type specification file). When this occurs, the hard wired type specification / default value takes precedence over the file based one. It is thus possible to override a file base type specification by adding a hard wired type specification with an identical name and hierarchy position in the derived class configuration method.
Default values
Both hard wired and IDL environment configuration type specifications may have default values attached to them. This allows concise environment configuration data files to be created, by specifying only the differences between the defaults and the required values for that environment.
The following example is a copy of the previous hard wired example environment configuration class, with default values attached to the properties that can have sensible defaults.
// Example environment configuration with sensible defaults.
class EnvConfig : public EnvironmentConfiguration {
public: EnvConfig(const Resource::Uri & input) : EnvironmentConfiguration(input) {}
public: void configure() const override {
value<int>("http.server.worker.count", 8);
group("file.serve"
, value<std::string>("location")
, unique<Resource::Uri>("document.root")
, value<int>("cache.ttl", 3600)
);
}
};
The following example is a copy of the previous IDL example environment configuration type specifications file, with default values attached to the properties that can have sensible defaults.
http.server.worker.count : int = 8
file.serve {
location : string
document.root : uri
cache.ttl : int = 3600
}
Application creation
Injector
In order to create an injector with both application and environment configuration, the environment configuration instance(s) are passed to the injector's create function in the same way as is done with the application configuration instance(s).
// Create an injector from a single application configuration // and a single environment configuration. auto injector = Injector::create(AppConfig(), EnvConfig(envProps));
The above code will parse the contents of the properties file and will build a corresponding set of bindings in the injector that correspond to the root properties (simple and composite) within. Simple properties become named value or unique bindings of the type specified in the environment configuration declaration, or std::string if no type specification was made for that property. Composite properties become named shared bindings of type EnvironmentProperties.
Any issues encountered during the build will be flagged via an EnvironmentPropertiesException.
All properties contained within a composite property become bindings within the resulting EnvironmentProperties instance. This may include other composite properties that themselves may contain their own bindings. EnvironmentProperties instances have a similar public API to part of the injector's API. Three get-instance calls are available:
| Get-instance call | Instances obtained |
|---|---|
| getValue | Value property non-polymorphic values |
| getUnique | Value property polymorphic values |
| getComposite | Composite properties |
Unlike the bindings of the injector class (which may include bindings of type EnvironmentProperties), the bindings contained within EnvironmentProperties instances are not automatically injected into other dependencies. Due to this, the EnvironmentProperties class does not have getInstance methods (which are used in the automatic injection functions of injectable classes).
The string to object conversion mechanism used for value property non-polymorphic and polymorphic values is Balau's universal from-string function. More information on the universal from-string function is available in the characters and strings chapter. For non-polymorphic value types, a from-string function should be defined with a reference to the destination object. For polymorphic value types, a from-string function should be defined with a reference to a std::shared_ptr<T> destination object.
Once the environment property bindings have been created, the root set of simple and composite properties can be used in the same way as all other bindings in the injector, including automatic injection into other dependencies.
Application
In order to use an environment's properties file(s), the C++ application's main function must provide a path to the environment's home directory / property file location(s). This can be achieved by various methods, a couple of simple ones being:
- passing the environment's home directory via a command line argument;
- passing an environment identifier via a command line argument and resolving the environment's home directory via a simple mapping.
The EnvironmentConfiguration class is designed to fail immediately if an environment configuration property file is not well formed, thereby preventing the application from starting up with an invalid configuration.
Credentials
Part of the environment configuration supplied to an application will be one or more credential properties. Unlike other environment configuration properties, credentials properties should not be checked into VCS and will thus exist in one or more separate properties files. This approach allows credentials information to be private to system administrators.
Applications that require credentials information should thus have at least two environment configuration properties files:
- one for the main environment configuration (checked into VCS);
- a second one containing credentials information and with a similar hierarchical structure to the main configuration file.
During creation of the environment configuration injector bindings, the main environment configuration and the credentials environment configuration will be merged together to form a single set of hierarchical properties that will be transformed into bindings.
Property type IDL
Property type specifications are defined in the hierarchical property format. In the IDL, the physical structure of the type specification hierarchy is similar to the hierarchy of corresponding environment configuration data files.
The following type strings are currently pre-registered for the environment configuration framework:
- byte;
- short;
- int;
- long;
- float;
- double;
- string;
- char;
- boolean;
- uri.
All the pre-registered types apart from the uri type are non-polymorphic value types (values obtained via subsequent getValue calls). The uri type is a polymorphic unique value type (std::unique_ptr values obtained via subsequent getUnique calls).
The environment configuration types map to the following C++ types.
| type string | C++ type |
|---|---|
| byte | signed char |
| short | short |
| int | int |
| long | long long |
| float | float |
| double | double |
| string | std::string |
| char | char |
| boolean | bool |
| uri | std::unique_ptr<Resource::Uri> |
As environment configuration type specifications have been designed to be programming language independent, unsigned integer types are not included as pre-registered types. If unsigned integer type specifications are required for a Balau based C++ application, they can be manually registered by calling the EnvironmentConfiguration::registerUnsignedTypes function. Calling this function before creating the injector will register the following types:
| type string | C++ type |
|---|---|
| unsigned byte | unsigned char |
| unsigned short | unsigned short |
| unsigned int | unsigned int |
| unsigned long | unsigned long long |
Other non-standard types may also be manually registered by calling one of the following functions for each custom type, before creating the injector.
- EnvironmentConfiguration::registerValueType<T>
- EnvironmentConfiguration::registerUniqueType<T>
The first function should be used to register non-polymorphic value types (for values obtained via subsequent getValue calls). The second function should be used to register polymorphic unique value types (for std::unique_ptr values obtained via subsequent getUnique calls).
In addition to the type string used in type specification files, the polymorphic function takes a cloner function. This function will be used to clone the std::unique_ptr prototype value on each subsequent call to getUnique.
The full signatures of the two functions are as follows.
/// /// Add a non-polymorphic type custom property binding builder /// factory to the global property binding builder factory map. /// template <typename ValueT> void registerValueType(const std::string & typeString); template <typename BaseT> using UniquePropertyCloner = std::function< std::unique_ptr<BaseT> (const std::unique_ptr<const BaseT> &) >; /// /// Add a polymorphic type custom property binding builder /// factory to the global property binding builder factory map. /// template <typename BaseT> void registerUniqueType(const std::string & typeString, const UniquePropertyCloner<BaseT> & cloner);
An example of a hierarchical property type specification file follows.
http.server.worker.count : int
file.serve {
location : string
document.root : uri
cache.ttl : int
}
In the above example, the : separator has been used in order to accentuate that the property values are type annotations. Alternative = or whitespace separators can also be used if preferred, athough the visual representation of default values will look less attractive if an = token is used for both the property delimiter and the devault value.
Property type specification files look very similar to property files, as they have a similar hierarchical structure.
As discussed previously, if a type specification is absent for a particular property, a binding for the property will be created with type std::string. Type specification files may thus only include type specifications for properties that have types other than the default std::string.
Regardless of type, it can be useful to include type specifications for certain std::string properties, in order to specify default values for those properties. The following type specification file is the same as the previous one, but with default values for worker count and cache TTL.
http.server.worker.count : int = 8
file.serve {
location : string
document.root : uri
cache.ttl : int = 3600
}
Configuration cascading
An application that uses the environment configuration framework will consume the following information:
- file based type specifications and default values;
- class based type specifications and default values;
- file based property values.
The information from these three sources is merged together to form a hierarchical set of value, unique, and shared bindings in the injector.
There are thus two merges that occur during the configuration of the injector:
- merging of type specification / default value hierarchies;
- merging of default values from the previous merge and property values.
The first type of cascade consists of the priority merging of type information and associated default values, sourced from potentially multiple configuration producing applications. Type specification / default value overrides can thus be specified, either in additional type specification files or hard coded in an EnvironmentConfiguration derived configuration class.
The second type of cascade consists of the merging of the environment's property values with the default values resulting from the type specification priority merge. Unlike the restrictions on the main property values, this value merge allows duplicate values to be specified. The default values in the merged type specifications are overridden by the environment's property values.
Example configuration
A full example is presented here (taken from the Balau tests). The example has a single type specification file, a single derived environment configuration class with hard wired type specifications, and a property value file.
The specification file contents is as follows.
http.server.worker.count : int = 6
value.multiplier : double = 123.456
file.serve {
location : string
document.root : uri
cache.ttl : int = 10000
options {
identity : string = Balau Server
404 : uri = file:404.html
}
}
The hard wired type specification class is as follows.
struct EnvConfig : public EnvironmentConfiguration {
EnvConfig(const File & env, const File & spec)
: EnvironmentConfiguration(env, spec) {}
void configure() const override {
value<int>("http.server.worker.count", 16);
value<double>("value.multiplier", 12.55e-3);
group("file.serve"
, value<std::string>("location", "/")
, value<int>("cache.ttl", 3600)
, group("options"
, value<std::string>("identity", "My Server")
);
);
value<double>("value.fraction", 0.432);
}
};
The environment's property value file is as follows.
file.serve {
location = /doc
document.root = file:doc
}
The intermediate trees generated from the three sources are illustrated in the following diagrams.
Once merged, the resulting hierarchical bindings defined in the injector are illustrated in the following diagram.
Design
This section provides a summary of some aspects of the philosophy and design of the hierarchical environment configuration framework. It is not necessary to read this section in order to use the framework.
Overview
The aim of the Balau environment configuration framework is to provide a way to specify environment specific, injectable configuration that:
- is type safe, via inter-environment type specifications;
- provides the possibility of specifying sensible, inter-environment defaults for properties;
- allows hierarchical property data to be specified;
- provides a hierarchical format that facilitates the specification of independent, composite hierarchical property subsets;
- has a familiar syntax;
- is programming language independent.
Background
Environment configuration in enterprise applications has been achieved in a variety of ways. One common technique is to supply environment configuration as a flat file containing key-value pairs. These key-value pairs are then loaded into the application injector to form injectable named string values. One common format for this approach is the .properties format often used in Java applications.
Other bespoke approaches to environment configuration include hierarchical configuration formats. Examples of these include XML based configuration (such as the format used in the configuration files of the Apache HTTP server), and curly bracket based hierarchical configuration blocks (the configuration files of the Nginx server being an example).
More recently, an elaboration on the traditional key-value approach has been to use YAML as a more visually informative format for key-value properties. This approach also allows string representations of complex values to be represented within the same physical structure as the key-value property structure (i.e. value lists defined in the standard YAML format).
Requirements
The key requirements determined during the design phase of the environment configuration framework were:
- provide a mechanism for composite properties, which represent standalone environment configuration property subsets, injectable as a single EnvironmentProperties class instance;
- allow types to be optionally specified for key-value properties;
- provide a mechanism for specifying sensible property defaults;
- separate configuration data into inter-environment data (property types, default property values) and intra-environment data (environment specific property values);
- provide an include mechanism, which allows a hierarchical property file to be split across multiple physical files;
- do not mix hierarchical structure with string representations of typed values (i.e. string representations of a complex data structure assigned to a value property);
- do not implement required/optional specifications for properties;
- base the hierarchical property format on a well known non-hierarchical property format, with minimal differences to the original format;
- do not infer the hierarchical structure from indentation, use the well known "{" and "}" block delimiter token pair instead;
- use the standard string type of the implementing programming language (e.g. std::string for C++) for value properties that do not have a corresponding type specification.
File format
The resulting format chosen for both hierarchical environment configuration data files and type specification files is described in detail in the property parser chapter. The format is based upon a hierarchical extension to the .properties file format. Composite properties are defined via "{" and "}" delimited blocks. The include mechanism uses the "@" token to specify a URI to be included.
Other than the addition of the special "{", "}" and "@" characters and the corresponding non-special, escaped "\{", "\}" and "\@" character pairs, the hierarchical property format is identical to the original non-hierarchical property format.
Specification files
Environment configuration type specification files represent inter-environment configuration data. This data is defined by an owning application and is shared across multiple environments and consuming applications. The two pieces of information provided for each value property are:
- the property's value type;
- the property's optional default value.
Value files
Environment configuration value files represent intra-environment configuration data. This data is unique to an environment, but can be shared across multiple applications for that environment. Environment configuration data files contain string representations of each value property contained in the configuration hierarchy.
If a type specification for a value property is specified in the cascade of the supplied type specification files, the value property will be typed. Otherwise, the property will be bound as a string.
Configuration cascading
An important part of the environment configuration design was getting the semantics of merging multiple type specification / default value and property value sources together. This involves the merging of information of the following types:
- file based type specifications and default values;
- class based type specifications and default values;
- file based property values.
There are three consequential information merges that could possibly occur:
- merging type specification / default values;
- merging property values;
- merging default values and property values;
Finalising the exact semantics of each type of information cascade was a major part of the design process of the environment configuration framework.
The first type of cascade consists of the priority merging of type information and associated default values, sourced from potentially multiple configuration producing applications. During the design analysis phase, it was concluded that duplicate type specification / default values should be allowed. This allows type specification / default value overrides to be specified, either in additional type specification files or hard coded in an EnvironmentConfiguration derived configuration class.
The second type of cascade would consist of the priority merging of the property values of the environment from multiple property value sources. This type of merging is not supported in the environment configuration framework. Taking into account that each property value becomes an injector binding, the presence of duplicate property values is effectively the same as the presence of duplicate value or unique application configuration bindings. As the injector does not allow such duplicate bindings, this restriction is carried through to property values. This is the case both within a single property value file processed by an EnvironmentConfiguration instance (property value duplication), and across multiple application and environment configuration instances (binding duplication) potentially specified to the injector.
The third type of cascade consists of the merging of the environment's property values with the default values resulting from the type specification priority merge. Unlike the restrictions on the main property values, this value merge does allow duplicate values to be specified. The default values in the merged type specifications are overridden by the environment's property values.
Consequently, there are only two types of information merge that occur in the environment configuration framework:
- merging type specification / default values;
- merging default values and property values;
No required properties
One aspect of the design that matured during the development of the environment configuration property framework was whether type specification files should have provision for required/optional property annotations.
Overview
When considering this proposal, the first impressions of all involved developers were that the capability of specifying which environment properties are required would improve the correctness and safety of consuming applications. This however changed after a more deep consideration of the implications of such a feature, including when taking into account the releases cycles of multiple applications in an enterprise. It was subsequently concluded that required/optional annotations would have a net negative effect on the development and release process.
When a single version of a consuming application is made in isolation, required/optional property annotations are clearly an advantage. Such annotations would provide both a visual indication for developers and a validation mechanism at application startup.
However, when a more global consideration is made which takes into account multiple environments, multiple applications, and multiple application versions, the required/optional annotation feature forms a likeness to the abandoned required/optional designs of wire data formats such as Protocol Buffers and Apache Thrift.
Analysis
The stated advantages of required/optional annotations in the environment configuration framework were:
- a validation mechanism run during application startup;
- a clear visual indication for developers that are creating or modifying environment configurations.
The primary disadvantage of required/optional annotations in the environment configuration framework is that once a property is marked as required in a producing application's type specification declarations, the release cycles of the suite of consuming applications in an enterprise would be constrained to be synchronised. Such a release cycle synchronisation is extremely inefficient in an enterprise of any complexity, and in the worst case would result in breakages in the contracts between applications.
This disadvantage is exactly the same disadvantage which led to the abandonment of required/optional annotations in wire data formats such as Protocol Buffers and Apache Thrift.
Alternatives
When examining in more detail the overall design, the stated advantages of required/optional annotations can be achieved via other means.
Validation
Modern enterprise quality software development is performed in the context of test driven development. When using a dependency injection framework, development should include testing of the application configuration. Such configuration testing is described in the injector chapter for the Balau injector.
When the injector configuration is tested, the validation mechanism provided via required/optional annotations is rendered unnecessary.
If, in exceptional circumstances, additional runtime validation is required, this additional validation can be placed within the configure method(s) of the EnvironmentConfiguration derived classes of the application.
Visual indications
Whilst visual indications of required properties for developers would be useful, the lack of required/optional annotations in type specification files can be mostly mitigated by prioritising the use of sensible property value defaults. The design of the hierarchical environment configuration framework includes provision for inter-environment property defaults, specified within the type specification files of producing applications.
If use of the default values feature is made a primary part of the development process, new properties that are introduced into the release of a component will not require the developers of consuming applications to "mend" their applications when upgrading to new versions of the consumed type specification files. Instead, the default values of new properties will be picked up automatically. Subsequent overriding of the new properties' default values can then be made if necessary at a later date.