Tags

,

I’m in the process of distributing an erlang application over many nodes, which is different from distributing control (via kernel configuration) of the application. In so doing, a number of releases need to be generated, each with their own set of app.config and vm.args. Using rebar and reltool, this process is actually quite simple. Here are a few tips for making it work.

Preliminaries

In reltool.config, change a copy directive to a template directive. This will route the source file to the variable interpolation process. While basic, it is easy to overlook this and you may wind up with a rug burn on your head while trying to determine why none of your {{variables}} are being interpolated.

{overlay, [
           {mkdir, "log/sasl"},
           {copy, "files/erl", "{{erts_vsn}}/bin/erl"},
           {copy, "files/nodetool", "{{erts_vsn}}/bin/nodetool"},
           {<strong>template</strong>, "files/app.config", "etc/app.config"},
           {<strong>template</strong>, "files/vm.args", "etc/vm.args"}
           ]}.

Default Variables

The vars.config is generally used as the overlay file, which is specified in the reltool.config via the overlay_vars key. This file provides the values to populate a template, which is applied to any file specified for templating. These values can be overridden later by specifying another overlay vars file. Having vars.config as a default is great when you need to support both single node dev applications and multi-node production applications. Basically, the vars.config acts as the dev version, so it requires minimal configuration for the team, while my overlays exist for the production environment when I distribute the applicatio

The structure of the vars.config are key-value pairs as tuples. Keys are atoms that reference variable names surrounded by two curly braces in the template file. Values are really simple values such as numbers or strings. If you attempt to enter an arbitrary term as the value, you’ll be rudely surprised. Terms can, however be represented in string form, and when properly interpolated, the app.config will contain the proper term structure. When using strings or binary strings, a similar cautionary note is required: Strings in the vars.config appear as raw text, so any binary string values need to have the variable surrounded in quotation marks.

Wrong

This example will not work since the value in the vars.config is provided as a binary string.

app.config vars.config

{amqp_host, {{amqp_host_variable}}}


{amqp_host_variable, <<"127.0.0.1">>}.

Right

Instead, only the string part of the value should be specified, with the binary string syntax left in the app.config.

app.config vars.config

{amqp_host, <<"{{amqp_host_variable}}">>}


{amqp_host_variable, "127.0.0.1"}.

Basically if you remember that the interpolated values should be simple types, then it will work as expected. Trying to get fancy will result in disappointment.

Alternate Overlays

As noted above, alternate overlays can be used by specifying an overlay_vars during the rebar generate process. Rather than using this file in place of the one specified in overlay_vars, values contained in this config override any existing values defined in the original vars.config. This means you can use basic (sensible) defaults in the vars.config and then create overrides in as many alternate configs as you want (one for each node). Hence any common configuration options can be defined once in the default vars.config and node-specific values are set in individual node configs. The relationship is analogous to a base class and an inheriting sub-class. If a release is composed of many nodes, it is easy to set up make targets for each node, so that a different overlay is used for each.

One-to-Many Templating

Multiple templates can be defined, and the same set of variables will be used for each template file. The great thing about this is that you can create variables in your vm.args for the node name and set that in the overlay vars. Any unused variables are simply ignored, as you would expect.

Data flow for generating templated configuration files

This regime is fairly flexible though it does have some limitations. Clearly there is a risk of naming collisions, so be sufficiently verbose.

Conclusion

When planning out a multi-node release, reltool and rebar can ease the burden of managing configuration files. Each environment can have its own target within a Makefile, which subsequently generates separate releases with their own overlay configuration. Each node-specific configuration file can specify node names, cookies, and other details to ensure a smooth release process.

The only outstanding piece is to join the nodes together so they can communicate. If running all nodes on a single machine, don’t be fooled that epmd sees them all. Even though they are all listed, the nodes don’t actually know about each other until you execute an explicit ping.