As other languages, Elixir comes with an application configuration system used to statically configure applications at compile-time and runtime. The system consists of the Config module and various scripts in the config/ directory interpreted by the Mix compiler. Phoenix adds a conventional structure on top, pre-populating configuration files and organizing them by Mix environment.
As the size of config files grows about linearly with app complexity, developer time spent in these files increases likewise. In this blog post, I am going to showcase two small adjustments you can apply to the default application configuration setup in Phoenix apps to improve their developer experience.
Disclaimer: These techniques are likely not novel or mind-blowing, and very likely we do not have invented them. Nonetheless, they are worth looking at. My apologies to other authors for not providing links to related blog posts, which I may have read but have forgotten about since.
Recap: Factory defaults
When you run the mix phx.new generator to create a new Phoenix application, you are presented with a default set of application configuration files in the config directory:
my_app $ tree config/
config/
├── config.exs
├── dev.exs
├── prod.exs
├── runtime.exs
└── test.exs
0 directories, 5 files
While these files are scaffolded by code in the phx_new package, the original idea behind the setup is described in the Configuration section of the Mix documentation. Notably, only two of these scripts have technical relevance:
config.exsis executed by Mix at compile-timeruntime.exsis executed at runtime, before the application is booted
The remaining files’ content is specific to the Mix environment and loaded at the end of config.exs:
import_config "#{config_env()}.exs"
This mechanism is by no means part of Mix though. The documentation only says, ’it is common’ to set up an Elixir project this way. I guess it is safe to assume that it is inspired by config/environment.rb of Rails heritage.
Drawbacks
While this setup is functional and well-known by many, it has some drawbacks. Three come to mind, and I hope you can agree with them:
- When you work on these files, your file access pattern does not align well with their environment-based structure.
For example, when you add for a new library to your dependencies, you commonly edit the library configuration across all Mix environments, which means opening and navigating between (including config.exs) up to 4 files in your editor. Likewise, asserting what values a config variable may take across environments is difficult without a single-screen view of it. By constrast, in my experience, we almost never have any interest in looking at all variables for a specific environment at the same time. To summarize, the co-location of related code isn’t exactly the greatest in this setup.
- Especially related to Phoenix, as it inevitably sets standards with its boilerplate: The common use of “overrides” across files makes it easy to make wrong assumptions about values at runtime.
For example, setting a config variable in config.exs as a default for all environments and overriding it in dev.exs for :dev. To know the value from looking at the source, you need to open all of these files and locate it in any of them. Overriding it again in runtime.exs (or previously release.exs), e.g. based on an environment variable, used to be good for a little extra head scratching when you mistakenly accessed the variable at compile-time. The latter issue has been neatly resolved with the addition of Application.compile_env/3 in Elixir 1.10, but the essence is again that settings of a single variable may be distributed across several files, leading to bad discoverability and inconvenient editing.
- The files get exceedingly long and unwieldy to work with.
As every single bit of configuration for the :dev environment lives in dev.exs, config files grow infinitely alongside the application itself.
Now make some changes...
Whenever we started a fresh Phoenix project in recent years, one of the first things we did is rearrange the generated files in config/. For some reason, I like to refer to the following ideas as “rules”, even though the first is a refactoring and the second is more like a habit, and no-one is there to enforce rules anyway.
Rule 1: Split configuration files by topic
Here is a shortened listing of config/ from one of our projects:
config/
├── config
│ ├── endpoint.exs
│ ├── logger.exs
│ ├── i18n.exs
│ ├── mocks.exs
│ ├── oban.exs
│ ├── repo.exs
│ └── tesla.exs
├── config.exs
└── runtime.exs
1 directories, 9 files
As can be seen, we have a config/config subdirectory containing a handful of files with names such as endpoint.exs, i18n.exs, or oban.exs. These files are imported from config.exs using import_config/3 as usual, while config.exs is empty otherwise. Environment-specific values are then applied conditionally within the topic-based files like this:
config :my_app_web, MyAppWeb.Endpoint,
http: [ip: {127, 0, 0, 1}, port: 4000]
if config_env() == :prod do
config :my_app_web, MyAppWeb.Endpoint,
http: [ip: {0, 0, 0, 0, 0, 0, 0, 0}, port: 80]
end
Sometimes also like this, totally up to personal taste:
{ip, port} =
case config_env() do
:dev -> {{127, 0, 0, 1}, 4000}
:test -> {127, 0, 0, 1}, 4001}
:prod -> {{0, 0, 0, 0, 0, 0, 0, 0}, 80}
end
config :my_app_web, MyAppWeb.Endpoint,
http: [ip: ip, port: port]
Naming these files is predictably hard, but recall is great and you quickly learn to locate what you need in an instance. We often group related configuration in generalized topic-based files such as i18n.exs, which contains settings for gettext and ExCldr. Alternatively, we simply use a library name if no useful generalization is applicable (oban.exs). Discovering or editing a config variable now requires opening only one file instead of four, and config files generally remain short.
Rule 2: When it can be configured at runtime, it must be configured at runtime.
This reads more like a rule.
To avoid any potential confusion around config variables which are set to default values at compile-time in some environments (usually :dev and :test) and set from the system environment at runtime in others (usually :prod), we decided to not have this duality at all. The technical implementation is straightforward: If a variable needs to have a dynamic value at runtime and hence is configured in runtime.exs, it must not be defined in any of the compile-time config files at the same time. That includes setting all default values for local environments in runtime.exs. The “must not” is enforced by convention, resulting from the convenience of the setup.
Here’s an example of how one could restructure the default Ecto repository configuration (generated by phx_new 1.7.7) to align it with this rule.
Before:
config :my_app, MyApp.Repo,
username: "postgres",
password: "postgres",
hostname: "localhost",
database: "my_app_dev",
pool_size: 10,
...
if config_env() == :prod do
database_url =
System.get_env("DATABASE_URL") ||
raise """
environment variable DATABASE_URL is missing.
For example: ecto://USER:PASS@HOST/DATABASE
"""
config :my_app, MyApp.Repo,
url: database_url,
pool_size: String.to_integer(System.get_env("POOL_SIZE") || "10"),
...
end
After:
defmodule SystemConfig do
def get(envvar, opts \\ []) do
value = System.get_env(envvar) || default(envvar, config_env())
case Keyword.get(opts, :cast) do
nil -> value
:integer -> value && String.to_integer(value)
:boolean -> value in ["1", "true", "TRUE", true]
# ... more casts
end
end
defp default("POOL_SIZE", _), do: "10"
defp default("DATABASE_URL", env) when env in [:dev, :test] do
"ecto://postgres:postgres@localhost/my_app_#{env}"
end
defp default(key, env),
do: raise "environment variable #{key} not set and no default for #{inspect(env)}"
end
config :my_app, MyApp.Repo,
url: SystemConfig.get("DATABASE_URL"),
pool_size: SystemConfig.get("POOL_SIZE", cast: :integer)
Besides ensuring we can’t mix up compile-time and runtime, this gives us the benefit of control over the application in development using environment variables, for example:
# Try this in a newly generated Phoenix app.
# Hint: It won't work due to the config_env() == :prod condition in runtime.exs
POOL_SIZE=1 mix phx.server
But runtime.exs is even longer now!
Following the 12-factor app conventions, our apps exclusively use environment variables for runtime configuration (as opposed to reading .yaml files or similar), and we often end up with a runtime.exs files of several hundred lines.
Hence, the sketched solution above has its limits. There are in fact quite a few libraries offering more scalable ways of configuring Elixir apps at runtime. Yet so far, our hacky RuntimeConfig module in runtime.exs has served us well and we have not felt the need for anything else. Perhaps, as this file is a mere endless list of env vars and defaults values, it does not impose much cognitive load on the reader and it’s OK to let it grow a long beard. Besides, as pointed out before, Phoenix’ default configuration unavoidably suffers from the same issue at some point in the application lifecycle, and does so not only in runtime.exs but also config.exs and the environment-specific files.
Side-note: I am hoping for this limitation of runtime.exs to be eventually lifted, giving us the ability to import_config/1 nested files from a config/runtime/ directory within runtime.exs. Though, I understand that this requires more changes than just removing the raise, given how releases are assembled.
Summing up
Summing up, this blog post now consists of a lot of words for these two not-so-mind-blowing “adjustments”. Yet, they made an astonishing impact on the developer experience around config/ in our Phoenix apps. To the point that, when coming back to older projects or contributing to foreign projects, defining a variable in the environment-based setup feels a bit clumsy.
At bitcrowd, where we switch between different client projects a lot, we value conventions above everything. However, re-evaluating them at times can’t hurt and in this case we have decided to deviate from the standard. Perhaps someone feels inspired to do the same.
PS: If you enjoyed reading this, maybe you're interested in working with Elixir at bitcrowd? Check our job offerings!