Exploring the new native env file support in Node.js

Published on Oct 10, 2024 by Ian Homer

Section III on Config in The Twelve-Factor App from 2011 provided architectural principles guiding the benefits of separating configuration from the code. It set a gold standard that led the way for environment variable practices and in particular how environment variables can be bound to a running process. For example, this approach plays in an important part in ENV support in Dockerfiles.

On Sep 04, 2023 Node v20.6.0 was released, providing built-in .env file support. Let's look at that functionality, look at how it has evolved since this initial release and look at alternative approaches available.

Node .env file support

Let's create an .env file. This file is a list of name value pairs for the environment variables we want to set. For example it might look like:

FOO=from-env
BAR=from-env

We can run a node process to inspect the environment variables that have been set.

node --env-file .env -e "console.log(process.env)"

This command prints out process.env. This is an object containing the environment variables for the process and returns all the environment variables from your shell, along with the environment variable from the env file that was injected. We can see that the environment variables from our .env file have been injected.

{
  USER: 'ian',
  LANG: 'en_GB.UTF-8',
  HOME: '/Users/ian',
  FOO: 'from-env',
  BAR: 'from-env',
  ...
}

Now that's pretty neat. No libraries needed and the environment file injection comes out of the box. Let's continue to explore when we need more that these basics and when we may use other approaches.

Multiple .env files

We can create another file, perhaps one with variations for a development environment called .env.dev.

BAR=from-dev

Multiple --env-file arguments can be referenced on the node command line.

node --env-file .env --env-file .env.dev -e "console.log(process.env)"

The last env file referenced overrides from previous env files, which we can see from this output.

{
  FOO: 'from-env',
  BAR: 'from-dev',
  ...
}

Overriding environment variables

If we have set a variable in shell, e.g. with export FOO=from-shell or for a single command by prefixing command with parameter assignments, e.g. setting the variable with NAME=<value> before the command, we see that the value from the shell takes precedent.

FOO=from-mod node --env-file .env --env-file .env.dev -e "console.log(process.env)"

{
  FOO: 'from-mod',
  BAR: 'from-dev',
  ...
}

Setting environment variables for a single command by prefixing with parameter assignments is a common feature for Bourne like shells.

For bash shell see bash environment documentation
In fish shell it was available from fish v3.1

You can test this quickly with

FOO=from-mod node -e "console.log(process.env.FOO)"

or with a more generic shell command

FOO=from-mod env

both of which should show you that the process has access to the environment variable that was set.

Passing in NODE_OPTIONS

The processing of the --env-file argument takes place as the node starts up and crucially before the Node environment variables are processed. For example we can now set up the inspector for debugging.

Create the file .env.debug with the arguments to wait for the debugger to connect and take control.

NODE_OPTIONS=--inspect-brk

Open up the node debugger in Chrome DevTools. You can open this from View -> Developer -> DeveloperTools in the Chrome browser menu. Then click on the "Open dedicated DevTools for Node.js" icon.

Now run the command with this env files

node --env-file .env.debug -e "console.log(process.env.FOO)"

You should see the debugger attaching

Debugger listening on ws://127.0.0.1:9229/c84d172c-bf64-43b1-a525-b9d2d143ae5e
For help, see: https://nodejs.org/en/docs/inspector
Debugger attached.

You can then debug the process in the Chrome DevTools.

Debugger from env-file

Multi-line values

v20.1.20 (March 26 2024) brought support for multi-line values #51289.

Create an .env file with a variable that spreads across multiple lines. Note that the value should be wrapped in double quotes to mark the start and the end of the value.

FOO="Value
over
many
lines"

And then run

node --env-file .env -e "console.log(process.env.FOO)"

Which will return the multi-line value

Value
over
many
lines

Optional environment files

Imagine you have some default configuration in place that is a good starting default. It is useful for the presence of the env file to be optional, such that it can be provided only if any of the defaults need to be overridden.

From Node.js v22.9.0 (Sep 17 2024) we can now use the --env-file-if-exists argument.

node --env-file-if-exists .env.opt -e "console.log(process.env.FOO || 'default')"

Which, when the file .env.opt does not exist, runs without error.

.env.opt not found. Continuing without it.
default

If we use the -env-file argument we see that the process exits with an error.

❯ node --env-file .env.opt -e "console.log(process.env.FOO)" || echo "oops"
node: .env.opt: not found
oops

When we create the file and set the variable, we now have access to the variable.

> echo FOO=from-opt > .env.opt
> node --env-file-if-exists .env.opt -e "console.log(process.env.FOO || 'default')"
from-opt

Ignoring the export keyword

For compatibility purposes, it is also convenient that the env file loading in Node.js ignores the export keyword.

With the .env file

export FOO=from-env

the FOO variable is loaded OK

> node --env-file .env -e "console.log(process.env.FOO)"
from-env

Some other env file tooling, such as direnv uses .envrc which are treated as scripts and need this export keyword. I'll go in to a little more detail on direnv below. Ignoring this keyword helps with compatibility of these files. It's not a guarantee of full compatibility, but it helps.

Loading env file programmatically

From v20.12.0 (26th March 2023) we can load environment files programmatically. The default .env file is loaded by calling process.loadEnvFile().

> node -e "process.loadEnvFile() ; console.log(process.env.FOO)"
from-env

Or we can load from a specific env file.

❯ node -e "process.loadEnvFile('.env.dev') ; console.log(process.env.BAR)"
from-dev

Programmatic access is useful if you wanted to load different env files in different contexts. Note that this is loaded after the node process has started so setting of the env variable NODE_OPTIONS will have no effect.

Parsing env files directly

We can process env files directly with the util.parseEnv function. This takes the raw content of a .env file and returns an object containing the variables.

❯ node -e "console.log(util.parseEnv('FOO=from-parse\n'))"
{ FOO: 'from-parse' }

I can see this being useful when:

you don't want the variables to be available as global process environment variables, perhaps credentials that are needed only for a specific integration, but must not be available to other parts of the stack.
you wish to validate environment files.
you need different values to be loaded from different contexts within a single process.

dotenv and dotenvx libraries

This native functionality in Node.js is powerful, however it is not yet a drop in replacement for the awesome dotenv library and the "better dotenv" dotenvx from the creators of dotenv. The dotenv and dotenvx libraries provide functionality over and above what is (currently) natively available in Node.js, such as:

Overriding environment variables from the shell. Node.js always takes the value from the shell if it has been set.
Variable expansion - reference and expand variables already set with ${FOO}.
Command expansion, e.g. $(whoami)
Encrypted .env files
Next.js env loading convention, such as the environment specific loading of env files following the naming convention of .env.$(NODE_ENV).

See the extensive set of examples dotenvx for more details.

direnv

It is also worth mentioning the direnv command. This is an extension for shell and can be used for more than Node.js. It works by loading (and unloading) environment variables on a per-directory basis.

You'll need to install direnv locally and hook it into your shell.

Once you have it installed, you can create an .envrc in a directory

echo export FOO=from-direnv > .envrc

And when you change into that directory, or a child of that directory, you'll get a message from direnv asking whether you'd like to approve the content. You'll also be asked for this approval whenever the .envrc file changes, perhaps by you directly or via a git pull. This protection is required, since this .envrc is a script that is run and could have an unintended side effect.

direnv: error /home/bob/project/my-env/.envrc is blocked. Run `direnv allow` to approve its content

When you allow the content with direnv allow, you'll get a message indicating that environment variables are loaded.

direnv: loading /home/bob/project/my-env/.envrc
direnv: export +FOO ~XPC_SERVICE_NAME

If you move out of the directory, e.g. cd .., you'll get a message indicating that the environment variables are unloaded.

direnv: unloading

Read the direnv docs for many examples and patterns of usage.

Summary

In practice, I have been using direnv a lot. It satisfies many of my use cases and is incredibly flexible, working for all runtimes and commands (not just Node.js). This is especially useful since I do most of my work in the command line and I can be in full control of what environment variables get loaded for each project or directory I am in. These environment variables then are used by any command that I happen to be running, whether that be a node process or something else.

For Node.js centric projects, I have found the dotenv and more recently the dotenvx project invaluable. It is straightforward to configure how the components of your system load in each environment, whether that be local development machines or production. Often in production, environment variables are injected from the container such as Docker, but having the components wired with dotenv gives excellent flexibility.

The new Node.js features mean that in many cases this native functionality will suffice without the need of the dotenv library. Yes, there are useful features in dotenv, but by default I'd probably start a project with the native env file handling and only move to dotenv library if the extra features were valuable. Features I often rely on are the variable expansion and Next.js environment loading convention.

A year ago, I'd consider bringing dotenv library in early on a project. Nowadays, with the new native handling of env files, some projects may not benefit from the extra features the dotenv library provides. If the native functionality started supporting other features, such as the variable expansion, the case for the dotenv library will weaken. It is also very useful that the native functionality can inject NODE_OPTIONS and other node environment variables. Supporting native env file in Node.js was a smart move and well worth exploring.