Cross Compiling an Erlang Application

Aug 22, 2015

I am developing an Erlang application on my laptop for a production system that runs FreeBSD 10.x. By default, the compiled releases are for OSX/darwin and do not run on the production target. This post details how to cross compile an Erlang project for FreeBSD, however, it should be applicable to any operating system.

This short guide begins by creating a new project with rebar3 and ends with it running on both a development machine and a FreeBSD server. This picks up after the Getting Started guide for the rebar3 tool and covers project creation, build configuration, and distribution packaging.

Getting Started

The command rebar3 should be available and return output similar to below:

 1 $ rebar3
 2 Rebar3 is a tool for working with Erlang projects.
 3 
 4 
 5 Usage: rebar [-h] [-v] [<task>]
 6 
 7   -h, --help     Print this help.
 8   -v, --version  Show version information.
 9   <task>         Task to run.
10 [ snipped ]

First, create a new project called myapp; change directory (cd) to the new directory myapp afterwards.

 1 $ rebar3 new release myapp
 2 ===> Writing myapp/apps/myapp/src/myapp_app.erl
 3 ===> Writing myapp/apps/myapp/src/myapp_sup.erl
 4 ===> Writing myapp/apps/myapp/src/myapp.app.src
 5 ===> Writing myapp/rebar.config
 6 ===> Writing myapp/config/sys.config
 7 ===> Writing myapp/config/vm.args
 8 ===> Writing myapp/.gitignore
 9 ===> Writing myapp/LICENSE
10 ===> Writing myapp/README.md
11 $ cd myapp
12 tree
13 .
14 ├── LICENSE
15 ├── README.md
16 ├── apps
17 │   └── myapp
18 │       └── src
19 │           ├── myapp.app.src
20 │           ├── myapp_app.erl
21 │           └── myapp_sup.erl
22 ├── config
23 │   ├── sys.config
24 │   └── vm.args
25 └── rebar.config
26 
27 4 directories, 8 files

Next, open apps/myapp/src/myapp_app.erl and add a short debug statement to the start function. It should look like the function below.

1 start(_StartType, _StartArgs) ->
2     io:format("myapp is starting up~n"),
3     'myapp_sup':start_link().

Finally, start the application with rebar3 shell and confirm the above line is displayed.

 1 $ rebar3 shell
 2 ===> Verifying dependencies...
 3 ===> Compiling myapp
 4 Erlang/OTP 18 [erts-7.0.3] [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [dtrace]
 5 
 6 Eshell V7.0.3  (abort with ^G)
 7 1> ===> The rebar3 shell is a development tool; to deploy applications in production, consider using releases (http://www.rebar3.org/v3.0/docs/releases)
 8 myapp is starting up
 9 ===> Booted myapp
10 ===> Booted sasl
11 [ snipped ]
12 1> q().
13 ok
14 $

The program starts and prints out the debug line; success!

Packaged Release

Now to package it for distribution. To do this, run the tar command with rebar3 as prod tar.

 1 $ rebar3 as prod tar
 2 ===> Verifying dependencies...
 3 ===> Compiling myapp
 4 ===> Starting relx build process ...
 5 ===> Resolving OTP Applications from directories:
 6           myapp/_build/prod/lib
 7           myapp/apps
 8           /usr/local/Cellar/erlang/18.0.3/lib/erlang/lib
 9 ===> Resolved myapp-0.1.0
10 ===> Including Erts from /usr/local/Cellar/erlang/18.0.3/lib/erlang
11 ===> release successfully created!
12 ===> Starting relx build process ...
13 ===> Resolving OTP Applications from directories:
14           myapp/_build/prod/lib
15           myapp/apps
16           /usr/local/Cellar/erlang/18.0.3/lib/erlang/lib
17           myapp/_build/prod/rel
18 ===> Resolved myapp-0.1.0
19 ===> tarball myapp/_build/prod/rel/myapp/myapp-0.1.0.tar.gz successfully created!

Here is what is happening:

as selects the profile prod, which is defined inside rebar.config. This profile was created by the new command executed initially.
tar is a command that creates an erlang application release and packs it into a tar file.

Local Machine

On the local development machine, extract the archive and run the release with bin/myapp-0.1.0 foreground.

 1 $ mkdir -p ~/unzip
 2 $ tar xf _build/prod/rel/myapp/myapp-0.1.0.tar.gz -C ~/unzip
 3 $ cd ~/unzip
 4 $ bin/myapp-0.1.0 foreground
 5 Exec: erts-7.0.3/bin/erlexec -noshell -noinput +Bd -boot releases/0.1.0/start -mode embedded -config releases/0.1.0/sys.config -boot_var ERTS_LIB_DIR erts-7.0.3/../lib -args_file releases/0.1.0/vm.args -- foreground
 6 Root: ~/unzip
 7 myapp is starting up
 8 
 9 [ snipped ]
10 ^C
11 $

Success! (CTRL+C to exit)

Remote Machine

Copy the archive to a remote system of a different operating system. The rest of this post assumes the remote system is FreeBSD 10.x.

 1 $ scp _build/prod/rel/myapp/myapp-0.1.0.tar.gz remotesystem
 2 myapp-0.1.0.tar.gz                                                                          23% 1424KB 303.8KB/s   00:15 ETA
 3 $ ssh remotesystem
 4 remotesystem> uname -a
 5 FreeBSD remotesystem 10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274401: Tue Nov 11 21:02:49 UTC 2014     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64
 6 remotesystem> mkdir myapp
 7 remotesystem> tar xf myapp-0.1.0.tar.gz -C myapp/
 8 remotesystem> cd myapp
 9 remotesystem> bin/myapp-0.1.0 foreground
10 exec: myapp/erts-7.0.3/bin/erlexec: Exec format error
11 Exec: myapp/erts-7.0.3/bin/erlexec -noshell -noinput +Bd -boot myapp/releases/0.1.0/start -mode embedded -config myapp/releases/0.1.0/sys.config -boot_var ERTS_LIB_DIR myapp/erts-7.0.3/../lib -args_file myapp/releases/0.1.0/vm.args -- foreground
12 Root: myapp
13 exec: myapp/erts-7.0.3/bin/erlexec: Exec format error
14 remotesystem>

Not a success!

Preparing to Cross Compile

According to the documentation, a different Erlang distribution can be used by setting the include_erts and system_libs configuration settings in the target profile (prod).

Getting the Distribution

The two ways I have gotten an Erlang distribution for another operating system are:

Install onto a system and copy the directories
Download and extract the correct package from a package manager

The follow steps cover the second option - downloading and extracting a binary distribution.

Binary packages for FreeSBD 10.x (amd64) are available here. As of August 23, 2015, the version available is erlang-18.0.3,3.txz (download). If this is out of date, visit the link above and search for erlang-.

1 $ wget http://pkg.freebsd.org/freebsd:10:x86:64/latest/All/erlang-18.0.3,3.txz
2 $ mkdir -p ~/unzip/erlang/freebsd
3 $ tar xf erlang-18.0.3,3.txz -C ~/unzip/erlang/freebsd
4 $ mv ~/unzip/erlang/freebsd/usr/local/* ~/unzip/erlang/freebsd
5 $ rm -rf ~/unzip/erlang/freebsd/usr/local/

Update rebar.config to use this alternative erlang and erts distribution.

 1 {erl_opts, [debug_info]}.
 2 {deps, []}.
 3 
 4 {relx, [{release, {'myapp', "0.1.0"},
 5          ['myapp',
 6           sasl]},
 7 
 8         {sys_config, "./config/sys.config"},
 9         {vm_args, "./config/vm.args"},
10 
11         {dev_mode, true},
12         {include_erts, false},
13 
14         {extended_start_script, true}]
15 }.
16 
17 {profiles, [{prod, [{relx, [{dev_mode, false},
18                             {include_erts, "/ABSOLUTE/PATH/unzip/erlang/freebsd/lib/erlang"},
19                             {system_libs, "/ABSOLUTE/PATH/unzip/erlang/freebsd/lib/erlang"}]}]
20             }]
21 }.

Naming Nodes

Before the final build, there is one more change to make.

By default, config/vm.args will try to use a long name. A long name must follow the format node@host and be resolvable, otherwise the application will not start. A short name, however, is just node. Use a short name by changing -name myapp to -sname myapp.

1 -sname myapp
2 
3 -setcookie myapp_cookie
4 
5 +K true
6 +A30

Creating the Archive

Create the final artifact by executing the tar command again.

 1 rebar3 as prod do clean, tar
 2 ===> Cleaning out myapp...
 3 ===> Verifying dependencies...
 4 ===> Compiling myapp
 5 ===> Starting relx build process ...
 6 ===> Resolving OTP Applications from directories:
 7           myapp/_build/prod/lib
 8           myapp/apps
 9           ~/unzip/erlang/freebsd/lib/erlang
10           myapp/_build/prod/rel
11 ===> Resolved myapp-0.1.0
12 ===> Including Erts from ~/unzip/erlang/freebsd/lib/erlang
13 ===> release successfully created!
14 ===> Starting relx build process ...
15 ===> Resolving OTP Applications from directories:
16           myapp/_build/prod/lib
17           myapp/apps
18           ~/unzip/erlang/freebsd/lib/erlang
19           myapp/_build/prod/rel
20 ===> Resolved myapp-0.1.0
21 ===> tarball myapp/_build/prod/rel/myapp/myapp-0.1.0.tar.gz successfully created!

as selects the profile prod
do allows executing several commands in a single call
clean will clean the environment of previous compiled files
tar creates a tar archive and executes required steps (compile)

Finale

scp the newest archive to the remote system and run it!

1 $ bin/myapp foreground
2 Exec: myapp/erts-7.0.3/bin/erlexec -noshell -noinput +Bd -boot myapp/releases/0.1.0/start -mode embedded -config myapp/releases/0.1.0/sys.config -boot_var ERTS_LIB_DIR myapp/erts-7.0.3/../lib -args_file myapp/releases/0.1.0/vm.args -- foreground
3 Root: myapp
4 myapp is starting up
5 
6 [ snipped ]

Success!