Elixir Umbrella Applications and Testing with Mox

#anchorBreethe

Throughout this post, we will use Breethe as an example. Breethe is a Progressive Web App that gives users quick and easy access to air quality data for locations around the world. Pollution and global warming are getting worse. The first step to understanding and solving these problems is to raise awareness by providing everyone with easy access to accurate data.

The application is open source and we encourage everyone interested to look through the source for reference. The server for this application was implemented using an Elixir umbrella application which will be the focus of this post. The client for Breethe was built with Glimmer.js, which we discussed in previous posts:

• From SPA to PWA
• Building a PWA with Glimmer.js

#anchorUmbrella applications and separating concerns

When we first started building Breethe, we asked ourselves a simple question which would dictate the structure of the application and our motivation for using an umbrella app to organise our code. This question was: what if we want to change our air quality data provider? It turns out this wasn't just speculation as we are now in the process of doing just that and our decision to use an umbrella app will make the process tremendously easy.

Using an umbrella allowed us to split our server into two very distinct applications, communicating together by way of rigorously defined APIs. The first application functions as the data handling entity of the project - named breethe (see below). It communicates with the air quality data provider (a third-party API) to gather the data, then processes and caches it for future use. The second application in the umbrella is the web interface built with Phoenix - named breethe_web. It requests the data from the first application, serializes it to JSON and delivers the payload to the client in compliance with the JSON:API specification.

Here's an overview of the umbrella structure used for Breethe:

apps
├── breethe
│   ├── README.md
│   ├── config
│   ├── lib
│   ├── mix.exs
│   ├── priv
│   └── test
└── breethe_web
    ├── README.md
    ├── config
    ├── lib
    ├── mix.exs
    ├── priv
    └── test

We have defined a clear boundary between the business logic and the webserver. This is cool because the umbrella becomes modular like Lego and who doesn't like Lego? Need to change the air quality data provider? No problem, simply change the data application, leaving the webserver untouched as long as the data app continues to implement the same interface. The same would work the other way round if we wanted to change the webserver.

However, for this approach to work well, the APIs used to communicate between the different applications in the umbrella need to be carefully defined. We want to keep the interfaces as little as possible to keep complexity contained. As an example, here are the publicly available functions on the breethe app in the umbrella:

# apps/breethe/lib/breethe.ex
def get_location(location_id), do: # ...

def search_locations(search_term), do: # ...

def search_locations(lat, lon), do: # ...

def search_measurements(location_id), do: # ...

Equally, these are the only functions the Phoenix web app (or any other app in the umbrella) can call on the breethe app. These principles are of course not only applicable at the top level of the application but also within its internal logical contexts. For example, within the breethe app, we have isolated the functions explicitly making requests to third-party APIs and abstracted them away behind an interface. This, again, reduces complexity and facilitates testing as we can isolate the different components of the business logic. This philosophy lends itself very well to being tested using Mox.

#anchorTesting domains independently using Mox

Mox, as the name suggests, is a library that defines mocks bound to specific behaviours. A behaviour is a set of function signatures that must be implemented by a module. Consequently, Mox guarantees the mocks for a module be consistent with the original functions they replace during testing. This rigidity makes the tests more maintainable and requires that the behaviours for each module be meticulously defined; precisely the qualities desired when implementing the APIs within our umbrella.

For example, let's consider mocking the public API for the breethe application when testing breethe_web. As the bridge between the two is only composed of the four functions shown in the previous section, mocking the breethe application's public interface when testing the webserver only requires mocking those four functions. Naturally, this is only reasonable if we separately test the breethe application in full, from interface to database. Crucially, it is the singularity of the interface which allows this degree of separation between the two applications in the umbrella both in testing and in production.

Let's take a look at the controller action for a location search by id:

# apps/breethe_web/lib/breethe_web/controllers/location_controller.ex
@source Application.get_env(:breethe_web, :source)

def show(conn, %{"id" => id}) do
  location =
    id
    |> String.to_integer()
    |> @source.get_location()

  render(conn, "show.json-api", data: location, opts: [])
end

The interesting part is in the call to the breethe application:

|> @source.get_location()

The reason we're using the @source module attribute is to be able to switch between the mock and the real function defined on the breethe application; this is defined in the config files:

# config/config.exs
config :breethe_web, source: Breethe

# config/test.exs
config :breethe_web, source: Breethe.Mock

By default @source points to the Breethe module - the breethe application's public API used in production and development. During testing it switches to the Breethe.Mock module, which defines the mocks.

The test for this controller action is meant to check two things. Firstly, that the router redirects the connection to the appropriate controller action. Secondly, that the controller action processes the call and queries the breethe application correctly using the right function defined on the latter's API - in this case get_location(location_id).

# apps/breethe_web/test/breethe_web/controllers/location_controller_test.exs
describe "show route: returns location" do
  test "by id" do
    location = insert(:location, measurements: [])

    Breethe.Mock
    |> expect(:get_location, fn _location_id -> location end)

    conn = get(build_conn(), "api/locations/#{location.id}", [])

    assert json_response(conn, 200) == %{
             "data" => %{
               ...
             }
           }
  end
end

I've broken it down into its four main parts:

1. It sets up the test data with ExMachina.
   
   

location = insert(:location, measurements: [])

2. It defines a mock in the Breethe.Mock module for the get_location(location_id) function defined in the breethe application's API and sets the return value to the location we created at 1. The mock is passed as an argument to the expect clause which verifies the mock is executed during the test (instead of the real function).
   
   

Breethe.Mock
|> expect(:get_location, fn _location_id -> location end)

As long as we’ve established the callback in the behaviour implemented by the Breethe module, we don’t need to explicitly define the Breethe.Mock module (Mox creates it). Here's the callback for this particular function (for reference, it isn't coded in the test).

# apps/breethe/lib/breethe.ex
defmodule Behaviour do
  @callback get_location(location_id :: integer) :: %Breethe.Data.Location{}
end

3. It builds a connection and makes a call to the webserver's route designed to handle a location search by id.
   
   

conn = get(build_conn(), "api/locations/#{location.id}", [])

4. It tests the JSON response (abridged for brevity) is correct by asserting on the attributes of the location created in 1. and returned from the mock in 2.
   
   

assert json_response(conn, 200) == %{
          "data" => %{
            ...
          }
        }

Using mocks greatly simplifies the testing process. Each test can be smaller and more specific. Each test is faster as we are not making calls to the database or external systems; we are only running the anonymous functions that define the mocks. For instance, the mock in our example above only executes:

fn _location_id -> location end

Finally, each mock is self-contained in the test defining it and the callback insures the mock matches the original function signature. The result is robust, fast and modularised tests.

#anchorConclusion

Elixir umbrella apps shine when structuring projects containing clear boundaries between their constituent parts. The philosophy they implement deeply resembles that of functional programming (and Lego), where small building blocks combine into a larger whole. It is however important to be precise when defining the internal APIs of the application as they act as the glue holding everything together. Lastly, Mox is a wonderful tool for testing. Not only does it make mocking APIs very simple and elegant, it also encourages best practices such as defining behaviours to keep the code consistent and robust.