Back to all posts

The interaction of API calls w/ Ecto.preload and Mox


In previous articles, the AK Alembic1 has discussed building API clients, but we left out how they’d be used in an application. For this discussion, let’s say you’re working with a blog-style application whose principal resources are posts and comments. Posts can have many comments, however these comments are managed through an API. In your Post controller you’ll use Ecto to load the post, but how do you get the comments?

Ecto.preload/3 can take a function. The docs state “the function receives the IDs to be fetched and it must return the associated data… This is useful when the whole dataset was already loaded or must be explicitly fetched from elsewhere”. In our case, ‘elsewhere’ will be an API. To slightly adjust the example code:

comment_preloader = fn post_ids -> CommentsApi.fetch_comments_by_post_ids(post_ids) end
Repo.all from p in Post, preload: [comments: ^comment_preloader]

If you’re testing with Mox, you’ll want to add something like this to your test:

CommentsApi.ClientMock
|> expect(:fetch_comments_by_post_ids, fn(_ids) -> comments end)

This will succeed… Until you add an additional preloader, in which case your test will raise an exception like:

(Mox.UnexpectedCallError) no expectation defined for Comments.fetch_comments_by_post_ids/1 in process #PID<#.#.#>

This outcome was initially surprising. Once I realized what was happening I classified it as “a good surprise”. If you visited the Ecto docs you’d see there’s also an option in_parallel, with the text “If the preloads must be done in parallel. It can only be performed when we have more than one preload and the repository is not in a transaction. Defaults to true”. So, by default, Ecto will preload each association in a different process. The actual code can be found here, and it uses Task.async_stream/3. However, the documentation for the Mox library states “All expectations are defined based on the current process”. That expectations are defined within a single process is the culprit behind our issue.

Thankfully, the solution is pretty simple. Mox supports multi-process collaboration, so you can adjust your tests with:

setup :set_mox_global

Now you can preload multiple associations through APIs and write your tests without making real HTTP Calls.

A Note on Application Boundaries

Since it’s been nearly 3 years since José Valim published “Mocks and explicit contracts”, it’s worth reiterating the advice on application boundaries. If you’re mocking an HTTP library’s response in your application code you should adjust your application boundaries. For this example, since we’re expecting a list of Comments, we create a mock that returns a list of Comments. In practice, this has meant that one of the first steps to integrating an API client with an application is often to write a behaviour for that client.

defmodule CommentsApi.Client do
  @callback fetch_comments_by_post_ids([integer]) :: %CommentsApi.Comment{}
end

Then we can create a module which delegates to the actual client:

defmodule CommentsApi do
  @client Application.get_env(:my_app, :comments_api_client)

  defdelegate fetch_comments_by_post_ids(ids), to: @client
end

In production, the client will be set to the real API client, but in tests, it’ll be a mock like:

Mox.defmock(CommentsApi.ClientMock, for: CommentsApi.Client)

1 I wanted to acknowledge that the name AK Alembic was coined by my colleague Josh Adams. An alembic is an alchemical still, and from my perspective, a perfect name for our Elixir ‘distillations’.


Do you love Elixir? So do we!

Annkissam has been building production applications with Elixir since 2015 and we're excited to leverage the experience we've gained over the years to help others use this amazing technology.