A tour of Elixir performance & monitoring tools

NASA Mission Control Room

Updated 4/23/17: AppSignal has exited BETA and Scout has released a BETA agent.

You’re diving into Elixir and are getting close to releasing your first production Elixir app. How do you profile and monitor it in production?

Below is a tour of what’s available in the Elixir ecosystem today. Note that many tools are Erlang-based, which you have access to via Elixir.

Metric Collection

Generic metric collection is the sensor array for your app: a counter for user signups here, collecting average call times for a critical HTTP call there, etc. These services assist with collecting and aggregating metrics and send the metrics off to another service (ex: Graphite, InfluxDB, Datadog, etc) for visualizations and alerting.

Exometer

Ulf Wiger and Magnus Feuer’s Exometer is the defacto Erlang instrumentation package. There are several parts to Exometer:

1. Recording Metrics: you can tell Exometer to update values and have Exometer run functions at intervals of your choosing to record values.
2. Storing Metrics: raw metrics are stored in via ETFs in the Beam VM.
3. Reporting Metrics: metrics are aggregated then forwarded onto external systems like Graphite, AMNQ, or StatsD.

The following story by Michael Schäfermeyer provides a great overview of Exometer:

Monitoring Phoenix_As lots of engineers in the Elixir community I have a ruby background, using rails as my go-to MVC framework._medium.com

Elixometer

Pinterest’s Elixometer is a light Elixir wrapper around Exometer that makes defining/updating metrics easier, plus has a nice @timed annotation to easily time a function’s execution time:

# Timing a function. The metric name will be [:timed, :function]# Key will be: prefix.dev.timers.timed.function@timed(key: "timed.function")def function_that_is_timed doOtherModule.slow_methodend

BEAM Resource Usage

There are several tools for monitoring the resource usage of the BEAM VM, which runs your Elixr and Erlang apps. BEAM feels more like an operating system versus a typical VM and the monitoring tools for it have the sophistication to match.

Observer

The default goto is Observer: it’s likely already installed on your computer. Try launching Observer via iex:

iex(9)> :observer.start

Observer can also connect to a remote system.

Wombat

A full-featured operations and maintenance framework, Wombat is commercial software available via Erlang Solutions.

Recon

Recon is one of the modules in the Erlang Recon application. The module provides access to high-level resource usage information of the Beam VM.

For example:

iex(11)> :recon.info(self)

[meta: [registered_name: [],memory_used: [memory: 143208, message_queue_len: 4, heap_size: 10958,total_heap_size: 17730,garbage_collection: [max_heap_size: %{error_logger: true, kill: true,size: 0}, min_bin_vheap_size: 46422, min_heap_size: 233,...

Application Monitoring

Application performance monitoring (APM) services instrument your Elixir function calls and provide a breakdown of performance via transaction traces. Most APM tools are orientated around monitoring web requests vs. general performance monitoring.

AppSignal

AppSignal has an officially-supported app monitoring agent for Elixir.

Scout

At my day job, we’ve released an app monitoring agent for Elixir. Learn more.

New Relic

New Relic does not have an officially supported agent, but there are a couple of community-developed agents available:

• New Relixir
• new_relic — the author of the new_relic module (Roman Smirnov) mentioned in a response that new_relixir can trigger performance (memory & cpu) issues triggered by newrelic-erlang. His module addresses these issues.

My take: app monitoring is a magical form of monitoring: you get a lot with little work. However, making that magic happen does involve hooks into applications that can cause crashes. For that reason, I’d lean towards services that have official support and a team dedicated to keeping their monitoring agents updated.

Tracing and Profiling

Sometimes you need to walk through an application’s execution frame-by-frame to debug an issue. There are several tools that provide this for Erlang and Elixir apps.

Recon Trace

Part of the previously mentioned Recon application, Recon Trace is an Erlang module that allows for safe tracing of functions in a production environment.

Tap

Tap is a thin Elixir wrapper around Recon Trace, providing an easy Elixir interface to Recon Trace:

iex(1)> require Tapniliex(2)> Tap.call(String.strip(_, _), max: 4)2iex(4)> String.strip("test", ?t)"es"21:52:36.972255 #PID<0.88.0> String.strip("test", 116)21:52:36.972711 #PID<0.88.0> String.strip/2 --> "es"

Erlyberly

Elyberly provides a GUI interface for debugging and profiling Erlang and Elixir apps. In addition to tracing, Elyberly collections information on processes and memory usage. Not that Elyberly is not meant to trace production systems as it has no overload protection.

The folks at Plataformatec have a great getting started guide for Elyberly.

Redbug

Redbug is a production-safe debugging/tracing utility for Erlang. Roberto Aloi has a great Redbug tutorial on his blog.

DBG

DBG is an Erlang module for tracing. There’s also a thin Elixir wrapper around DBG called…DBG.

Exception Monitoring

Many of the existing exception monitoring services already support Elixir. Elixir and Erlang have a unique take on exceptions, so it’s important to see how each service’s exception logic matches your needs.

Some of the options:

• HoneyBadger
• Sentry
• Rollbar

My take: I prefer services that use the language they monitor in production as they’ll have first-hand experience on its monitoring pains. HoneyBadger has experience with Elixir in their own production apps.