Automate the Boring Stuff with Elixir
TL;DR: Connect n8n & Livebook to build Elixir automations & app prototypes entirely in the browser.
The more I learn Elixir, the more I want to use it everywhere for everything. Python & JavaScript have a plethora of platforms to write & deploy code from your browser like Val.Town or Replit, and while Elixir has Phoenix.new for web apps, I wanted something simpler for backend-y projects.
Enter: n8n + Livebook + Webhooks.
With this setup, I can focus on passing & transforming JSON payloads, making it easy to automate hundreds of services using Elixir, Erlang, Python, & JavaScript. To set expectations accordingly though, there are downsides which will be covered in depth.
Let's begin.
Table of Contents
n8n Explained
Livebook Explained
n8n & Livebook Resources
Livebook Apps Explained
Bonus: Elixir in WebAssembly??
n8n Explained
n8n is a low/no-code automation platform for combining over 400 integrations in a visual node-based UI, saving you from writing boilerplate code so you can focus on the core app functionality. Check the n8n features page for a detailed explanation. People build some seriously wild workflows with it, especially for AI Agent Armies to run your whole business. They're all the rage on YouTube these days.
Source: Reddit
That's a pretty wild example but hopefully you get the gist of what's possible.
Livebook Explained
Livebook is to Elixir as Jupyter Notebooks are to Python. They make it easy for beginners to get started with Elixir, allow data scientists to share analysis & experiments, and even allow you to connect to a remote Elixir node to run scripts & utility projects. Checkout livebook.dev to learn more.
HTTP APIs in Livebook
For this particular project, I followed this guide from the Livebook Blog about exposing an HTTP API from within a notebook.
The authors walk through an example they created that tracks the number of days they've successfully avoided breaking GitHub builds. It's worth a read to understand the nuances of this setup, although it leaves out critical details, like how to access the exact notebook URLs, which I cover in depth below.
n8n & Livebook Resources
I used the following guides to get both services online, which took a few hours of setup time. This is the first hurdle to cross; actually building the infrastructure.
Fly.io recently announced docker-compose compatibility, which means you can combine both of these on a single node and use the internal docker network to eliminate latency!
Livebook Apps Explained
If you follow the Livebook HTTP API blog post, you'll need to add /proxy/ to your notebook URL to make it accessible. The downside of this is that the Session ID changes frequently, making it hard to persist the connection.
Assuming you expose a /health endpoint from your notebook, you'd hit it via:
LIVEBOOK_SERVER/proxy/sessions/SESSION_ID/health
Livebook apps allow you to "soft deploy" a notebook to a more persistent URL, which in our example, changes the webhook URL to:
LIVEBOOK_SERVER/proxy/apps/APP_NAME/health
where APP_NAME is the name of the notebook.
By setting the LIVEBOOK_APPS_PATH = "/data/apps" env var in your fly.toml, notebooks saved in /data/apps will be started automatically with Livebook. Check out the many other env vars to fully customize it to your needs.
Livebook App Lifecycle
If you make edits to the notebook saved in /data/apps, Livebook won't automatically apply them since the app is running in another process. There may be a more elegant way to handle this, but to my knowledge, restarting the entire Livebook service is the only way to update the app. Keep in mind, it takes a few seconds (10-30) for the app to come online after a restart, so don't use this for any production/customer-facing workflows.
Ideally, we could integrate an Elixir runtime into n8n like the Python & JavaScript nodes....
Bonus: Elixir in n8n via WebAssembly??
As of the last few weeks, a major announcement has the Elixir ecosystem buzzing. You can now run Elixir in the browser using Popcorn!
"Popcorn bridges your Elixir code with Wasm-compiled AtomVM runtime. It runs plain Elixir as-is (including OTP patterns and supervision trees) right inside the browser, which makes it ideal for many use cases, like offline, local-first applications. No servers or internet connection required."
Give this episode of the Thinking Elixir podcast a listen if you'd like to learn more. I wonder how much work it'll take to add Elixir as a first-class language node in n8n. That would make this setup FAR more streamlined, and hopefully broaden Elixir adoption.
It's officially on my "projects for the recurse center" ideas list :)