Integrating Generative AI into Elixir-based applications by using the Jido agentic framework

AI Engineer

I am an AI Engineer passionate about building intelligent, resilient technology. With a focus on scalable machine learning and automation, I’m dedicated to engineering systems that last, from concept to deployment.

Hire your team!

Do you need people who can do what you're reading about and more?

Introduction

This document provides a reference case study that demonstrates our initial experience integrating agents and generative AI into Elixir-based applications by using the Jidoagent framework and its Jido AIextension. The example uses a minimal LiveView chat application. In addition, this document mentions alternative approaches, examines the current state of agent libraries in Elixir, and discusses the need for them.

Background

The following document and code examples assume that the reader has a basic understanding of:

Elixir programming language
Generative AI concepts

The concepts presented in this document are designed to be accessible to readers with general experience in Elixir development.

Architecture Overview

Example chat application diagram, with session context persistence.

Ecosystem

Integrating generative AI functionalities often relies on Python-based solutions ^[1]. However, many organizations prefer consistency in their technology stacks ^[2]. For example, companies with Elixir as their core programming language typically aim to preserve the majority of their codebases in it, introducing other languages only when the benefits outweigh the added complexity and issues ^[3].

The generative AI Python ecosystem includes mature frameworks such as PydanticAI, LangGraph, and LlamaIndex. These solutions offer extensive functionality for agent communication, orchestration, workflows, and RAGs within the Python ecosystem.Comparable frameworks in Elixir remain in their early adoption phase. However, Erlang/OTP, which serves as the foundation for Elixir, is closely aligned with many of the core design principles in mature AI agent frameworks. Specifically, its design is heavily inspired by the actor model ^[4a]^[4b]^[4c], offering a system-level foundation for implementing agent-like behaviors using lightweight processes (0.5 - 2kB per process versus 8MB default stack reserved per OS thread in standard Python setups) ^[5] and supervision natively within Erlang and Elixir applications.

Generic client-server communication model.

Actor

The Actor model is a computational model in which computations are performed by actors/agents, which are independent, isolated, stateful entities that communicate through message passing. This approach closely aligns with the design of many AI agent frameworks, emphasizing directional or bidirectional graph structures in which each node coordinates message exchange. A key difference in generative AI systems is that the computational phase often depends on external LLM inference APIs, where response latency and retries/back-offs significantly influence how long the agent waits to complete the computational phase.

The actor (Agent) orchestrates various tool calls based on user input.

Architectural Implications

Python agentic frameworks excel at planning algorithms, RAGs, and vector-store integrations. They enable quick composition of a single agent’s chain of thought, but scaling to thousands of concurrent agents means juggling asyncio, celery, or other task queues for extra work processes.

BEAM (The virtual machine Erlang and Elixir builds upon) flips those trade-offs, you can run thousands of lightweight processes with its isolated heap and mailbox, which enables to handle scenarios like one agent per user session trivial and fault tolerant (if an agent crashes, a supervisor will restart it without affecting other agents).

Jido

GitHub - agentjido/jido_ai: Jido Actions and Skills for interacting with LLMs

GitHub - agentjido/jido: 🤖 Autonomous agent framework for Elixir. Built for distributed, autonomous behavior and dynamic workflows.

Jido is one of the growing propositions for an agentic framework in the Elixir world, as the README says:

Jido provides the foundation for building autonomous agents that can plan, execute, and adapt their behavior in distributed Elixir applications. Think of it as a toolkit for creating smart, composable workflows that can evolve and respond to their environment...

Jido, among others, defines four primary primitives:

Actions - Actions represent distinct units of work. Each action contains validated input and output schemas, ensuring predictable data flow. Actions are reusable between Agents.

The following example shows how to execute individual actions using Jido.Exec.run/2

defmodule Example.Action.Add do
  use Jido.Action,
    name: "add_one",
    description: "Add one to given integer",
    schema: [integer: [type: :integer, required: true]]

  def run(%{integer: integer} = _params, _context),
    do: {:ok, %{integer: integer + 1}}
end

# Run
Jido.Exec.run(Example.Action.Add,  %{integer: 0})

‍

Signals - Signals provide a way to define, route, and dispatch events. Each signal can be tracked, replayed, and verified against the schema.

The following example shows how to publish and subscribe to events.

# Simplified version of: <https://github.com/agentjido/jido_signal?tab=readme-ov-file#quick-start>
# Usually the signal bus should be started in application.ex
Jido.Signal.Bus.start_link(name: :example)

defmodule Example.Subscriber do
  use GenServer

  def start_link(_opts \\ []), do: GenServer.start_link(__MODULE__, %{})
  def init(state), do: {:ok, state}

  def handle_info({:signal, %{type: "user.created", data: %{user_id: user_id}} = _signal}, state) do
    IO.puts("Hello #{user_id}")
    {:noreply, state}
  end

  def handle_info({:signal, _signal}, state), do: {:noreply, state}
end

# Start subscriber and subscribe to user events
{:ok, sub_pid} = Example.Subscriber.start_link()
{:ok, _sub_id} = Jido.Signal.Bus.subscribe(:example, "user.*", dispatch: {:pid, target: sub_pid})

# Create signal
{:ok, signal} = Jido.Signal.new(%{
  type: "user.created",
  data: %{user_id: "0"}
})

# Publish event
Jido.Signal.Bus.publish(:example, [signal])

‍

‍Agents - Agents encapsulate schema-validated state and coordinate workflows and actions.

The following example shows how to integrate an agent with a web search tool.

defmodule Example.Agents.Tools.WebSearch do
  # Jido actions are converted to Agent AI tools
  use Jido.Action,
    name: "web_search",
    description: "Search the web for information",
    category: "Web",
    tags: ["web", "search"],
    vsn: "1.0.0",
    compensation: [
      timeout: 30_000,
      max_retries: 3,
      enabled: true
    ],
    schema: [
      query: [type: :string, doc: "The query to search the web for", default: ""]
    ]

  @search_impl Application.compile_env(:example, :search_impl, Example.Tavily)

  # All tools need to implement run function.
  def run(%{query: query} = params, _context), do: @search_impl.search(query, params)
end

defmodule Example.Agents.WebQA do
  use Jido.Agent, name: "example_agent"

  @system_prompt "Answer user question aided by web search tool"
  @user_prompt "<messages><%= @message %></messages>"
  @agent_prompt Jido.AI.Prompt.new(%{
                  messages: [
                    %{role: :system, content: @system_prompt, engine: :eex},
                    %{role: :user, content: @user_prompt, engine: :eex}
                  ]
                })
  @agent_tools [Example.Agents.Tools.WebSearch]
  @agent_model {:openai, model: "gpt-4o"}
  @agent_verbose false

  # Agents are built upon GenServer
  def start_link(_opts \\ %{}) do
    Jido.AI.Agent.start_link(
      agent: __MODULE__,
      ai: [
        model: @agent_model,
        verbose: @agent_verbose,
        prompt: @agent_prompt,
        tools: @agent_tools
      ]
    )
  end

  # We just forward the request
  defdelegate tool_response(pid, message, kwargs \\ []), to: Jido.AI.Agent
end

‍

‍Workflows - Workflows are patterns that define how agents and actions are composed and executed. They coordinate state across multiple runs and ensure the atomicity of operations.

The following example shows how to compose multiple actions into a workflow and pass it to an Agent.

# Example actions that will receive Integers and perform computation on them.
defmodule Example.Action.Add do
  use Jido.Action,
    name: "add_one",
    description: "Add one to given integer.",
    schema: [integer: [type: :integer, required: true]]

  def run(%{integer: integer} = _params, _context), do: {:ok, %{integer: integer + 1}}
end

defmodule Example.Action.Multiply do
  use Jido.Action,
    name: "multiply",
    description: "Multiply given integer by four.",
    schema: [integer: [type: :integer, required: true]]

  def run(%{integer: integer} = _params, _context), do: {:ok, %{integer: integer * 4}}
end

defmodule Example.Action.Divide do
  use Jido.Action,
    name: "divide",
    description: "Divide givien integer by two.",
    schema: [integer: [type: :integer, required: true]]

  def run(%{integer: integer} = _params, _context), do: {:ok, %{integer: integer / 2}}
end

# Assembly of workflow (multiple steps of Actions)
defmodule Example.Workflow.Math do
  use Jido.Actions.Workflow,
    name: "math_workflow",
    description: "Simple calculator workflow",
    schema: [integer: [type: :integer, required: true]],
    workflow: [
      {:step, [name: "step_1"],
        [%Jido.Instruction{action: Example.Action.Add, params: %{level: :info, message: "Step 1"}}]},
      {:step, [name: "step_2"],
        [%Jido.Instruction{action: Example.Action.Multiply, params: %{level: :info, message: "Step 2"}}]},
      {:step, [name: "step_3"],
        [%Jido.Instruction{action: Example.Action.Divide, params: %{level: :info, message: "Step 3"}}]},
      {:step, [name: "step_4"],
        [%Jido.Instruction{action: Example.Action.Multiply, params: %{level: :info, message: "Step 4"}}]}
    ]
end

# Invocation in Agent
defmodule Example.Agents.Calculator do
  use Jido.Agent,
    name: "example_agent",
    actions: [Example.Workflow.Math],
    schema: [integer: [type: :integer, required: true]]

  def calculate(pid, params \\ %{}),
    do: call(pid, %Jido.Instruction{action: Example.Workflow.Math, params: params})
end

# Start agent
{:ok, pid} = Example.Agents.Calculator.start_link()
# Fire up calculate function
Example.Agents.Calculator.calculate(pid, %{integer: 10})
# Result
{:ok, %{integer: 88.0}}

‍

Reference Implementation

Our implementation focused on a minimal LiveView chat application aided by Jido AI, in which the user interacts with an LLM on technical topics. All conversations are monitored with a topic drift agent that checks for unwanted attempts to use the LLM for actions it was not designed for. Each user conversation is an isolated process that maintains conversation history only for this particular session using Erlang term storage (ETS). The Erlang telemetry module provided the telemetry.

The complete source code is available on GitHub:

Project - Example Implementation based on LiveView, Jido
Livebook - Livebook Implementation based on Kino, Jido, and Rustler

Observations and lessons learned

One of the key strengths of Elixir agent libraries like Jido is that they are built upon the battle-tested OTP framework with nearly 20 years of active contribution. The ability to rapidly prototype agents thanks to ready-to-use abstractions like Jido.AI.Agent and Jido.AI.Skill is another advantage. A notable highlight was the responsiveness of the core maintainer, issues are addressed quite fast, creating a positive impact for future contributions and users of Jido / Jido AI. Additionally, what was quite refreshing when moving from the Python world, and generally is more about the Elixir / Erlang ecosystem, we could hot-swap agent code at runtime via OTP’s release handling system - this greatly helped with the development and testing of how different system prompts impact the AI agents without any downtime.

However, several challenges exist. The documentation is limited and contains few complete examples. In some cases, even the md files reference components that do not exist in the library ^[6a], which can be confusing. Some of the previously mentioned abstractions are tightly coupled/fixed, meaning certain use cases - like passing state into tool calls require diving into source code and making modifications ^[7]. This increases friction and extends the time needed to adapt the framework to the given problem. The following problem is not specific to Jido but reflects the general state of generative AI tooling in Elixir; for compute-intensive tasks such as document processing, implementations often rely on Rust ^[8]. Elixir excels at concurrency and fault tolerance, not numerical workloads, so this division could be pragmatic, though it does introduce cross-language complexity.

Summary of Findings

The foundational capabilities required for building agent-based systems already exist within the Erlang/OTP platform. Developers can leverage Elixir's native concurrency model, supervision trees, and message-passing architecture to implement AI agent-like behaviors without requiring a dedicated agentic AI framework.

The emerging frameworks such as Jido with Jido AI or SwarmEx provide functionality for agent implementation and orchestration and they are worth keeping an eye on for future updates as they mature; many features can be selectively implemented based on project-specific requirements by combining libraries such as OTP (gen_server, gen_event, gen_statem, ...), LangChain(Elixir), and Req / HTTPoison, among others, for HTTP client integration. This approach allows teams to retain control over system design, favouring simplicity, transparency, and maintainability.

Ultimately, effective agent-based systems rely on software architecture, appropriate data modeling, and system design, all of which scale effectively.

Related Work

SwarmEx - Lightweight agent orchestration example in Elixir. While documentation behind livebook examples is limited, the code is clean, with minimal buildup which allows you to adapt its concepts to your own use case.
Langchain - Elixir take on the the Python LangChain presented in a functional way, providing various abstractions over various LLM providers, tool calling. It also includes support for chat models hosted with Bumblebee.
RAG - An example of performing retrieval augmented search (RAG) in pure Elixir leveraging the Nx library. It also supports integration with external providers.
Rustler - Safe integration of Rust for creating Erlang NIF functions

References

Other: