Architecture

Fermix ships as a single self-contained binary: install it, run it, and everything it needs runs inside that one program. Under the hood it is an Elixir umbrella application (several closely related apps built and shipped as one). Everything Fermix itself does runs in one OS process under OTP supervision (OTP is Elixir’s framework for running and automatically restarting long-lived processes). There are no HTTP bridges between apps, no separate worker pool, and no message broker. Every long-lived piece of state is a named GenServer (a small supervised process) or a registered ETS table (an in-memory key-value table built into the runtime). The only components outside that one process are external helpers the daemon spawns and supervises as subprocesses: signal-cli for the Signal channel, local MCP server processes, the managed browser behind the browser tool, and — when the experimental, off-by-default computer-use feature is enabled — the native computer-use helper.

Umbrella apps

App	Role
fermix_core	Runtime core: agents, providers, tools, memory, prompt composition, sandbox, scheduled jobs, MCP, realtime voice, traces, setup, readiness
fermix_channels	Messaging-platform adapters (Telegram, WhatsApp, Slack, Discord, Signal, CLI), the gateway, ingress authorization (deciding who is allowed to talk to the agent), idempotency (making sure a retried message is not processed twice)
fermix_web	Phoenix: webhooks, health endpoints, home dashboard + setup LiveView, static assets
fermix_nif	Native C NIF: a POSIX process-group kill shim (kill_pgid) used by the subprocess-lifecycle sweep to reap every descendant of an external command. Built via elixir_make; no Rust toolchain
fermix_opik	Opt-in Opik telemetry exporter (Opik is an external tool for inspecting agent runs). Inert unless FERMIX_OPIK_ENABLED is set; never sits on the reply path

fermix_channels depends on fermix_core. fermix_web depends on both. fermix_core does not depend on Phoenix. All inter-app calls are BEAM messages (in-process messages between Elixir processes; BEAM is the Erlang/Elixir virtual machine the binary runs on) or direct function calls, never localhost HTTP.

Runtime topology

+----------------------------------------------------------------+
| BEAM VM  (one Burrito-packaged binary, "fermix")               |
|                                                                |
|  fermix_core     -- agents, providers, tools, memory, sandbox, |
|                     MCP, realtime, jobs, prompt, traces        |
|  fermix_channels -- Telegram, WhatsApp, Slack, Discord,        |
|                     Signal, CLI, gateway, ingress              |
|  fermix_web      -- Phoenix: webhooks, health, LiveView UI     |
|  fermix_nif      -- C NIF: process-group kill (command sweep)  |
|  fermix_opik     -- Opik telemetry export (opt-in)             |
|                                                                |
|  All apps :permanent under one umbrella supervisor.            |
|  All inter-app calls are BEAM messages -- no localhost HTTP.   |
+----------------------------------------------------------------+
         ^                            |
         | Webhooks (HTTPS)           | Outbound HTTP (Req)
         | Long-poll (Telegram)       |   --> LLM providers
         | WS (Discord)               |   --> channel APIs
         | Unix socket (CLI)          |   --> MCP servers
         | Subprocess (signal-cli)    |   --> verified plugin +
         v                            v       helper downloads
  Channel providers               Internet

See supervision for the full OTP supervision tree.

Message-to-reply flow

This is the path a message takes from arrival to reply:

1. Channel adapter receives input
   - Webhook:    Phoenix endpoint -> WebhookController
                 (idempotency check here: a retried webhook delivery of an
                  already-seen message id is acknowledged but never re-run)
   - Long-poll:  Telegram.Poller
   - Gateway:    Discord.Gateway
   - Subprocess: Signal.Listener
   - CLI:        ChatCommand -> daemon Unix socket   (daemon = the long-running Fermix background process)
   - Voice:      Realtime.LocalVoiceSocket -> SessionServer

2. Adapter normalizes payload -> %FermixChannels.Message{}

3. FermixChannels.Gateway.ingest/2:
   - Authorize sender via Gateway.Authorizer
       :operator (full surface) | :guest (read-only) | denied (dropped, no reply)
   - Audio transcription (audio attachments become text)
   - Inbound image download (media-capable channels; sibling to audio transcription)
   - Build reply_fn closure
   - Empty check: no text and no media -> a brief "looks empty" reply,
     and no turn is scheduled
   - Parse slash commands -> dispatch, or :passthrough -> deliver_to_agent

4. Gateway.Queue: one FIFO turn per conversation key
   (FIFO = first-in, first-out: messages in a conversation are handled
    in arrival order, one at a time)
   {channel, chat_id, thread_scope}

5. Queue checks out a turn-state snapshot from MainAgent (which owns
   the cached runtime context) and runs the turn in a supervised
   TurnRunner task:
   - ConversationStore: recent history
   - Preflight auto-compaction if the previous turn crossed the threshold
   - PromptComposer: cached base prompt + runtime section + fresh date note
   - Capabilities.Registry: tool schemas filtered by trust + policy
   - Persist the user message, then run AgentLoop

6. AgentLoop (bounded LLM/tool loop):
   - adapter.chat/3 -> provider -> tool_calls
   - For each tool_call:
       capability_allowed? -> Capability.execute(args, context)
   - adapter.continue/3 with tool results
   - Repeat until tool_calls = [] or iteration cap
     (default 100; a system-side knob, not a config.toml section)

7. Gateway delivers the reply, then commits the turn (TurnRunner.commit):
   - deliver_reply via reply_fn
   - ConversationStore.add_message (assistant reply; the user message
     was already persisted at turn start)
   - Memory.Reviewer.start_background (time-gated) -> Memory.Admission
     -> SQLite + Memory.PromptFiles.rebuild
   - Post-delivery auto-compaction check (CompactionConfig threshold)

8. Channel adapter sends outbound message via channel API

See the agent loop and channels for deeper coverage of steps 6 and 3 respectively.

Trust boundaries

Every message is resolved to a trust level before the agent runs. Trust determines which capabilities are reachable.

Source	Resolved trust	Capability surface
Local CLI, daemon socket	:operator	Full: read-only, read-write, exec, network, external_api, gui_control
Realtime voice (local)	:operator	Same as main agent
Channel owner_user_id	:operator	Full surface
Channel allowlist entry (not owner)	:guest	Read-only only
Channel sender, no allowlist match (allowlist = the list of users explicitly permitted)	denied	Dispatcher logs and returns :ok; agent is never invoked
Scheduled job (operator-created)	:operator	Full surface
Scheduled job (guest-created)	:guest	Read-only plus network-class tools; any tool allowlist is clamped to what the creator could call

A missing trust: on a registry call resolves to :guest (least privilege). There is no “allow all” escape hatch: an unknown sender is always denied. Adding a user to a channel allowlist (allowed_user_ids or allowed_sender_ids, depending on the channel) grants :guest trust; only owner_user_id grants :operator.

The capability filter runs inside FermixCore.Capabilities.Registry. Trust is the load-bearing filter: it controls whether tools such as shell, file_write, and web_fetch are reachable for a given message.

The sixth capability class, gui_control, belongs to the computer_use tool (desktop control — experimental and off by default). It is operator-only and extra-gated: the tool registers only when computer use is enabled and its native helper is installed, it is never delegated to sub-agents, and scheduled runs cannot start desktop sessions — only an attended origin (an interactive chat, fermix ask, or a voice session) can.

See ingress and trust and capabilities and tools for full details.

Persistence layout

All runtime state is written under FERMIX_HOME (default ~/.fermix).

Artifact	Path	Owner
Operator config	~/.fermix/config.toml	Setup wizard + operator edits
Provider tokens	~/.fermix/auth.json (mode 0600)	Auth.Store
SQLite store	~/.fermix/memory.db	Memory.Repo
Prompt memory files	~/.fermix/memory/<agent_id>/	Memory.PromptFiles
Prompt bootstrap files	~/.fermix/bootstrap/<agent_id>/	Seeded by setup, read by BootstrapLoader
Sandbox workspace (default write floor)	~/.fermix/workspace/	Sandbox
Skills (user-installed)	~/.fermix/skills/	SkillRegistry (bundled skills ship inside the binary; plugin skills live under the plugin store)
Installed plugins	~/.fermix/plugins/ (also caches the native computer-use helper’s binary here)	Plugins.Registry
Sandbox grant records	~/.fermix/grants/	Sandbox.ConfigMutation
Logs (rotating)	~/.fermix/logs/fermix.log (10 MB x 5)	:logger_std_h
Traces	~/.fermix/traces/YYYY-MM-DD/<type>.jsonl	FermixCore.Trace
Scheduled-job artifacts	~/.fermix/job_runs/<run_id>/	Jobs.Runner
Daemon CLI socket	~/.fermix/daemon.sock (mode 0600)	Fermix.CLI.Daemon
Realtime socket	~/.fermix/realtime.sock (mode 0600)	Realtime.LocalVoiceSocket

SQLite is canonical for durable memory. Prompt memory files are derived artifacts, not the source of truth. ConversationStore keeps each conversation’s history in memory and writes every message through to SQLite via a supervised background task with bounded retries (up to three attempts), so a restart recovers recent conversation state from SQLite.

Design choices

One BEAM VM, no HTTP between apps. FermixWeb calls FermixChannels.Gateway.ingest/2 directly. The gateway dispatcher casts each inbound message to Gateway.Queue; the queue then makes a synchronous GenServer.call to MainAgent to check out a turn-state snapshot, and runs the agent loop in a supervised TurnRunner task. This eliminates an HTTP hop and reduces service-availability checks to a single Process.alive?/1 call.

FIFO turns per conversation. Gateway.Queue keeps at most one active turn per {channel, chat_id, thread_scope} key, plus a FIFO pending queue. If a new same-conversation message arrives while a turn is running, it waits behind the active turn and runs in arrival order after that turn completes — it does not cancel the in-flight turn. (Only /stop cancels an active turn.) Different conversations run independently and concurrently.

SQLite for durable memory, in-memory state for the hot path. SQLite handles atomic durable writes and full-text search. The hot path never waits on disk: conversation history is served from the ConversationStore process’s own memory, and remembered facts are cached in an ETS table (Memory.Store). The write-through design means a daemon restart recovers the most recent state from SQLite.

One capability registry, three kinds. FermixCore.Capabilities.Registry is the only place agents read capabilities from. It holds built-in tools, skills, and MCP tools (tools provided by external MCP servers; MCP is the Model Context Protocol, a standard way for programs to expose tools to an AI model) under a single filter surface (trust, policy, allowed-tools list, excluded categories, excluded names). Trust is the load-bearing filter. When tool-schema deferral is on (default), plugin and MCP tool schemas are withheld from the provider wire (not sent to the model up front): only names are listed, and three bridge tools (tool_search, tool_describe, tool_call) let the model resolve and invoke them on demand. Deferral is controlled by [fermix_core.tools.tool_search] enabled; set enabled = false to inline all schemas.

Channel ingress fails closed. An enabled channel without an explicit owner or allowlist rejects every sender it does not recognize. There is no “allow all” default.

Native helpers are subprocesses, not in-process libraries. Anything that could crash or hang — signal-cli, local MCP server processes, the managed browser, the native computer-use helper (experimental, off by default) — runs as a separate OS process the daemon spawns and supervises, so a helper failure can never take down the agent runtime. The computer-use helper is additionally version-checked at session start: the daemon performs a handshake and refuses to run a helper whose wire protocol does not match the running build (for example after a partial upgrade), failing loudly instead of misbehaving. Every external command the daemon runs (the shell and git tools, sandboxed command profiles, signal-cli, credential helpers, plugin runtime probes) gets the same treatment: each runs under a supervised CommandHost that opens the port itself and owns the command’s OS process group, so the group is never unowned. Every ending sweeps the whole group — a normal exit, the requester dying, or the daemon shutting down kills the group outright, while a timeout or an output-cap truncation sends SIGTERM, drains for a grace period, then unconditionally SIGKILLs — so a hung command cannot orphan grandchild processes. The one piece of in-process native code is a small C shim that delivers those group-kill signals as a direct syscall: deliberate, because the sweep has to work even when the process table is exhausted and spawning a helper would itself fail.

Observability is opt-in and side-car. (Side-car: it runs alongside the main flow and only watches, never handling the reply itself.) fermix_opik attaches to shared telemetry events but never sits on the reply path. Disabling it adds zero overhead to a turn. Trace writes (FermixCore.Trace) are best-effort: a failed write is logged and does not affect the reply.