Docs/Concepts/Supervision tree

Supervision tree

How Fermix structures its OTP supervision tree across the core, channels, web, and realtime applications.

This page explains how Fermix keeps its internal moving parts alive and restarts them cleanly when something fails. Fermix runs as a single program built on the BEAM (the Erlang/Elixir virtual machine) using OTP, Elixir’s framework for running and supervising long-lived processes. A supervisor starts its children and applies their restart policies. The application has one root supervisor per sub-app; processes that depend on each other appear in ordered child lists so that failures and restarts cascade correctly. Some temporary sessions, including live transcription, are owned and monitored by their consumers rather than restarted by a supervisor.

For a broader view of how these processes fit together, see architecture and the agent loop.

Core supervisor

Module: FermixCore.Application starts FermixCore.Supervisor. Strategy: :rest_for_one

With :rest_for_one, if a child crashes, every child that was started after it is also stopped and restarted in order. This matters here because most processes in the list depend on earlier ones: the tool registries depend on the ETS tables (fast in-memory tables built into the runtime) owned by Capabilities.Registry, the agent depends on all of the above, and so on. Silently keeping a stale registry alive after a crash would cause subtler failures than a clean restart.

The trade-off: a MainAgent crash also brings down the jobs runner and anything started after it. Any in-flight scheduled work is lost. For a single-user local daemon this is the correct choice over silently broken cross-references.

The numbers below show the complete possible start order. Conditional children are omitted when their gate is closed; the remaining children keep their relative order.

# Child Purpose
1 FermixCore.CommandHost.Supervisor Dynamic supervisor for per-command CommandHost owners. Every external OS command the daemon runs gets a supervised host that owns the command’s process group and sweeps every descendant when the command ends: a normal exit, requester death, or daemon shutdown sweeps with an immediate group SIGKILL; a wall-clock timeout or an output-cap breach sends a graceful SIGTERM first, drains briefly, then unconditionally SIGKILLs — so a hung skill or shell command can never leak orphaned subprocesses. Children are :temporary (a crashed command is a single-command fault, never restarted); it sits first in the tree so every process that can run a command starts after it exists.
2 Task.Supervisor (FermixCore.TaskSupervisor) Hosts every short-lived background task: per-message agent loops, deferred deliveries, extraction passes.
3 Finch (FermixCore.Finch) Shared outbound HTTP connection pool for provider and channel API calls. Configured here with two pool processes per host and a 15-second idle cap on kept-alive connections, so a request right after the host wakes from sleep is served fresh instead of queuing behind a stale socket being torn down.
4 FermixCore.Trace JSONL trace writer; keeps one open file per {date, type} pair.
5 FermixCore.Auth.TokenSupervisor Always started. Supervises per-OAuth-profile token managers via an internal dynamic supervisor and registry.
6 FermixCore.Auth.TokenManager (conditional) Started when Codex is routable (either as the configured primary or as any fallback in the provider chain); refreshes Codex OAuth tokens.
7 FermixCore.Browser.Supervisor Supervises the browser session registry, dynamic supervisor, and profile manager for the browser tool.
8 FermixCore.Capabilities.Registry Owns the ETS table of all registered capabilities.
9 FermixCore.Capabilities.BuiltinSeeder Restart: :transient. Registers every built-in tool synchronously at boot, then returns :ignore.
10 FermixCore.Sandbox.CommandCapabilities Registers sandbox preset commands as built-in capabilities; depends on the registry being alive.
11 FermixCore.Plugins.Dist.Installer GenServer (a single supervised process that handles one thing at a time) that sweeps transient-staging plugin artifacts at boot and serializes plugin install/uninstall/gc mutations.
12 FermixCore.Plugins.CapabilitySeeder Restart: :temporary. Registers capabilities from any enabled plugins into the capability registry.
13 FermixCore.Agents.SkillRegistry Discovers core, local, and plugin skills and mirrors them into the capability registry.
14 FermixCore.Capabilities.MCP.RuntimeStatus In-memory status table for every outbound MCP client: connecting, ready, or the classified reason it stopped. It sits outside the MCP subtree on purpose — a remote server’s subtree is :temporary, so a status table living inside it would vanish exactly when it holds the only explanation of why the client is gone. Nothing here is persisted; a daemon restart starts over at connecting.
15 FermixCore.Capabilities.MCP.Supervisor Starts every configured outbound MCP server (an external tool server speaking the Model Context Protocol), discovers its tools, and registers them as :mcp capabilities.
16 FermixCore.Memory.Repo SQLite-backed durable memory store.
17 FermixCore.Memory.ConversationStore ETS hot path for conversation history with SQLite write-through.
18 FermixCore.Memory.Store Key-value memory store backed by ETS and SQLite.
19 FermixCore.Setup.SecretWriteLog Records successful secret writes since boot so credential rotation can require a restart even when the persisted @keyring marker is unchanged.
20 FermixCore.Setup.SecretAclState Keeps Doctor’s most recent keychain-permission measurement so status reads do not prompt for keychain access.
21 FermixCore.Setup.BootReport Aggregates the “daemon is ready” signal consumed by the UI and CLI.
22 FermixCore.Setup.RestartState Tracks the boot configuration and persisted-file baseline. Publishes restart requirements and refuses saves against an externally changed config until reload.
23 FermixCore.Agents.AgentSupervisor Dynamic supervisor for subagent processes (short-lived helper agents the main agent spawns to do a piece of work): the workers behind the subagents tool and delegated skill runs (skill_run).
24 FermixCore.Agents.MainAgent Restart: :permanent. The single persistent main agent.
25 FermixCore.Jobs.RunnerSupervisor Dynamic supervisor for scheduled-job runners.
26 FermixCore.Meetings.Supervisor Always present. Owns meeting-session registration, temporary session workers, and boot reconciliation. Enabling the feature does not itself open a browser or start recording.
27 FermixCore.Jobs.Scheduler Claims due jobs and spawns runners. Wakes precisely when the next job is due, with a 60-second reconciliation sweep as a safety net.
28 FermixCore.Temporal.Scheduler The reminders clock: the single owner of reminder claims. It sweeps rows left mid-delivery by a previous run, arms one timer for the nearest due reminder, claims due rows, and starts a delivery worker for each. It performs no network work itself.
29 FermixCore.Temporal.DeliverySupervisor Bounded dynamic supervisor for reminder delivery workers, each :temporary. It starts after the scheduler, and that order is load-bearing: under :rest_for_one a scheduler crash tears down this supervisor and every in-flight worker before the scheduler restarts, so no delivery worker can outlive the scheduler that claimed its row.
30 FermixCore.Temporal.FollowupSupervisor Bounded dynamic supervisor for the follow-up check-in runs that trail delivered reminders, each :temporary, at most two at a time — a run refused at the cap is skipped and traced, never queued. It sits after the delivery supervisor in the same :rest_for_one chain, so a temporal-scheduler crash tears down in-flight check-ins along with deliveries; a check-in is best-effort by design and dies with the rail.
31 FermixCore.Harness.Supervisor The coding agents rail: an internal :rest_for_one chain of Harness.Manager (admission, terminalization, and reconciliation authority), Harness.RunSupervisor (dynamic supervisor for :temporary per-run workers), and Harness.DeliveryWorker (durable delivery-outbox drain), so a Manager crash re-reconciles live runs instead of orphaning them. Always started, even when the feature is disabled — boot reconciliation (which finalizes stale rows as interrupted) and the delivery drain must finish in-flight work regardless; only its timers are feature-gated.
32 FermixCore.Management.Lifecycle (conditional) Owns the bounded lifecycle lease used for restart, shutdown, and app-managed service changes.
33 FermixCore.Management.Doctor (conditional) Runs and retains cancellable Doctor sessions with deadlines.
34 FermixCore.Management.Jobs (conditional) Owns cancellable setup operations such as sign-ins, installs, permission grants, and workspace discovery.
35 FermixCore.Management.Plugins.Discovery (conditional) Retains the latest plugin workspace-discovery results for settings clients; cleared on restart.
36 Fermix.CLI.Daemon (conditional) Unix-socket chat, control, and management RPC. Started when :daemon_socket_enabled is set, including standalone daemon runs and the app-managed engine.
37 FermixCore.Realtime.Supervisor (conditional) Started when realtime voice is enabled in config with the openai provider and an OpenAI API key is present (on the daemon path where the realtime socket is enabled).
38 FermixCore.ComputerUse.Supervisor (conditional) Started when computer use is enabled and the native computer-use helper (a separate program that drives the host desktop) is installed. OS permission state is deliberately not part of this gate — a missing Screen Recording or Accessibility grant is surfaced as a diagnostic by fermix doctor instead of hiding the feature. Supervises per-conversation session processes. Experimental and off by default.
39 FermixCore.ComputerHistory.Supervisor (macOS only) Present even when history is disabled, so retention continues. Its controller starts capture and summarization workers only when history is enabled and the shared helper is installed.
40 FermixCore.SkillCuration.Scheduler (conditional) Started when skill curation is enabled and memory persistence is on. A small clock that ticks every six hours and starts a curation pass when the cadence is due; the pass itself runs as a monitored task, so this process never blocks on it. See skills.

Children 32 through 36 share the daemon-socket gate. The management owners start before the socket accepts requests, so every exposed operation has its state owner available. The same implementation serves the macOS app and browser setup; settings and secret writes remain daemon-owned.

MainAgent and AgentSupervisor

MainAgent (child 24) is the single permanent top-level agent. It owns runtime-context cache state and checks out turn-state snapshots for Gateway.Queue, which enforces one FIFO turn per conversation (first-in, first-out: one message at a time, in arrival order).

AgentSupervisor (child 23) is a dynamic supervisor that starts AgentServer workers on demand. Each AgentServer runs one delegated task with its own agent definition, session ID, provider, registry, and parent metadata. These are the worker processes behind the subagents tool and delegated skill runs (skill_run).

Skill and capability registration order

Children 8 through 15 form a strict dependency chain: the registry must be alive before the seeder populates it, the seeders must complete before the skill registry and MCP supervisor add their own entries, and the MCP status table must exist before the MCP supervisor that writes to it. The :rest_for_one strategy ensures any failure in this range restarts the whole chain in order.

Meetings and transcription

Meetings.Supervisor uses :one_for_one and starts these children in order:

  1. Meetings.Registry: maps each meeting ID to its live session.
  2. Meetings.SessionSupervisor: dynamic supervisor for :temporary meeting sessions.
  3. Meetings.Sweep: a :transient boot worker that marks stranded active rows as failed with daemon_restarted, then exits normally.

This tree is always present, including when meetings are disabled. It opens no browser or recording stream by itself. An authorized join_meeting starts one session, which owns the Google Meet sidecar or Zoom RTMS source, transcription, and the summary task. Sessions are not automatically restarted into meetings after a failure. See meeting notes.

Transcription.StreamSession defines the common API; it is not a process or a supervisor. Transcription.open_stream/2 starts the selected backend’s session with the calling process as its consumer. Each session monitors that consumer and releases its resources when the consumer dies or the stream ends. Deepgram and SpaceXAI own vendor WebSocket connections; the local backend owns one fermix-stt process per stream; OpenAI uses the chunked adapter. A local helper crash is reported to the consumer and is not automatically respawned.

Computer History

On macOS, ComputerHistory.Supervisor uses :rest_for_one with this order:

  1. ComputerHistory.DynamicSupervisor: owns the runtime capture and summary workers.
  2. ComputerHistory.Controller: reconciles whether those workers should be running.
  3. ComputerHistory.Retention: sweeps old raw events and caps spool and audit size, including while history is disabled.

The controller starts ComputerHistory.Capturer and ComputerHistory.Summarizer.Scheduler when history is enabled and the shared native helper is installed. A dynamic-supervisor restart restarts the controller, which recreates the required workers. The capturer owns the Accessibility observation process; the scheduler starts bounded summarization cycles about every 30 minutes. Pausing capture is enforced at ingest, while disabling tears down the runtime workers. The retention process remains.

The tree is absent outside macOS. Presence of its workers does not establish a compatible helper or a working provider; see Computer History for the current helper limitation, provider permissions, and diagnostic commands.

Channels supervisor

Module: FermixChannels.Application Strategy: :one_for_one (each child is restarted on its own if it crashes, without disturbing the others)

# Child Started when
1 FermixChannels.Gateway.Queue Always. FIFO turn queue: enforces one active turn per conversation key.
2 FermixChannels.Gateway.BackgroundSupervisor Always. Dynamic supervisor for background (non-blocking) channel tasks.
3 FermixChannels.Gateway.Commands.Sandbox.Confirmations Always. Tracks pending sandbox grant confirmations from channel slash commands.
4 FermixChannels.Gateway.Commands.Soul.Confirmations Always. Tracks pending soul-command confirmations.
5 FermixChannels.Gateway.Idempotency Always. Idempotency cache for webhooks and outbound media: remembers what was already handled so a retried delivery is not processed twice.
6 FermixChannels.Gateway.AlbumBuffer (Telegram) Always. Coalesces multi-image Telegram media groups arriving across poll cycles into a single turn.
7 FermixChannels.Gateway.AlbumBuffer (WhatsApp) Always. Coalesces per-image WhatsApp webhook deliveries with no media_group_id into a single turn.
8+ Transport children (from ChannelRegistry) Per-channel: Telegram poller, Discord gateway, Signal listener, and the ACP listener (a socket accept loop plus one supervised peer process per connected client). Started only when the respective channel is enabled with the matching transport mode. The ACP listener is authorized by the local socket it binds rather than by a sender id, so the ingress check described below does not apply to it.

Both album buffers are always started; the channel’s configuration only tunes their debounce window, and a buffer for an unused channel simply sits idle.

WhatsApp and Slack adapters receive inbound messages through Phoenix webhook endpoints rather than long-running supervised processes; they do not appear in the transport children list.

Before starting its children, the channels application fails fast on a misconfigured slash-command registry (a duplicate command name or alias would silently shadow a command, so boot aborts instead). It also refuses to start any remote channel adapter that is enabled but has no owner_user_id or allowed_*_ids configured: the refusal is logged as an error, and the channel stays offline rather than accepting messages from unauthorized senders. See ingress and trust for how sender authorization works.

Web supervisor

Module: FermixWeb.Application Strategy: :one_for_one

  1. FermixWebWeb.Telemetry
  2. DNSCluster
  3. Phoenix.PubSub (named FermixWeb.PubSub)
  4. FermixWebWeb.Endpoint

The endpoint only binds a network socket when server: true is set. BootProfile enables it for a standalone daemon run or an app-managed engine. CLI paths such as version halt before the web application starts; browser setup uses the running daemon’s endpoint.

Realtime supervisor

Module: FermixCore.Realtime.Supervisor Strategy: :one_for_one (conditional child of the core supervisor)

  1. Realtime.SessionSupervisor: dynamic supervisor for per-session servers.
  2. Task.Supervisor: for transient realtime work.
  3. Realtime.LocalVoiceSocket: Unix-domain accept loop.

Boot gating: this supervisor is started only when realtime is enabled in config, the provider is "openai", and an OpenAI API key is present. Otherwise it stays unstarted and adds no overhead.

See realtime voice for configuration details.

Boot sequence

The core application runs the following steps before starting the supervision tree:

FermixCore.Application.start/2

  BootProfile.select(build_identity, standalone_detector)
  app engine:       prepare daemon gates -> start_supervision_tree()
  standalone CLI:   cli_dispatch(argv)
  source:           start_supervision_tree() with configured runtime gates

standalone "run":
  BootProfile.prepare(:standalone_cli)   # PATH baseline, web endpoint, daemon/voice socket gates
  start_supervision_tree()

start_supervision_tree:
  remember_launch_cwd()                  # used by sandbox path policy
  ConfigStore.ensure_workspace()         # mkdir ~/.fermix/*
  BootstrapRename.run()                  # migrate legacy bootstrap file names
  IdentityName.reconcile()               # rewrite IDENTITY.md Name: to configured assistant name
  AuthStore.validate_permissions!()      # auth.json must be 0600 (or missing)
  setup_file_logger()                    # rotating fermix.log
  redact_default_logger()                # secret-redacting formatter on console/crash logs
  Trace.TelemetryHandler.attach()        # bind :fermix.* telemetry
  Sandbox.DecisionTelemetry.attach()     # bind :fermix.sandbox.decision
  maybe_ensure_sidecar()                 # daemon boot only; shared desktop/history helper
  children = [...]                      # records engine ownership when the daemon socket is enabled
  Supervisor.start_link(children, ...)

AuthStore.validate_permissions!() aborts startup if auth.json exists with permissions other than 0600. This runs before any network process starts so no credential is reachable over a misconfigured socket.

maybe_ensure_sidecar() runs only with the daemon socket enabled, outside test builds. It requests the shared helper when either computer use or macOS Computer History is enabled and the pinned helper is missing. The download happens before tool registration and supervisor gates read installation state. It is bounded at 30 seconds; a failed download is logged and leaves the helper unavailable. A later daemon start or setup install can try again. A present helper still has to pass the protocol handshake before use.

Meeting and local-transcription helpers are installed through setup, not downloaded at daemon boot. Their worker processes start only when a meeting or transcription request needs them.

  • Architecture: the full umbrella structure and cross-cutting invariants.
  • The agent loop: what MainAgent and AgentLoop do inside a turn.
  • Traces and telemetry: what FermixCore.Trace writes and how telemetry handlers attach.
  • Scheduled jobs: how Jobs.Scheduler and Jobs.RunnerSupervisor work.
  • Channels: which adapters are supervised and which use webhook ingress.
  • MCP: how Capabilities.MCP.Supervisor discovers and registers outbound MCP tools.
  • Realtime voice: gating conditions and session lifecycle for the realtime supervisor.
  • Meeting notetaker: admission, transcription, artifacts, and delivery.
  • Computer History: capture, summary, and retention behavior.

Next steps