# Capabilities and tools

> How Fermix represents, registers, filters, and executes tools: built-in, plugin, and MCP-discovered capabilities through a single shared registry, and how skills fit alongside.

Tools are the actions the agent can take: reading a file, running a shell command, searching the web, and so on. Every tool the model can call (whether it is built into Fermix, registered by an installed plugin, or discovered from an MCP server, a standard way for outside programs to expose tools to an AI agent) is one entry in a single shared list, the `Capabilities.Registry`. There is no separate set of "main agent tools" vs "voice tools" vs "scheduled-job tools." Trust level and policy filters decide which subset each call site sees. Skills are the one exception: they are instruction packages, not provider-visible tools, and surface through a compact catalog in the prompt plus the `skill_view` and `skill_run` built-ins — see [skills](/docs/skills).

## Capability shape

Each registered capability is a struct with these fields:

```elixir
%Capability{
  name: "file_read",
  description: "Read a UTF-8 text file from the workspace.",
  parameters: %{           # JSON Schema object
    type: "object",
    required: ["path"],
    properties: %{
      path: %{type: "string", description: "..."}
    }
  },
  kind: :builtin,          # :builtin | :skill | :mcp
  executor: {FermixCore.Tools.FileRead, :execute, []},
  hidden_from_agent?: false,
  policy_class: :read_only,
  metadata: %{
    category: :file,
    when_to_use: "...",
    examples: [...],
    failure_modes: [...]
  }
}
```

The `parameters` map is a JSON Schema object. `AgentLoop` passes the formatted schema to the configured provider verbatim. The `executor` tuple is called with the parsed argument map when the model issues a tool call.

## Three kinds of capability

| Kind | Source | Registered by |
|------|--------|---------------|
| `:builtin` | Pure-Elixir tool modules compiled into Fermix, sandbox preset commands, and the tools of ready plugins. | `BuiltinSeeder` at boot; `Sandbox.CommandCapabilities` for preset shell commands; `Plugins.Capabilities` for plugin tools (tagged `plugin_owned?` in metadata; a plugin that runs a local MCP process contributes its tools as `:mcp` instead). |
| `:skill` | Reserved in the schema for skills, but the live runtime deliberately does **not** register one capability per skill — skills surface through the prompt's skill catalog and the `skill_view` / `skill_run` built-ins. | `SkillRegistry` keeps skills in its own store and clears any stale `:skill` entries from the capability registry on reload. |
| `:mcp` | Outbound MCP server `tools/list` response, registered as `mcp_<server>_<tool>` (a plugin-owned MCP server uses `<plugin>_<tool>`). | The per-server `Capabilities.MCP.Server` process when its client connects; each of its tools is unregistered when the connection goes down. |

`unregister_kind/3` accepts a metadata match map so a whole family of capabilities can be removed atomically — for example, every `plugin_owned?` tool when plugins reload, or every sandbox preset command when the sandbox config changes.

For more on skills, see [skills](/docs/skills). For plugins, see [plugins](/docs/plugins). For MCP integration, see [MCP](/docs/mcp).

## Policy classes

Every capability carries one `policy_class` that describes its effect category:

| Class | Meaning | Example built-ins |
|-------|---------|-------------------|
| `:read_only` | Reads data without modifying anything. | `file_read`, `content_search`, `glob_search`, `git_read`, `memory_recall`, `tool_help`, `list_jobs`, `memory_sources_list`, `send_attachment` |
| `:read_write` | Writes or mutates local state. | `file_write`, `file_edit`, `git_write`, `memory_store`, `schedule_job`, `pause_job`, `resume_job`, `remove_job`, `skill_create`, `model_routing_config` |
| `:exec` | Spawns an OS process or delegated run. `shell` goes through the [sandbox](/docs/sandbox). | `shell`, `skill_view`, `skill_run` |
| `:network` | Outbound HTTP, validated by `Net.Guard`. | `web_fetch`, `web_search`, `browser` |
| `:external_api` | Calls a third-party API, typically an LLM provider. All discovered MCP tools default to this class (a per-tool `policy_class` override exists in `[mcp.servers.<server>.tools.<tool>]`). | `subagents`, `generate_image`, every `:mcp` capability |
| `:gui_control` | Drives the host desktop by screenshot and mouse/keyboard. Operator-only; never delegated to subagents (helper agents the main agent spins up for a piece of work). | `computer_use` (experimental, off by default) |

Subagent workers always run with the parent trust level's classes minus `:read_write` and `:gui_control`: delegated work can read, browse, and call APIs, but never writes local state or touches the desktop.

## Trust model

Trust is stamped at ingress (the point where a message enters Fermix, for example a chat channel or the CLI) and flows through `MainAgent` to `AgentLoop` to `CapabilityRegistry.list_for/2`, which applies the default policy filter for that trust level.

| Trust | Default allow | Default deny | Where it is set |
|-------|---------------|--------------|-----------------|
| `:operator` | `[:read_only, :read_write, :exec, :network, :external_api, :gui_control]` | `[]` | The human owner via CLI, daemon socket (the local connection to the long-running Fermix background process), voice click-to-talk, or the channel's `owner_user_id`. Also applies to scheduled jobs created by the operator and bundled or operator-installed skills. |
| `:guest` | `[:read_only]` | `[:read_write, :exec, :network, :external_api, :gui_control]` | Non-owner senders in `allowed_user_ids`; plugin-shipped skills (under `~/.fermix/plugins/`) whose code the operator has not reviewed. |
| `nil` | (treated as `:guest`) | | Any call path that did not set `:trust`. A missing trust value degrades to read-only; it never silently grants full access. |

A `list/2` call with no `:trust`, `:policy`, or `:policy_classes` option bypasses the default filter and returns every registered capability. Internal callers such as `fermix capabilities` use this primitive path.

See [ingress and trust](/docs/ingress-and-trust) for how trust is assigned per channel and per sender.

## Filter parameters

`CapabilityRegistry.list_for/1,2` accepts these options:

```elixir
filter() :: [
  allowed_tools:       [String.t()] | nil,
  policy:              policy_spec(),
  policy_classes:      [policy_class()] | nil,
  trust:               :operator | :guest | nil,
  kind:                :builtin | :skill | :mcp | :all,
  excluded_categories: [atom()] | nil,
  excluded_names:      [String.t()] | nil,
  include_hidden?:     boolean()
]
```

Filters compose in this order: kind, then policy, then allowlist (the explicit list of tool names permitted), then name exclusion, then category exclusion, then hidden filter.

Key semantics:

- `allowed_tools: []` means deny all. `allowed_tools: nil` means no narrowing by name.
- `policy: [:read_only]` is shorthand for `[allow: [:read_only], deny: []]`.
- `excluded_categories: [:scheduling]` removes every tool whose `metadata.category` matches, without listing each tool name.
- `include_hidden?: true` surfaces capabilities flagged `hidden_from_agent?: true`. Default is `false`. Operators use this flag to keep an MCP tool configured but invisible to the model.

## Registry API

| Call | Notes |
|------|-------|
| `register/2` | Insert a capability. Returns `{:error, {:duplicate_name, name}}` if a capability with that name already exists. |
| `unregister/2` | Remove by name. |
| `unregister_kind/2,3` | Remove every capability of a kind, optionally filtered by a metadata match map. Used to clear capability families atomically: plugin tools on reload, sandbox preset commands on config change. |
| `find/2` | ETS lookup by name. Hot path; bypasses the GenServer. |
| `list/1,2` / `list_for/1,2` | List all capabilities, with optional filtering. |
| `refresh/2` | Currently a no-op for every kind. Real re-syncs go through the owning components: plugin reload, `skill_reload`, and MCP connect/disconnect. |
| `resolve_policy/2` | Translate `(trust, policy)` into the effective allow/deny list. Public API. |

## Built-in tools

Thirty-five built-in tools ship unconditionally with Fermix, plus three bridges for deferred tools (`tool_search`, `tool_describe`, `tool_call`) when tool-schema deferral is enabled (on by default). Deferral, explained below, keeps rarely-used tool definitions out of each request to save tokens. Each implements the `Builtin.Tool` behaviour. One more built-in, `computer_use`, is seeded conditionally: it is **experimental and off by default**, and registers only when computer use is enabled in config and its native helper is installed (see the table row and the note below).

| Name | Policy class | Category | What it does |
|------|-------------|----------|--------------|
| `shell` | `exec` | system | Run a shell command via `CommandRunner` with sandbox-resolved working directory and env. |
| `file_read` | `read_only` | file | Read a UTF-8 text file, gated by the sandbox read-path check. |
| `file_write` | `read_write` | file | Write a file, gated by the sandbox write-path check. |
| `file_edit` | `read_write` | file | Replace an exact-string span in a file. |
| `glob_search` | `read_only` | file | Pattern-glob the workspace. |
| `content_search` | `read_only` | file | Regex-search file contents. |
| `git_read` | `read_only` | git | Run read-only git queries: status, log, diff, branch, show. |
| `git_write` | `read_write` | git | Run mutating git commands: add, commit, checkout, pull. Pushing is deliberately excluded. |
| `web_fetch` | `network` | web | Fetch a public URL through `Net.Guard`, with a body cap and IP pinning. Returns the page's readable text, or the JSON verbatim when the endpoint serves JSON. |
| `web_search` | `network` | web | Web search with DuckDuckGo as the default keyless backend; six configurable backends (Brave, Exa, Tavily, Parallel, Perplexity, Firecrawl) are supported when set up. A hard error on a configured backend degrades once to DuckDuckGo, loudly. |
| `browser` | `network` | web | Drive a supervised local browser for JavaScript-rendered pages, forms, and interactive content. The `screenshot` action returns the captured page as an image the model can see, not just a saved path; the managed browser caps live tabs and auto-closes the oldest non-active tab past the cap. |
| `memory_recall` | `read_only` | memory | Look up a stored memory by key, list this conversation's memories, or search stored memories and past conversation history. |
| `memory_store` | `read_write` | memory | Persist a memory candidate through the admission pipeline. |
| `memory_sources_list` | `read_only` | memory | List configured memory sources. |
| `subagents` | `external_api` | delegation | Run one or more temporary subagents for delegated work, concurrently up to a cap. |
| `skill_create` | `read_write` | skill_admin | Scaffold a new local Fermix skill with `SKILL.md` and starter evals. |
| `skill_reload` | `read_write` | skill_admin | Reload a skill from disk without restarting the daemon. |
| `skill_view` | `exec` | skill_admin | Show an installed skill's instructions and metadata. |
| `skill_run` | `exec` | skill_admin | Run an installed skill by name. |
| `skill_list` | `read_only` | system | List installed skills available to the agent. |
| `schedule_job` | `read_write` | scheduling | Create a cron-style scheduled agent run. The effective capability policy comes from the creator's trust level, not the model's. |
| `update_job` | `read_write` | scheduling | Update a scheduled job's schedule, prompt, or config. |
| `list_jobs` | `read_only` | scheduling | List existing scheduled jobs. |
| `list_job_runs` | `read_only` | scheduling | List past runs for a scheduled job. |
| `get_job_run` | `read_only` | scheduling | Fetch detail for one scheduled job run. |
| `run_job_now` | `read_write` | scheduling | Trigger an immediate out-of-schedule run of a job. |
| `pause_job` | `read_write` | scheduling | Pause a scheduled job. |
| `resume_job` | `read_write` | scheduling | Resume a paused scheduled job. |
| `remove_job` | `read_write` | scheduling | Delete a scheduled job. |
| `model_routing_config` | `read_write` | config | Read or update model routing config (provider, model, effort). |
| `generate_image` | `external_api` | media | Create or edit a raster image from a text prompt. The result is written under the sandbox `media/` floor and sent to the current chat automatically. Backend is `openai`, `openai_codex`, `xai`, or `google`, configured via `[fermix_core.tools.generate_image]`. `openai_codex` runs on a connected ChatGPT/Codex login instead of an image API key — billed to that subscription. It is opt-in and fails loudly if the login is missing or not entitled; it never silently falls back to another backend. |
| `tool_help` | `read_only` | system | Return full documentation for one registered capability. |
| `tool_search` | `read_only` | system | Keyword search (using BM25, a standard text-relevance ranking) over deferred plugin/MCP tool schemas by description. Registered when tool-schema deferral is enabled (default on). |
| `tool_describe` | `read_only` | system | Fetch the full schema for one deferred tool by name. Registered when tool-schema deferral is enabled (default on). |
| `tool_call` | `read_only` | system | Invoke a deferred tool by name; the loop unwraps it before policy and dispatch, so the real tool's own policy class is what gets enforced. Registered when tool-schema deferral is enabled (default on). |
| `request_directory_access` | `external_api` | system | Ask the owner, in chat, to approve access to a directory the sandbox currently denies. Surfaces only on attended operator turns; the owner confirms with a single-use 60-second `/confirm` token, the grant persists to `[sandbox] allowed_roots`, and on chat channels the original request resumes automatically. Guest, scheduled, and delegated runs never see it, and unsafe roots (all of `$HOME`, `~/.ssh`, OS roots) are refused before the owner is even prompted. See [sandbox](/docs/sandbox). |
| `send_attachment` | `read_only` | channel | Send a local file through the active channel reply port. URLs are not fetched. |
| `react` | `read_only` | channel | React to the user's message with a single fitting emoji instead of a text reply, for pure acknowledgements ("ok", "thanks", 👍). Advertised only on turns whose channel supports message reactions (on restricted channels the emoji choice is constrained to the channel's allowed set); a delivered reaction with no accompanying text ends the turn without a further model call. |
| `computer_use` | `gui_control` | computer | Drive the host desktop by screenshot and mouse/keyboard, one action per call. **Experimental and off by default.** Registers for the model only when computer use is enabled and the native helper is installed; operator-only and never delegated to subagents. |

The `browser` tool controls a supervised local browser process; web automation belongs there. The `computer_use` tool is a separate, **experimental** capability that drives the host desktop itself — the real mouse and keyboard of the logged-in session, guided by screenshots. It is **off by default**, and it registers for the model only when two things hold: computer use is enabled in config and the native helper is installed. The setup Plugins page (the "Computer Use" card) does both — it sets `[fermix_core.computer_use] enabled = true` and fetches the helper. The helper is fetched from the compux project's own signed release and verified against a SHA-256 checksum baked into your Fermix build, then cached under `~/.fermix/plugins/compux/`; `fermix plugins` does not manage it. You can also set the config key yourself: a daemon that has computer use enabled but is missing the matching helper downloads it on its next start (daemon boot only, bounded to about 30 seconds, and fail-soft — a failed fetch leaves computer use off until the next restart or the setup card). Supported on Apple Silicon (M-series) macOS and Linux x86_64. Config changes apply on the next daemon restart. Operating-system permissions (macOS Screen Recording and Accessibility) are a separate diagnostic, not part of that gate — `fermix doctor`'s computer-use check reports what is missing, so an ungranted permission informs the operator rather than silently hiding the tool. How far it may act follows the [sandbox](/docs/sandbox) mode (`strict` is look-only), and a session only starts from an attended origin — an interactive chat, `fermix ask`, or voice — never from a scheduled job. Before any action that would move the pointer or type — clicking, dragging, scrolling, moving the mouse, typing, pressing a key, or pasting — Fermix checks how long the machine has been idle. If you are actively using it, the agent waits a few seconds for a gap, then either takes its turn or holds the action back and tells you it stepped aside. Idle detection is macOS-only. Where it is unavailable the action proceeds, so this is a courtesy that keeps the agent from fighting you for the cursor, not a safety boundary. To take the machine back yourself, `/pause` hands the cursor and keyboard back to you: the computer-use session stays alive and refuses further actions until you `/resume` — unlike `/stop`, which tears the session down. See [computer use](/docs/computer-use) for the full platform, permission, action-loop, and safety model.

For per-tool parameter schemas, failure modes, and usage examples, see the [tool reference](/docs/tool-reference).

## Tool categories

Categories let a runtime context exclude a class of tools without naming each one individually (`excluded_categories: [:scheduling]`).

| Category | Tools |
|----------|-------|
| `:system` | `shell`, `tool_help`, `skill_list`, `tool_search`, `tool_describe`, `tool_call`, `request_directory_access`, sandbox preset commands |
| `:file` | `file_read`, `file_write`, `file_edit`, `glob_search`, `content_search` |
| `:git` | `git_read`, `git_write` |
| `:web` | `web_fetch`, `web_search`, `browser` |
| `:media` | `generate_image` |
| `:memory` | `memory_recall`, `memory_store`, `memory_sources_list` |
| `:scheduling` | `schedule_job`, `update_job`, `list_jobs`, `list_job_runs`, `get_job_run`, `run_job_now`, `pause_job`, `resume_job`, `remove_job` |
| `:delegation` | `subagents` |
| `:skill_admin` | `skill_create`, `skill_reload`, `skill_view`, `skill_run` |
| `:config` | `model_routing_config` |
| `:channel` | `send_attachment`, `react` |
| `:computer` | `computer_use` |
| `:plugin` | Every tool registered by a ready [plugin](/docs/plugins) |

## How tools are presented to the model

`MainAgent` fetches the filtered capability list for the current trust level and conversation context, then formats each capability's JSON Schema into the provider-specific tool-call format before passing it to `AgentLoop`. The loop calls the provider, parses any tool calls in the response, looks up each capability in the registry by name, and dispatches to the executor. Results are appended to the message list and the loop continues until the model issues a final text response or the iteration cap is reached (default 100 for interactive, sub-agent, and scheduled-job turns; these caps are internal system settings, not `config.toml` keys).

Sandbox enforcement applies to every `shell` call regardless of how it reaches the executor. See [sandbox](/docs/sandbox) for path grants, env allow/deny, and command presets.

## Tool-schema deferral

By default, the full definitions (schemas) for plugin and MCP tools are left out of the request sent to the provider to save tokens; only the tool names remain listed in the runtime prompt. The model fetches a full definition on demand when it actually needs that tool. Three bridge tools handle these deferred tools:

| Bridge | What it does |
|--------|-------------|
| `tool_search` | BM25 search over the deferred catalog to find tools by description. |
| `tool_describe` | Fetch the full schema for one deferred tool by name. |
| `tool_call` | Invoke a deferred tool by name; the loop unwraps it so traces and policy see the real tool name. Direct calls by name also work. |

Deferral is on by default. To disable it, set `[fermix_core.tools.tool_search] enabled = false` in `config.toml`. With deferral off, all schemas are included inline on every call.

## Skills and MCP as capability sources

Skills are **not** registered as provider-visible tools. They live in their own `SkillRegistry`; the model discovers them through a compact skill catalog in the runtime prompt and interacts with them via the `skill_view` and `skill_run` built-ins. Each skill's `SKILL.md` still defines its own allowed-tool boundary (`allowed_tools`), so a skill's sub-turns can be restricted to a subset of built-ins. Bundled and locally installed skills run at operator trust; plugin-shipped skills run at guest trust. A skill's name may not collide with a registered capability name.

Ready plugins register their tools directly into the same registry (as `:builtin` kind, tagged `plugin_owned?` in metadata so they can be swapped atomically on plugin reload). A plugin that runs a local MCP process contributes its tools through MCP discovery instead, named `<plugin>_<tool>`.

Outbound MCP tools enter as `:external_api` policy class by default (overridable per tool with `policy_class` in `[mcp.servers.<server>.tools.<tool>]`) and are removed when the MCP connection drops. Operators can set `hidden_from_agent = true` on any MCP tool in `config.toml` to keep it configured but invisible to the model.

See [skills](/docs/skills), [plugins](/docs/plugins), and [MCP](/docs/mcp) for setup and configuration details.