Meeting notetaker

Fermix can attend a meeting for you, listen, and send you the notes afterwards. You paste the meeting link into a chat and ask it to join; it joins, transcribes what is said with speaker names attached, and when the meeting ends it writes a summary back into the same conversation.

It is off by default, and it joins only when you ask for it in that turn — never on a schedule, never off a calendar invite it happened to read, and never because a link appeared in something someone else wrote. A forwarded invite or a pasted agenda is material to report on, not an instruction to obey. It also attends one meeting at a time: asking for a second names the meeting already in progress instead of queueing behind it.

Two platforms are supported and they work in completely different ways. Google Meet is the path that has been run against live meetings. The Zoom lane is implemented but has not been exercised against a real Zoom meeting, and it depends on a paid Zoom add-on that is not self-service — read Zoom before you plan around it.

Two lanes, one feature

On Google Meet, Fermix drives a real browser and joins as a visible, signed-in participant. On Zoom there is no participant at all: Zoom itself streams the meeting audio to Fermix. The link decides the lane, and there is no crossover — a Meet link never rides the Zoom path, and an unconfigured lane refuses with its own reason rather than quietly trying the other one.

You, in a chat with Fermix
  │   "join this meeting: <link>"
  ▼
join_meeting                    attended owner turn only · one meeting at a time
  │
  ├── meet.google.com/…  ──▶  fermix-meetbot sidecar
  │                              ↕  a real Chromium, signed in as your bot account
  │                           Google Meet: a visible participant, muted, no camera
  │
  └── zoom.us/j/…        ──▶  Zoom RTMS subscription
                                 ↕  WebSocket, audio only
                              Zoom: no participant; Zoom shows its own banner
                                      │
                                      ▼
                    your configured transcription backend
                                      │
                                      ▼
        attributed transcript  ──▶  summary  ──▶  the conversation that asked

Lane	Google Meet	Zoom
How it gets in	A browser sidecar signs in as a dedicated Google account and knocks like any guest	Zoom Realtime Media Streams (RTMS) pushes meeting audio to Fermix over a WebSocket
Visible in the roster	Yes, under the bot account’s own profile name	No — there is no participant
Consent surface	One chat message posted on admission, then silence	Zoom’s own in-meeting banner, shown and controlled by Zoom
Host control	Admit from the waiting room; remove at any time	Turn app sharing off mid-meeting
Whose meetings	Any meeting the bot account is admitted to	Only meetings hosted by your own Zoom account, or by a host who has enabled your RTMS app
What you must obtain	A spare Google account	Four values from the Zoom App Marketplace, plus a paid Zoom add-on
Status	Run against live meetings	Implemented; not validated end to end

What it does while it is there

Both lanes behave the same once capture starts.

• It transcribes continuously through your configured transcription backend, and attributes each line to a speaker.
• It never speaks. On Meet it posts exactly one chat message when it is admitted; after that it is silent for the rest of the meeting. On Zoom it posts nothing, because there is nothing there to post from.
• It leaves on its own. The meeting ends when the host removes it, when you ask it to leave, or after four hours of capture. On Meet it also leaves about a minute after the last other participant does (long enough for a dropped host to come back), and a notetaker admitted to a room nobody else ever enters gives up after ten minutes. On Zoom, where the only evidence of presence is who is transmitting, it leaves after ten minutes of nobody transmitting.
• Then it writes the notes and delivers them to the conversation the request came from, or to the owner’s inbox when that conversation has no channel to send into.

The three tools behind this — join_meeting, leave_meeting, and list_meetings — are attended-owner-only. An attended turn means an interactive chat or CLI turn, or a live voice call; a guest, a scheduled job, a delegated sub-agent, and a coding agent continuation never see them. The check runs again at execution, so calling one by name from an unattended turn is refused rather than served. See the tool reference for their parameters.

On macOS, Fermix holds a system sleep guard while capture is running and releases it when the meeting ends. A join that fails after the initial acknowledgment sends the reason back to the conversation that requested it.

Turning it on

The notetaker is configured entirely from the web setup page. There is no terminal-wizard step and no fermix setup flag for it.

1. Run fermix setup and open the link it prints, then go to the Plugins tab.
2. Find the Meeting Notetaker card and click Enable. This downloads the fermix-meetbot sidecar — a single signed executable, verified against a checksum baked into your Fermix build — and then has that sidecar install its own version-matched Chromium. Both steps run from the card; there is no npx, no package manager, and nothing to type in a terminal. The browser download is around 150 MB, so give it a minute.
3. Click Configure to open the card’s panel, and use Sign the bot in to sign the bot’s Google account in (see Google Meet below). Installed is not the same as ready: until the sign-in finishes, the card says Sign-in needed and the Meet lane cannot join anything.
4. If you want the Zoom lane too, fill in the four Zoom fields in the same panel (see Zoom).
5. Save, then Apply & restart. Enabling the notetaker and changing its settings both take effect on the next daemon restart.

Enabling the subsystem on its own advertises nothing. The meetings tools require either the Meet sidecar and its browser to be installed, or a complete set of Zoom credentials. Google sign-in is a separate prerequisite for a successful Meet join: installed tools can still refuse a meeting when the bot’s browser session is missing or expired.

The sidecar is published for Apple Silicon macOS and for Linux on both aarch64 and x86_64. On any other host — an Intel Mac, for instance — the download refuses with a “no build for this machine” error rather than installing something that cannot run.

To check the state without joining anything, run fermix doctor. Its meetings row reports disabled, or which lanes are usable, or enabled but no lane is usable followed by the specific thing that is missing.

Google Meet

The Meet lane is a real browser. The sidecar drives a Chromium that is signed in to a Google account you control, and that browser joins the meeting the way a person would: it opens the link, knocks, and waits in the waiting room until the host admits it.

Because it is an ordinary participant, everything a host can do to a participant works on it. The host admits it or does not. The host can remove it at any moment, and removal ends the capture immediately.

Create an account for it first

Use a dedicated Google account, not your own. The bot’s browser stays signed in to whatever account you give it, and that account is what appears in the meeting.

Two things follow from that, and both matter:

• Name the account something obviously non-human. Meet shows the account’s own profile name in the roster, so an account called “Fermix Notetaker” reads correctly and an account called by your own name reads as you sitting silently in the meeting. The display name in Fermix’s own settings does not change what Meet shows.
• Do not use your personal or work account. The sign-in is a real, persistent browser session that the sidecar owns.

Create one at Google account signup if you do not have a spare. Nothing else on the Google side is needed: no API key, no Google Cloud project, no OAuth client, and no Workspace admin involvement.

Sign the bot in

The sign-in is one deliberate act you perform by hand, once.

1. Open the setup Plugins tab, find the Meeting Notetaker card, and click Configure.
2. Expand the Google Meet section and click Sign the bot in.
3. A real Chromium window opens on your desktop, showing Google’s ordinary sign-in page. Sign in to the bot account yourself, including any two-factor step. Nothing types a password for you — Fermix never sees the credentials, and it never reads inside the browser profile afterwards. The window waits about ten minutes for you to finish.
4. When the sign-in completes the window closes and the card’s Google Meet section flips to ready.

The signed-in state lives in a Chromium profile on your machine. Signing in again (the button becomes Sign in again) replaces it, which is also the fix if Google ever invalidates the session.

Tip: adding the bot account’s email address to a meeting invite lets it skip the waiting room entirely, because Meet treats an invited account as expected.

What the room sees

The notetaker joins with no camera and a muted microphone, and it never unmutes. On admission it posts a single message in the meeting chat and then stays silent. The default line names itself and you, and says what it is doing:

👋 Fermix Notetaker here — Alex's AI notetaker. Taking text notes only
(no audio kept); the host can remove me anytime.

That message is the consent posture for this lane, so it is on by default. You can change the wording, or the name it appears under in the announcement, in the card’s Configure panel; leaving the announcement text blank uses the line above with your own name filled in from your personalization settings. You can also switch the announcement off, which is a real choice with real consequences: on Meet the bot is still visible in the roster, but nobody is told what it is for.

Zoom

The Zoom lane has no bot. It uses Zoom RTMS (Realtime Media Streams), an official Zoom feature that streams a meeting’s audio out to an application you have registered with Zoom. Nothing joins the meeting, nothing appears in the participant list, and nothing is posted in the chat.

What participants see instead is Zoom’s own in-meeting banner — the notice that the content of the meeting is being shared with one or more apps. That banner, and the host’s ability to switch sharing off from it mid-meeting, are the entire consent surface for this lane. Fermix does not add to it and cannot suppress it.

RTMS only reaches meetings hosted by your own Zoom account, or by a host who has explicitly enabled your RTMS app. That is a Zoom platform boundary, not a missing setting; no value you can enter unlocks other people’s meetings.

Status: this lane is implemented but has not been run against a live Zoom meeting. It also has a hard commercial prerequisite, described next, that keeps it out of reach for most accounts. Google Meet is the supported path today.

Prerequisites, before any of the fields matter

Prerequisite	Where it lives	What happens without it
Zoom Developer Pack credits on the account	A paid Zoom add-on. It is not self-service; Zoom routes it through Contact Sales.	Everything can be configured correctly and Zoom simply never sends the stream-started event. There is no error to read — the meeting just runs and nothing arrives.
Share realtime meeting content with apps, enabled by an account admin	Zoom web portal → Account Management → Account Settings → Zoom Apps	The stream never begins.
The app set to auto-start for meetings	The app’s own settings in the Zoom App Marketplace	The stream never begins unless someone starts the app inside each meeting.

The credit requirement is the one to check first. It is the difference between “this needs another hour of configuration” and “this account cannot do it at all.”

Building the Zoom app

Per Zoom’s RTMS documentation, RTMS is set up on a General app, user-managed, created in the Zoom App Marketplace:

1. Sign in to the Zoom App Marketplace and choose Develop → Build App → General App.
2. Set the app to user-managed.
3. Under Scopes, search for RTMS and add the RTMS scopes.
4. Under Features → Event Subscriptions, add a subscription and choose WebSocket as the delivery method.
5. In that subscription, subscribe to the RTMS Started and RTMS Stopped events. See adding RTMS features for the current list.
6. Activate the app for your own account.

Where each value comes from

Fermix asks for four values, in the Zoom section of the Meeting Notetaker card’s Configure panel.

Field in Fermix	Where Zoom shows it	Notes
Account ID	The App Credentials page of a Server-to-Server OAuth app	See the note below — a General app does not issue one.
Client ID	The app’s App Credentials page	Copied as-is.
Client secret	The same page, revealed once	Stored in your OS keychain, never in config.toml. The field shows “Already set” afterwards; leave it blank to keep the stored value.
Webhook subscription ID	Embedded in the WebSocket connection URL Zoom shows for the event subscription you created in step 4	Zoom does not label it as a field. The URL looks like wss://ws.zoom.us/ws?subscriptionId=…; the subscription id is the value of that query parameter. See Zoom’s WebSocket documentation.

One of these four does not line up with the app type Zoom’s RTMS documentation describes, and it is worth knowing before you start. A General app issues a Client ID and a Client Secret; it does not issue an Account ID. The Account ID belongs to Zoom’s Server-to-Server OAuth app type, which is a different app. So the set of values Fermix asks for does not map cleanly onto any single documented Zoom app today. Fermix requires all four before it treats the lane as configured, and it refuses at the join gate rather than half-way through the handshake when one is missing. Combined with the credit prerequisite above, this is why the Zoom lane is described here as unvalidated rather than as a working recipe.

Asking for notes

You trigger the notetaker by giving Fermix the meeting link and asking it to join, in the same message or in the same turn. It answers immediately and does the joining, waiting, capturing, and summarizing afterwards on its own.

You:     Join this and take notes: https://meet.google.com/abc-defg-hij

Fermix:  On my way in — I'll knock and wait for the host to admit me.
         I'll send the notes here when the meeting wraps up.

Ask it to leave early and it still delivers what it heard:

You:     You can drop off now.

Fermix:  Left the meeting. Writing up what I heard — notes coming shortly.

And to find out where things stand:

You:     Are you in a meeting right now?

Fermix:  Yes — "Design review", on Google Meet, capturing since 14:02.

The notes you get

When the meeting ends, Fermix delivers a written summary into the conversation the request came from, structured as:

• TL;DR — what the meeting was about and what came of it.
• Decisions — what was actually settled.
• Action items — with the owner named where the meeting named one.
• Open questions — what was raised and left unresolved.
• Links — anything shared in the meeting.

Capture failures produce partial notes when a transcript was recovered. A crashed Meet sidecar, a lost Zoom stream, or failed transcription labels the resulting summary as partial. Asking the notetaker to leave or removing it from the room ends capture normally; the summary covers what it heard before leaving. If no delivery target can be resolved, delivery fails and the summary stays on disk, where list_meetings keeps reporting its path.

Long transcripts are summarized in up to twelve parts before the final notes are assembled. A longer transcript can exceed the summary limit; the full transcript remains on disk. A failed part fails the summary rather than silently omitting it.

Transcription and summary models

Meetings use the backend selected under [fermix_core.transcription], or the transcription_backend override in the meeting settings. Deepgram, SpaceXAI, and the on-device local backend stream transcription; OpenAI transcribes short spoken chunks in order. Configure the shared backend on setup’s Voice notes tab. The local backend currently has a downloadable build for Apple Silicon macOS only.

The summary is a separate model call. Using local transcription does not make the summary local: the transcript and participant names are sent to the summary provider. The summarizer treats both as untrusted content and has no tools.

To choose a summary model, edit these keys in config.toml, then restart:

[fermix_core.routing]
meeting_provider = "openai"
meeting_model = "gpt-5.6-luna"
meeting_reasoning_effort = "low"

All three are optional. With no provider or model override, the summary uses the primary provider and its configured fallback chain. A provider or model pin selects one route; a model without a provider uses the primary provider. Setting only the reasoning effort preserves the provider chain. These settings have no setup panel or chat-tool editor. fermix doctor checks them in its routing row.

Artifacts on disk

Every meeting writes into <FERMIX_HOME>/workspace/meetings/<meeting id>/, normally under ~/.fermix:

File	Contents
transcript.jsonl	One line per attributed speech segment.
transcript.md	The same transcript, timestamped and speaker-labelled, for reading.
summary.md	The generated meeting notes, including a partial-capture warning when applicable. Written when summarization succeeds.
meta.json	Platform, title, times, status.
audio.raw	Only present when retain_audio is on.

That directory sits inside the workspace sandbox floor, so the ordinary file tools can read it back: you can ask about a meeting days later and Fermix can go and look. Raw audio is not kept unless you turn it on — the text transcript is what persists by default.

Configuration reference

[fermix_core.meetings] keys

Key	Default	Notes
enabled	false	Master switch. On its own it advertises no tools; a lane has to be usable first.
bot_name	"Fermix Notetaker"	The name used in the default announcement. On Meet, the roster shows the Google account’s own profile name instead.
announce	true	Post one consent message in the meeting chat on admission. Google Meet only; Zoom has no chat to post into.
announce_message	unset	Replaces the default announcement. Blank uses the built-in line, filled with bot_name and your [fermix_core.personalization] user_name.
transcription_backend	unset	Use a different speech-to-text backend for meetings only. Blank uses the globally configured one.
retain_audio	false	Keep audio.raw alongside the transcript. Not exposed in setup; set it in config.toml.
zoom_account_id	unset	Zoom RTMS.
zoom_client_id	unset	Zoom RTMS.
zoom_client_secret	unset	Zoom RTMS. Stored in the OS keychain and referenced as @keyring in config.toml — see auth and secrets.
zoom_ws_subscription_id	unset	Zoom RTMS.

The four Zoom keys are all-or-nothing: the lane counts as configured only when every one of them resolves to a real value, so a partly-filled set refuses at the join gate instead of failing deep inside Zoom’s handshake.

[fermix_core.meetings]
enabled = true
bot_name = "Fermix Notetaker"
announce = true
retain_audio = false

Settings are read once when a meeting is requested and never re-read, so editing config while a meeting runs cannot change that meeting’s posture mid-flight. Changes apply on the next daemon restart. The same keys are listed in the configuration reference.

What is deliberately not here: there is no knob for how often it summarizes, no recording toggle beyond retain_audio, and no setting that lets RTMS reach meetings you do not host. The first two are fixed behavior; the third is Zoom’s boundary, not Fermix’s.

When it refuses

Each refusal is its own message naming the fix, rather than a generic failure.

Refusal	What it means	Fix
meetings_disabled	The subsystem is off.	Enable it from the setup Plugins tab, then restart.
sidecar_not_installed	The Google Meet sidecar is not installed.	Enable the Meeting Notetaker card, which installs it.
Sign-in needed	The sidecar is installed but the bot account is not signed in.	Sign the bot in, in the card’s Configure panel.
zoom_rtms_not_configured	One or more of the four Zoom values is missing.	Fill all four in the card’s Configure panel.
unrecognized_meeting_url	The link is not a Meet or Zoom meeting link.	Give it the join link verbatim; it refuses rather than guessing at a room.
max_concurrent	It is already in another meeting.	It names that meeting; ask it to leave first.
not_attended	The turn is not an attended, top-level owner turn.	Ask from your own chat, your CLI session, or a live voice call. Scheduled jobs, sub-agents, and guests never get this tool.

On Google Meet, being denied admission, blocked, or asked to sign in again are all reported as what they are. The notetaker never reports itself as present in a meeting it did not get into.

Related pages

• Setup: the web setup page the Meeting Notetaker card lives on.
• Plugins: the same setup tab, and how the other install-on-enable cards work.
• Configuration: every config.toml key, including transcription backends.
• Tool reference: parameters and failure tags for join_meeting, leave_meeting, and list_meetings.
• Sandbox: why the meetings directory is readable by the file tools.