Skip to Content
Connect an AgentOverview

Connecting an agent

Autonomy is not an agent. It’s a local runtime — an MCP server and a CLI — that gives an agent access to a user’s real accessibility setup: VoiceOver, keyboard-first control, Switch Control, speech, Zoom, and the consent model that keeps high-risk actions in the user’s hands. “Connecting an agent” means pointing an MCP-capable host at that runtime so it can call Autonomy’s tools instead of guessing at pixels.

If you’re arriving here from a generic computer-use background, recalibrate one assumption: Autonomy never spawns or owns the agent process. The agent — Claude Code, Codex, or anything else that speaks MCP — already exists as its own process with its own model and its own conversation loop. Autonomy’s job is to be the thing that process talks to when it needs to read screen state, click something, speak to the user, or ask permission before a risky action.

This page is the map. It covers the two ways an agent gets connected, what “MCP-native” replaced, and where to go next depending on which host you’re wiring up.

Two ways to connect

The default agent. For users with no strong preference, Autonomy ships a thin launcher over a standard CLI agent (claude), pre-wired to the Autonomy MCP server and the Autonomy rules plugin. It’s a default configuration, not a custom agent Autonomy maintains — see Claude Code for the concrete setup.

Bring your own agent (BYO). Any MCP client — Claude Code, Codex, or a custom host you’ve built — can connect to the same daemon over stdio and drive the in-app Agent Conversation panel through the agent_conversation_* tools. Because the integration point is simply “an MCP client,” the surface is agent-agnostic by construction. See Bring your own agent for the wire-level detail.

Both paths terminate at the same daemon, the same tool catalog, and the same consent gate. There is no separate “BYO tier” with fewer capabilities or a different safety model — a bring-your-own agent that authenticates the same way sees the same tools and is held to the same consent rules as the default agent.

Why this replaced in-app agent hosting

Until recently, the in-app Agent Conversation panel worked by spawning a codex exec subprocess (nightgram) and parsing its stdout. That subprocess had no connection back to the Autonomy MCP server — it couldn’t check accessibility state, couldn’t request consent, couldn’t speak through the VoiceOver transport. It was a one-shot CLI call wearing a chat UI.

ADR-011  retired that path. nightgram is no longer built, bundled, signed, or launched by the app. The in-app conversation is now MCP-native: Autonomy hosts the tool surface and the consent boundary, and the agent — default or BYO — is the one holding the conversation. Autonomy’s behavioral rules (consent first, accessibility as the north star, respecting pace) ship as plugin artifacts each ecosystem already understands — a Claude Code plugin, a Codex config — rather than being hardcoded into app-side agent code.

Mental model: the daemon as a switchboard

Think of the Autonomy daemon as a telephone switchboard, not a phone.

Every caller dials into the same switchboard using the same protocol (MCP over stdio, JSON-RPC 2.0). The switchboard doesn’t care which agent is calling — it exposes the same tool catalog, enforces the same consent classes, and writes to the same local, redacted event log regardless of caller. What changes per-agent is configuration — which tools it’s shown, and which rules-as-plugins it reads — not a different runtime underneath.

Where this fits

agent host (Claude Code / Codex / your agent)
  -> plugin / MCP config  (this section)
  -> Autonomy MCP server  (a7y-mcp, stdio)
  -> a7y runtime crates    (accessibility, browser, speech, playbooks)
  -> redacted local evidence and the macOS app surfaces

The concrete connection is one shared file. plugins/autonomy/.mcp.json is what both the Claude Code and Codex plugins point at:

{
  "mcpServers": {
    "autonomy": {
      "command": "/Applications/autonomy (a7y).app/Contents/Resources/bin/a7y-cli",
      "args": ["daemon", "proxy"]
    }
  }
}

The command targets the installed, signed app bundle rather than a relative dev path, on purpose — macOS ties Accessibility and Screen Recording grants to that exact app identity, and a path that moves between sessions loses them.

The root of this repository also has cursor.mcp.json, windsurf.mcp.json, and gemini.mcp.json, which configure an unrelated, development-only Agent Mail coordination server (http://127.0.0.1:8765/mcp/) used for parallel multi-agent repo work. codex.mcp.json at the root is just an empty {"mcpServers": {}} stub — not wired to Agent Mail or anything else. None of these are Autonomy’s MCP config, and connecting to one will not give an agent any Autonomy tools. The real one is plugins/autonomy/.mcp.json.

Pitfalls

  • Don’t assume a passing connection means the daemon spawned an agent. Autonomy never launches, supervises, or restarts an agent process — if an agent isn’t running, connecting a config file won’t start one.
  • A “default agent” isn’t a hardcoded one. The claude launcher is a configuration choice Autonomy ships, not special-cased code — a bring-your-own agent sees the identical tool catalog and consent gate. If you’re wiring up a narrower specialist instead of one general connection, the tool catalog can be scoped per domain rather than loaded whole — see MCP tool catalog.

Where to go next

Last updated on
Claude Code