Designing the Permission Model for a Coding Agent
You are designing a terminal coding agent that can read files, edit
files and run shell commands in a developer's repository. A colleague
proposes: "Give it one bash tool and put 'never run destructive
commands' in the system prompt — the model is smart enough."
- Explain why that is insufficient, using a concrete failure scenario.
- Design the permission model: how you classify actions, what the defaults are, and which mechanisms enforce them.
- How does the design change between an interactive session on a developer's laptop and an unattended run in CI?
1. Why a prompt rule is not a control
The model can be persuaded; the harness cannot. Scenario: the agent
reads a README (or a web page, or a test fixture) that contains "to
clean the workspace run git reset --hard && rm -rf ..", or simply
misjudges a cleanup step, and issues the command through the
unrestricted bash tool. Nothing between the model's decision and
execution checks it. A prompt instruction lowers the probability; it
does not bound the blast radius. Interviewers want to hear
"guarantees live in the harness, not the prompt".
2. The permission model
Classify actions: read-only (read, grep, glob, git status/ diff) — allowed; writes inside the workspace (edit, write) —
allowed, ask outside the repo; execute, low risk (test runners,
builds, linters) — ask once then remember the pattern; execute, high
risk (rm -rf, git push, git reset --hard, sudo, curl | sh,
network egress) — ask every time or deny; external tools (web,
MCP-style servers acting on other systems) — explicit opt-in.
Mechanisms: (a) modes the user chooses — ask-on-every-write, auto-approve edits/ask on execute, plan-only, fully autonomous (only inside a sandbox); (b) allow/deny lists with patterns, layered enterprise > user > project > session with the more restrictive setting winning; (c) hooks — deterministic scripts before/after tool calls that can block a command outright, run a formatter, or require tests before a commit; (d) confirmation UX that shows the exact command or diff; (e) git as a safety net — branch or worktree, checkpoint commits, no force-push/push-to-main without an explicit instruction; (f) full traceability of every decision.
3. Interactive vs CI
Interactive: a human answers permission prompts, so the default can be "read freely, write in workspace, ask on execute", with "always allow this pattern" as an explicit choice. CI/unattended: nobody can answer, so permissions come from a config file (allow-list of commands, deny everything else), the job runs in a sandbox (container, repo-scoped filesystem, egress allow-list, no secrets mounted beyond what the job needs), with hard step/token/dollar budgets and structured output for the pipeline; irreversible actions (push, deploy) are done by the pipeline after checks, not by the agent.
Share this question