I have a project that four AI agents work on while I’m not at my computer. One drives the application end to end and files bugs. One reads a bug and proposes how to fix it. I approve or push back. One implements the approved plan and opens a pull request. A fourth runs a long-horizon soak overnight, looking for drift that no short session can see.
There is no orchestrator. No message queue, no coordinator agent, no shared database, no framework. The agents never talk to each other and none of them holds state between runs.
QA agent → files GitHub issue (every 2 hours)
plan agent → posts a plan, waits (every 30 minutes)
↓
human swaps a label (the only gate)
↓
implement agent → opens a pull request (every 45 minutes)
↓
human merges (the other gate)
The interesting part isn’t that agents can write code. That’s table stakes now. The interesting part is what you have to build around them so that a fleet of stateless, forgetful, one-shot processes reliably converges on working software instead of thrashing.
The problem with “just let the agent fix it”
The first version of this was two jobs: a QA agent that filed issues and an implementer that took issues off the backlog and opened PRs. No plan step, no human in the middle.
It worked, in the sense that PRs appeared. It also produced a specific and expensive failure mode: the first time I saw a questionable approach was as a finished pull request built on top of it.
An agent would decide that a crash in a state transition was best fixed by guarding a null. That’s a real change, it passes tests, the repro stops reproducing, and it is completely wrong — the null should never have been there, and now the actual bug is buried one layer deeper with a plausible-looking fix sitting on top of it.
Reviewing that costs more than writing it. I have to reconstruct the reasoning from a diff, decide it’s wrong, explain why, and throw the run away.
The plan agent exists to move that conversation earlier, to when it is still cheap. Its entire job description is one question:
How should this be resolved?
It posts the answer as a comment and stops. It has no Write or Edit tool. It cannot branch, commit, or open a PR. It literally cannot write code even if it decides it should.
That constraint is the whole design. An agent that could write code, told not to, will eventually write code — usually as a “quick spike to check whether the approach works.” Removing the tool removes the temptation.
The backlog is the state
Here’s the part I think is genuinely worth stealing.
There is no coordinator holding state between runs. The open issue list on GitHub is the state machine. Three labels track where any piece of work stands, and exactly one is present at a time:
| Label | Meaning |
|---|---|
needs-plan | Filed, not yet planned |
awaiting-human | A plan is posted, waiting on me |
plan-approved | Approved, ready to implement |
Approval is one command:
gh issue edit 214 --remove-label awaiting-human --add-label plan-approved
That’s it. That’s the entire human interface to the pipeline. No dashboard, no web app, no chat bot. I approve work from my phone in the GitHub mobile app while waiting for coffee.
No agent may ever apply plan-approved. That rule is what makes the whole thing safe to leave running. The plan agent can propose anything it likes; nothing gets built until a human moves one label.
Two lanes reach the implementer:
Planned lane blockers, majors, anything a human filed
file → needs-plan → awaiting-human → HUMAN → plan-approved → PR
Fast lane minor severity only
file → PR
The fast lane exists so the loop keeps shipping when nobody is around to approve anything. A cosmetic layout bug doesn’t need a design conversation. A blocker does.
Why this works better than a coordinator
Every agent framework I’ve looked at wants to own orchestration — a supervisor process that dispatches tasks, tracks status, and handles retries. That supervisor becomes the most complex and most fragile part of the system, and it’s the part you can’t inspect.
Using the issue tracker instead means:
- State is human-readable by default. I can look at the backlog and know exactly what every agent thinks is happening. No log spelunking.
- State is human-editable by default. If an agent gets something wrong, I fix it by editing a label. No admin endpoint required.
- Crashes are free. An agent that dies mid-run leaves the issue exactly as it found it. The next fire picks up from the labels. There’s no partial state to reconcile because there’s no state to be partial.
- Concurrency is a labeling problem. The implementer claims an issue with a
workinglabel before it reads any source, and skips anything that already has one. Two runs can’t collide because the claim is atomic and visible.
The cost is that everything has to be honest about the backlog. A stale label is a real bug, and I’ll come back to one that bit me.
The four jobs
All four are the same shell script with a different flag. Each is a single top-level claude -p session — a non-interactive one-shot. There is no subagent dispatch anywhere in the system.
./scripts/agent-run.sh # QA — exercises the app, files issues
./scripts/agent-run.sh --plan # plan — proposes an approach
./scripts/agent-run.sh --implement # implement — opens one PR
./scripts/agent-run.sh --long # soak — long-horizon run, drift check
The QA agent
Runs every two hours. Boots a dev server, drives headless Chromium through playwright-cli, and works through the application the way a user would.
Its first act on every run is regression: it re-runs the repro path of every PR merged since its last run. If a merged fix didn’t actually hold, it reopens the issue with fresh evidence. Fixes that don’t stay fixed are the failure mode this pipeline is most prone to, so it’s checked first, every time.
Then it explores. For each bug it files a GitHub issue with a severity label, a repro, expected/observed, and screenshots.
The plan agent
Runs every 30 minutes and usually exits without invoking a model at all. Before pulling the repo, the runner does a cheap gh sweep for issues needing a plan or carrying an unanswered human reply. If both lists are empty, the fire costs three API calls instead of a clone and a session.
That gate matters more than it sounds. A 30-minute cadence is 48 fires a day. Making the empty case nearly free is what lets the responsive case be that fast.
When there is work, it takes one issue — replies to humans before fresh plans, oldest first — reads the entire thread, reads the code the issue is about, and posts a plan naming actual files.
It is explicitly allowed to not know:
That framing decides how you should behave when you are unsure: ask. A question that saves a wrong implementation is worth far more than a confident plan that has to be unwound. You are the one job here that is allowed to not know.
Most agent prompts push toward confident output. This one pushes the other way, on purpose, because a question costs me 30 seconds and a wrong plan costs a 45-minute implementation run plus a review.
The implement agent
Runs every 45 minutes. Takes only plan-approved issues or unplanned minor ones, highest severity first, oldest first within a severity. Oldest-first is deliberate — newest-first starves a real blocker behind a stream of fresh cosmetic bugs.
Its order of work is fixed, and the first step is the one that does the most good:
1. Establish the starting state first. Always. Reproduce the bug with your own eyes before reading any source.
If it doesn’t reproduce, that’s the finding — comment with what you tried and apply needs-repro. Do not hunt for something to change so the run feels productive.
This single rule killed an entire class of garbage PR. Without it, an agent handed a bug report will read the code, find something that looks plausibly related, change it, and open a PR claiming a fix it never observed working. With it, the agent has a verification method before it has a hypothesis.
2–4 are the boring ones: understand before editing, make the smallest correct change, verify by re-running the original repro plus the project’s lint/typecheck/test gates.
5. Rebase before pushing. Not optional:
a PR opened today was 13 commits behind by the time anyone looked at it, and arrived conflicted
At this cadence the clone you started from is stale by the time you finish. The rebase instruction includes guidance on conflict resolution, because the conflicts here have a characteristic shape: they’re usually additive, not competing — main added an import and so did you, in the same place. The answer is keep both. Picking one side compiles fine and silently deletes working behavior.
6. Open a PR. Never merge it. Opening the PR is where the agent’s work ends. That’s the second human gate.
The soak agent
Runs overnight, on its own schedule. Where the QA agent covers correctness of the core loop in a short session, this one runs the application continuously for as long as its budget allows and records a JSON line of metrics after every iteration — read off real screens, never estimated, with null plus a note for anything unavailable.
This catches what no single session can: slow accumulation. Values that only ever climb. Collections that thin by a little every cycle. A distribution that quietly widens until it’s nonsense.
The judging rule is worth quoting because it’s the difference between a useful drift report and noise:
Iteration 1 is the baseline; there are no fixed thresholds. Compare late against early, same metric, same screen. Report both numbers and the direction, always — “68.1 → 79.4, rising in 22 of 23 iterations” is a finding; “it seems inflated” is not.
Drift alone isn’t a defect. It gets filed when it’s monotonic, large, or nonsensical.
Things that only show up when you actually run it
Every one of these came from something going wrong.
The reverse edge
The lifecycle runs backwards too: plan-approved → needs-plan.
When the implementer reproduces a bug, builds exactly the approved plan, and the original repro still fails, the plan was incomplete. Finishing the job would mean making a design decision nobody approved. So it comments with what it built, what still fails, and the specific unapproved choice — usually stated as two or three named options — and swaps the label back.
It opens no PR. A PR would claim the repro is fixed when it isn’t.
The planner then picks it up and amends the earlier plan rather than re-deriving it, because that bounce comment is the most valuable thing on the issue: it’s a run’s worth of confirmed evidence about what does and doesn’t work.
Leaving that relabel off is catastrophic in a specific way. An issue on plan-approved with no working claim and no open PR is indistinguishable from fresh approved work. The selection gate orders oldest-first — so an old bounced issue gets re-picked ahead of everything else, on every fire, forever, burning a full run each time and starving the backlog.
One issue sat in exactly that state and had to be relabelled by hand. The asymmetry that makes the routing work is small and load-bearing: needs-plan is excluded by the implementer’s gate and deliberately not excluded by the planner’s sweep.
Claim before you read
gh issue edit <n> --add-label working
The moment the agent commits to an issue — before reproducing it, before reading any source. A claim applied after the work protects nothing.
And the mirror rule, which is the one that gets skipped: release the claim if you finish without an open PR. Couldn’t reproduce, deferred as too large, decided it was already fixed, ran out of budget — remove the label as part of stopping. A stale claim quietly costs the backlog an issue forever.
One clone per job
All jobs originally shared a checkout. Every run begins by hard-resetting it:
git reset --hard origin/main && git clean -fd
Which means any job starting while another was mid-run reverted that job’s files, moved its branch pointer, and deleted its untracked work — silently, from underneath it.
Giving each job its own dev port stopped the servers contending and made this look solved. The checkout was still shared. It stayed latent only because implement runs used to be one-minute no-ops; once they started doing real work, the window opened wide.
Now each mode owns a clone and a port:
qa ~/.agent-runs/repo-qa :5173
implement ~/.agent-runs/repo-implement :5183
soak ~/.agent-runs/repo-long :5193
There’s a further subtlety I enjoyed too much: the process reaper matches the clone path as a substring of a command line. So a second concurrent implementer had to be named repo-2-implement, not repo-implement-2 — because slot 1’s opening sweep kills everything matching its own clone path, and repo-implement is a prefix of repo-implement-2. Slot 1 would decapitate slot 2 on the way in.
That’s the class of bug you get when agents are just processes. Which they are.
Deferring has to produce takeable work
The implementer has a scope gate — one run, one PR, so some issues genuinely don’t fit. Early on this was applied so conservatively that a single run declined all ten open issues and landed nothing, calling several “oversized” that were plainly one run’s work.
Two fixes. First, the gate now says what it means:
Tedious, unfamiliar, touching several files, or needing a bit of investigation first are not reasons to defer. If you are unsure, the answer is yes — start it.
Second, and more importantly: a deferral must file the bounded slices as new issues. A breakdown living in a comment is invisible to the selection rules, so every later run re-read the same issue and re-derived the same rejection forever. Filing turns analysis into takeable work.
Only a run that neither lands work nor creates takeable work has failed.
An empty queue is a healthy queue
A backlog with nothing eligible in it is a normal, correct state. It means work is queued on a human, not that the filter is broken.
This has to be said out loud in the prompt, because the temptation for an agent that lists nine open issues and finds none eligible is to conclude the filter must be wrong and take one anyway. It isn’t wrong. Report the counts by lifecycle label and stop.
The number I actually watch is how many issues are sitting on awaiting-human. When that’s non-zero, the pipeline is working correctly and the bottleneck is me — which is exactly where the bottleneck should be.
One prompt file, two consumers
The agent definitions live at .claude/agents/*.md. The scheduled runner reads the same file, strips the frontmatter, and uses it as the prompt.
So the file that defines the interactive subagent I can invoke by hand is the scheduled job. There’s no second copy to drift.
Project-specific knowledge is split out into a PLAYBOOK.md at the repo root — architecture, conventions, labels, what must never be touched. The agent definitions are deliberately generic; the PLAYBOOK is the part that changes per project. Three projects share the same agent definitions and have three different PLAYBOOKs.
Model tiering is a real cost lever
plan, implement → Opus 5 (judgment, code)
QA, soak → Sonnet (drive the app, report what you see)
The QA jobs run far more often and mostly need to follow a script carefully and describe what they saw. The two jobs that decide an approach or produce a diff get the expensive model. Pinned to explicit model IDs rather than aliases so they can’t silently drift when an alias gets repointed.
The trade-offs
This is not cheap. Implement runs are Opus with a 45-minute ceiling, firing 30+ times a day alongside 2-hourly QA runs and an overnight soak. It runs on a subscription rather than metered API, which is the only reason the economics work at all.
Review is the real bottleneck, and it moved rather than disappeared. I no longer write most of these fixes, but I do read every PR and approve every plan. That is a genuine amount of reading. The plan gate helps enormously — I catch bad approaches as a paragraph instead of a diff — but “agents write the code” does not mean “nobody reads the code.”
It only works because the project has strong gates. A pre-commit hook runs lint, typecheck, and the full unit suite. There are characterization tests that fail if output distributions shift, so a change that quietly alters behavior doesn’t sail through. Agents are explicitly forbidden from weakening a check to reach green. Without that, autonomous agents merging into a codebase is just a faster way to accumulate debt.
Coverage has holes, and you should name them. The pipeline drives a dev server in a headless browser, so anything outside that path — packaging, native shells, install flows — is untested by all of it. I decided that trade explicitly and wrote it into the PLAYBOOK, because an agent will otherwise try to build the packaged app and burn ten minutes of its run doing it.
Every agent gets exactly one turn. The session is claude -p; when the turn ends, the process exits and nothing resumes it. So the prompts have to say, in as many words:
never end a turn with work outstanding, and never say you will “check back once it finishes” — there is no one to check back, and the run simply stops where you left it.
The broader lesson
The valuable thing here isn’t any individual agent. Each one is a markdown file and a claude -p invocation. You could rebuild any of them in an afternoon.
The valuable thing is that the coordination substrate is a tool humans already use. Issues, labels, comments, pull requests. That single decision gets you observability, human override, crash recovery, and concurrency control for free, because those are all things GitHub already does.
And it forces the property that actually matters: every handoff between agents is a human-readable artifact. A plan is a comment I can argue with. A claim is a label I can remove. A rejection is an issue I can reopen. Nothing important happens in a place I can’t see.
The pattern:
autonomous agents + stateless one-shot runs + a shared human-readable queue
+ exactly one human approval gate before code is written
+ exactly one before it lands
The agents are the easy part. The queue discipline is the product.