Notes

A frontier model coordinates, cheap models execute

Field notes on an agent pipeline built on Orca: where the control plane ends, what the policy layer adds, and why an unacked Delivery looked like a lost notification.

Over four days, I ran one frontier model as coordinator and a set of cheap models as executors across two projects. They closed six epics and more than forty tasks. The test suite grew from roughly 1,100 tests to almost 2,000, and the executor showed cost estimates of $0.01 to $0.13 per task.

Orca runs the work and stores its state. I added a policy adapter to decide what each agent may touch, what evidence it must produce, and when to accept, retry, or close its work.

Two subagents once finished almost together. The coordinator handled the first, and the second appeared to vanish. An older FIFO Delivery was still waiting for an ack, so Orca kept replaying that batch while the second notification waited behind it. Acknowledging the full batch let the coordinator receive the next one.

1. What belongs in the policy adapter

Two-layer architecture. Orca owns Runs, Tasks, Dispatches, Deliveries, terminals, and worktrees. A policy adapter applies project profiles, permissions, compatibility rules, verifiers, and cleanup. The coordinator and workers use both layers without duplicating state.
Orca owns state; the pipeline decides what is allowed and what evidence is required to accept a result.

Orca already creates Runs, represents a DAG, dispatches a Task, keeps durable threads, supervises terminals, and manages worktrees. Reimplementing those features would mean keeping Orca’s Dispatch, a guard’s file, or a watcher’s PID in sync. I wanted one place to look up the state of a task.

The adapter handles decisions I can test:

Orca, control planePipeline, policy adapter
Run, DAG, Task, and Dispatchproject profile and effective concurrency limit
FIFO Delivery and durable threadscomplete resolution before ack
terminal and transcriptsafe agent command and startup proof
worktree lifecyclesetup, editable roots, and safe cleanup
worker outcomegates, QA, and retry policy

The adapter has no daemon, broker, SQLite database, NDJSON spool, or local queue. I also left Orca commands unwrapped unless a wrapper could apply or verify a policy.

2. Worktrees, specs, permissions, and verification

Every worker gets its own worktree, branch, and terminal. Workers do not share an index, build caches, or a development server. If a provider fails, I can replace the process and keep the work on disk.

The Orca Task contains the whole spec: objective, scope, criteria, paths, cases, and gates. I keep it there because a Task spec is immutable. A copy in .pipeline/specs/ could become an orphaned artifact, and dispatching a reference to an external file could leave the worker without its spec during recovery.

OpenCode starts from a default-deny base, then receives a stack layer and a profile overlay. A web worker cannot touch Electron; a security worker can edit two directories; a QA worker only writes evidence. Secrets, keys, .git/**, and .pipeline/** cannot be reopened by a later layer.

The coordinator reruns gates and reviews the diff, even when the worker reports that everything passed. I choose checks according to what the change can break. In Lawyer, the profile for a task that creates routes adds the production build and bundle verifier. In BIA, global style work adds computed CSS and Playwright. Typecheck, lint, and unit tests had passed twice while the application was visibly broken.

3. Processing a Delivery

Orca’s public mailbox has a precise contract. check returns the oldest FIFO Delivery. That Delivery may contain many messages, and Orca replays it unchanged until the coordinator acknowledges the entire batch with its deliveryId.

Mailbox cycle. Check gets the oldest Delivery; the coordinator processes every message; each completed worker is released, retained, or reused, while questions and escalations receive durable decisions; only then does it ack and check the next Delivery. If the coordinator crashes before ack, Orca replays the same Delivery and actions are idempotent.
The ack is the coordinator's commit. It comes after resolving the full batch, never after the first message.

The coordinator follows this loop:

  1. Get the oldest Delivery.
  2. Process every message in it.
  3. For each worker_done, validate Task, Dispatch, and outcome; then reuse, retain, or release the terminal.
  4. Leave a question as needs_input in its thread; block the Task on escalation.
  5. Send ack only after every decision is durable in Orca.
  6. Check again without waiting until all currently available Deliveries are empty.

If the coordinator closes between steps 3 and 5, Orca returns the same batch after restart. Actions therefore have to be idempotent: worker-release is idempotent, blocking an already blocked Task is idempotent, and reuse checks whether the next Task already has a Dispatch.

After worker_done, the Task and Dispatch are complete and the process remains idle. Sending another message to the completed Dispatch does not create a second attempt. The coordinator must reuse the terminal with another Task, retain it for debugging, or release it.

4. Notification reliability in Orca 1.4.182

Orca 1.4.182 includes the basic coordinator notice for worker_done (#12988). It does not include the retry that covers races, restarts, and stale waiters (#14332), merged after the 1.4.182 release.

I treat this version as degraded: a profile may request three workers, but its effective limit is one. Parallelism requires an explicit per-Run override, and the override is visible in the receipt.

The adapter keeps an exact compatibility table. I add the first release containing #14332 after verifying it and running a smoke test: two workers finish close together while the coordinator does not check during their work. The test exercises notification, Delivery replay, ack, and cleanup. A later version number alone is insufficient.

One coordinator terminal supervises one Run. Concurrent Runs use separate coordinators, which prevents waiters from competing over one inbox.

5. Working around the launch race

Creating a worktree, starting an agent, and injecting the task in one call can dispatch before the new agent is ready. The race is documented in #13488.

Split launch. Create a worktree, run setup, write permissions before startup, launch the agent, wait for tui-idle, dispatch to the existing terminal, and confirm a real transcript with worker-read.
The workaround ends on a public, structured signal: `worker-read`. It never tries to recognize wrapped words in the TUI.

Until a verified release fixes the race, I use separate steps:

  1. Create the worktree without an agent.
  2. Run setup.
  3. Write the OpenCode policy.
  4. Start the agent.
  5. Wait for tui-idle, then dispatch to that existing terminal.
  6. Confirm a real transcript through worker-read.

A quiet worker may still be working. Grepping the screen, measuring silence timestamps, or sending mailbox probes can mistake that state for a failure. The pipeline retries when dispatch has not started, the outcome has failed, the provider returns an error, or a gate fails.

6. Write permissions in OpenCode and Codex

OpenCode can prevent writes by path. Its policy composes as base + stack + overlay and is validated before startup. A replacement overlay may narrow the edit map, but it cannot erase non-negotiable denies.

Codex has different granularity. It starts with workspace-write, network disabled, and approvals disabled, following its official configuration. That contains the process within the workspace. Restricting edits to two directories inside that workspace needs another check: before accepting the result, the adapter compares the full diff (committed, staged, unstaged, and untracked) against the profile’s editable roots. OpenCode prevents the write; this Codex check detects it afterwards. Both can reject a result, but the exposure before rejection differs.

7. When to retry a worker

Relaunching a worker after 210 seconds of silence produces false positives. Long model turns, wrapped TUI output, and a busy coordinator can all make a healthy session look stuck when the coordinator reads and classifies its screen.

The recovery policy starts from states Orca actually knows:

  • a Dispatch never started;
  • the outcome is failed or the provider returned an error;
  • an independent gate failed;
  • an escalation blocked the Task.

After repeated failures, I split the Task, relaunch clean in the same worktree, and raise the model only if that is still necessary. This preserves the code from a broken session and lets me address an oversized task before spending more on the model.

Cleanup starts with worker-release, which archives output and releases the exact terminal; then worktree rm. If the checkout has uncommitted changes, cleanup refuses. It neither hides Orca errors nor falls back to raw Git.

8. What a worker can verify visually

An executor without vision can traverse the accessibility tree, assert text and structure, and produce screenshots. It cannot judge visual weight, contrast, or fidelity. That verdict belongs to whoever has eyes: the coordinator or the human.

The report needs to list what the worker could not judge. In one batch, a worker correctly proved that a grid had five cells and escalated that it could not know whether the empty tracks looked painted. Visual review found exactly that defect.

I also check the evidence files. Some screenshots weighed hundreds of kilobytes but were corrupt: their PNG signatures started with efbfbd50 instead of 89504e47. Checking the signature became a gate.

9. What it actually cost

These are the figures I observed between August 8 and August 11, 2026. They describe that run and are not a provider pricing table.

MetricObserved
Epics closed6
Tasks processed~40
Tests~1,100 → ~1,950, green
Executor estimate shown in the agent$0.01–$0.13 per Task
Subscription paid at the time$10/month; calls displayed $0.00
Time budget per Task45 minutes

Executor tokens cost little during this run. I spent more time dealing with ambiguous specs, checks that missed a broken interface, and incorrectly acknowledged Deliveries. Watching the wrong indicator could waste twenty minutes.

The adapter stores what a Task may touch and what evidence makes its result acceptable. Orca owns Runs, messages, Tasks, and Dispatches. That gives me an owner for each failure, a durable record of its state, and a recovery path I can follow without maintaining another copy of that state.