spec-orchestrate: A GAN-Loop Pipeline That Drives a GitHub Issue to a Ready PR on Its Own

Introduction

agent-skills, the open-source spec-driven development skill set published by ZenChAIne, has a new member: spec-orchestrate. It takes the spec skills we introduced in our practical guide series and binds them into a single conductor that drives one GitHub Issue from specification through adversarial review, implementation, acceptance testing, PR creation, and retrospective, end to end.

This article explains the design through the lens of a concept trending in agent engineering right now, the adversarial generate-and-critique loop, and shares data from the pipeline's first run on a real project.

What Was Missing From the Spec Skills?

Two things: a conductor that keeps driving the connections between skills, and a verifier that distrusts the finished work. The nine skills covered in our earlier series could already go from "write me a full spec" to a pull request, but three manual junctions remained.

1. Each hand-off was a suggestion, not a transition. spec-generator would finish and propose running spec-inspect; a human confirmed every step from spec to Issue to implementation.

2. No adversarial review of the spec itself. spec-inspect is a mechanical consistency check and spec-review targets implementation code. Nobody critically read the design, at the specification stage, with a model other than the one that wrote it.

3. No acceptance testing. spec-test generates unit tests from completion criteria, from the implementer's point of view. Nothing launched the finished app and verified it from the outside as a black box.

The update fills these gaps with three new pieces:

You can read it as an implementation of Anthropic's three-agent harness pattern (Planner / Generator / Evaluator with an external orchestrator), built on the skill mechanism instead of the Claude Agent SDK. See our introduction to harness engineering for that background.

What Is a GAN Loop?

A GAN loop is a development loop that separates the agent that produces an artifact from the agent that attacks it, and iterates until the critique passes. The name borrows from GANs (Generative Adversarial Networks), the 2014 framework by Ian Goodfellow and colleagues, often explained as a counterfeiter and an art appraiser sharpening each other: the generator learns to fool the discriminator, the discriminator learns not to be fooled.

Since late 2025, agent engineering has been borrowing this structure for development loops rather than model training. Anthropic's three-agent harness is the canonical example: an independent evaluator avoids the "grading your own homework" problem. spec-orchestrate is a lightweight take on the same idea. There is no gradient descent involved, only the adversarial back-and-forth, so we call it a GAN-lite arrangement. Two loops alternate:

• Spec loop: the spec author (Claude by default) writes the four spec documents; the adversarial reviewer (Codex by default) reads them critically and returns findings. Any Critical or Improvement finding sends the spec back
• Implementation loop: every implemented task is reviewed by the opposite model from the one that implemented it. Code written by Codex is reviewed by Claude, and vice versa

Why cross-vendor models? Because a model shares blind spots with itself. Asking the Claude that wrote a spec whether the spec has holes tends to reproduce the same holes. A model with different training data and different reasoning habits overlaps less. The CSRF catch in our first run, described below, is exactly this mechanism paying off.

This cross-model loop is also something the model vendors themselves are unlikely to ship. Anthropic's three-agent harness stays within the Claude family, and OpenAI's harness engineering is built around Codex; neither has much reason to wire a rival's model into its own tooling as a first-class critic. A vendor-neutral open-source skill set can: with nothing more than the roles in pipeline.yml and the agent-delegate public contract, Claude and Codex converse as equal members of one pipeline. Single-model configurations work too, but the freedom to pit vendors against each other is a structural advantage of spec-orchestrate.

How spec-orchestrate Is Designed

The orchestrator itself is deliberately thin. Its substance is a transition table, a stall detector, and an evidence rule.

A Thin State Machine

The pipeline advances through nine phases:

intake
  → spec_generate (four spec documents)
  → inspect (mechanical checks)
  → spec_review (adversarial spec review) ⇄ send-back
  → approval (human gate; auto mode passes through)
  → implement (delegated to spec-implement)
  → evaluate (acceptance tests) ⇄ send-back
  → pr (PR with evidence)
  → retrospective (aggregate, learn, improve)

spec-orchestrate never writes specs, code, tests, or reviews. Its SKILL.md carries the same blocking "orchestrator only" rule as spec-implement: it reads and writes the state file, dispatches workers, machine-verifies their results, and picks the next phase. Cheap mechanical checks (inspect) run before the expensive semantic review (spec_review), so a peer model's tokens are never spent on formatting defects.

Who runs each phase is resolved from roles in .specs/pipeline.yml. The default is a cross-model split: Claude authors specs, implements UI, and runs acceptance tests; Codex reviews specs and implements backend and test tasks. State lives in pipeline-state.json with a single writer, and resume-from-state is the normal path, not an exception: a multi-hour run that crashes picks up at the last recorded phase.

Stall Detection With Finding Fingerprints

The spec-review and acceptance-test loops have no hard round cap (the per-task fix loop inside the implement phase keeps its traditional three-attempt limit). Ten rounds on a hard review is healthy; what must not happen is a loop that stops making progress and spins forever. Progress is judged mechanically with finding fingerprints: a hash of the requirement ID, severity, normalized file path, section heading, and the first 80 normalized characters of the gist. A finding rephrased between rounds still hashes to the same fingerprint.

Comparing fingerprint sets across rounds yields three signals:

• S1 (recurrence): the same fingerprint survives three consecutive rounds
• S2 (non-decreasing): the Critical + Improvement count has not fallen for three rounds
• S3 (oscillation): the fingerprint set alternates between two states

Only when a signal fires does the pipeline enter arbitration. In manual mode a human chooses; in auto mode the stalled role is swapped to the opposite model (once, by default), and if the loop stalls again the run lands as a draft PR listing every unresolved finding. An unattended pipeline never presents unverified work as mergeable, and every arbitration is recorded in state and surfaced in the Issue or PR. A silently resolved stall is treated as a bug.

Evidence Over Self-Report

The acceptance-test runner, spec-evaluate, is built on one principle: an evaluator claiming "I tested it and it passed" is not trusted.

• Every test case runs by its declared method (playwright browser operation / command / file check), saving screenshots and logs to an evidence directory
• A PASS must point to evidence on disk. A PASS whose evidence file is missing or empty is rewritten to FAIL by machine verification, regardless of the report
• Each evidence manifest row's sha256 and size are recomputed and compared; a mismatch also downgrades the case to FAIL

The PR body is generated from the state and result files, never from the orchestrator's memory: the adversarial review history, a per-requirement pass/fail table, and an evidence manifest with file names, sizes, and sha256 hashes. Reviewers can confirm that no evidence was swapped after the run without any binary ever entering git history. This manifest scheme was not part of the original design: it is a refinement born from the first production run, described next.

What Happened in the First Production Run

We ran the pipeline in auto mode on an internal finance-related project, with a single GitHub Issue as input. It reached a ready (non-draft) PR with zero human input. The breakdown:

• The spec loop converged in 3 rounds. In round 1, the Codex reviewer flagged a missing CSRF protection as Critical: a defect the Claude author had not seen, of a kind no formatting check would ever catch. The spec was revised and passed the gate on round 3
• Implementation ran as 5 phases, split per the default roles: Claude on UI tasks, Codex on backend and test tasks, with every task reviewed by the opposite side
• All 13 acceptance cases passed, each backed by real-browser screenshots that survived the machine existence check (a PASS whose evidence pointer had been missing would have been forced to FAIL). The run also exposed a flaw in evidence handling: the screenshots were committed into git history, where binaries cannot be diff-reviewed and screens from a live system can carry personal data. That lesson produced the current scheme of keeping evidence local and putting only a sha256 manifest in the PR body
• The retrospective recorded its insights as observations only, declining to promote them to change proposals on the evidence of a single run: exactly the restraint it was designed to show

More than the completion itself, what we value from this run is confirmation that the pipeline stops where it should and doubts what it should. The CSRF catch shows the adversarial review is not decorative; the enforced evidence-existence check shows verification is a mechanism, not advice; the observation-only retrospective shows self-improvement does not run ahead of its data. And the evidence-handling flaw feeding straight into the sha256-manifest redesign is itself an instance of the structure this article is about: the pipeline gets better each time it runs. Trust in unattended automation accrues from failure-detection mechanisms actually firing, not from a count of happy paths.

How to Use It

Install the whole skill set:

npx skills add anyoneanderson/agent-skills -g -y

Assign roles in .specs/pipeline.yml (a template can be generated by spec-workflow-init; the pipeline also runs with built-in defaults if the file is absent). Roles are set per phase, and implementation further per task kind (UI / backend / test), so you can field each model where it is strongest: Claude on UI and interactive spec work, Codex on reviews, backend, and tests. The default cross-model configuration is arranged on exactly that logic:

roles:
  spec_author: claude
  spec_reviewer: codex
  impl_ui: claude
  impl_backend: codex
  impl_test: codex
  e2e_runner: claude
app:
  start: "npm run dev"
  url: "http://localhost:3000"
  ready_pattern: "ready in"

If you do not have Codex CLI, set every role to claude for a lightweight single-vendor configuration. Generator and critic still run as separate sessions, but expect a coarser net: same model, shared blind spots. If a codex role is configured and the CLI is missing, manual mode asks before reassigning to claude; auto mode reassigns automatically and records the override in state and in the PR body.

Two execution modes:

# manual: shape the spec in dialogue, approve it once, walk away
/spec-orchestrate --mode manual
 
# auto: hand over an Issue, come back to a PR
/spec-orchestrate --issue 42
 
# resume an interrupted run from its state file
/spec-orchestrate --resume

The only human gate in manual mode is spec approval. In auto mode there is none: an Issue goes in, a PR comes out, and a ready PR carries the claim that acceptance testing passed.

What Does the Retrospective Fix on Its Own?

The final phase aggregates the run records (round counts, blocker categories, stalls) and proposes improvements to the skill instructions themselves, but under a tiered permission model. Proposals must be backed by aggregated counts; insights without frequency evidence are recorded as observations and left alone.

Even Tier 1 goes through a branch and PR rather than a direct push, so a bad edit is one revert away. Guardrails run before any auto-merge: path normalization rejects ../ and symlink tricks that would disguise a Tier 2 file as Tier 1, and a line budget rejects edits that only ever add instructions (LLM self-improvement skews toward accretion).

Most automation peaks in quality the day it ships and drifts out of alignment from then on. spec-orchestrate points the other way: every run is machine-aggregated, and only improvements backed by frequency evidence flow back into the skills themselves. As with the evidence-handling redesign after the first run, the lessons of a failure become the spec of the next run. The biggest difference from one-shot agent execution is this property of getting better the more it runs.

We read this permission tiering as a microcosm of a broader question: which commit rights to grant an AI. Instructions that shape the agent's own behavior can be self-amended on evidence; contracts that other components depend on, and the executable surface, stay with humans.

FAQ

Q. How does spec-orchestrate differ from spec-implement?

A. spec-implement orchestrates the implementation segment, from an Issue to a PR. spec-orchestrate conducts the whole span around it: spec generation, adversarial spec review, human approval, acceptance testing, and retrospective. In its implement phase it calls spec-implement as a worker.

Q. Can I use it without Codex CLI?

A. Yes. Either set all roles to claude, or leave the codex roles in place: when the CLI is absent, the pipeline reassigns to claude (with confirmation in manual mode, automatically in auto mode) and records the override. You lose the cross-vendor blind-spot reduction, though.

Q. Won't the review loop run forever?

A. The spec-review and acceptance-test loops have no round cap, but they always terminate: finding fingerprints detect recurrence, non-decreasing counts, and oscillation, after which arbitration swaps the role once and, failing that, lands a draft PR with the unresolved findings listed. The per-task fix loop inside the implement phase keeps its three-attempt limit.

Q. Couldn't Anthropic's or OpenAI's own tooling do the same?

A. Division of labor within a single model family, yes. But wiring a rival vendor's model in as a first-class adversarial reviewer is not something vendor-native tooling is likely to offer. As a vendor-neutral open-source project, spec-orchestrate pits Claude and Codex against each other in one pipeline, lets you cast each model by its strengths (Codex on reviews, Claude on UI), and closes the loop by improving itself from its own run records.

Q. Should I merge an auto-mode ready PR as-is?

A. A ready PR is machine-backed: all acceptance cases passed and the evidence manifest is verifiable. We still recommend a human code review as the merge gate. The pipeline compresses verification effort, not merge responsibility.

Summary

With spec-orchestrate, spec-evaluate, and agent-delegate added, the nine spec skills stop being a chain of suggestions and become a pipeline that drives itself, with evidence. The core ideas: an adversarial cross-model loop (Claude generates, Codex critiques), fingerprints that detect only the loops that stop progressing, machine verification that refuses self-reported success, and self-improvement whose permissions scale with blast radius. The cross-vendor adversarial loop, the freedom to cast each model by its strengths, and the self-improving retrospective are the strengths we believe single-vendor native tooling can least easily replicate.

The first production run demonstrated not just that the pipeline can finish, but that its doubting mechanisms fire on real work: a CSRF gap caught at spec stage, 13 passing cases backed by machine-checked evidence, an evidence-handling lesson that fed straight back into the pipeline's design, and a retrospective that knew one run is not enough data. ZenChAIne continues to apply agent-skills to production projects and welcomes feedback and pull requests.

Repository: github.com/anyoneanderson/agent-skills

Introduction to Harness EngineeringHow rules, skills, hooks, memory, and feedback loops raise AI agent output quality, including Anthropic's three-agent (GAN-style) harness pattern.zench-aine.io

References

• agent-skills - GitHub
• spec-orchestrate SKILL.md
• spec-evaluate SKILL.md
• Generative Adversarial Networks (Goodfellow et al., 2014) - arXiv
• Anthropic — Effective harnesses for long-running agents
• Anthropic — Harness design for long-running application development