Skills / Go-to-market / grade-iterate

grade-iterate

Phase 3 of building a Claude Managed Agent — the bounded grade→iterate loop. Define a CMA outcome (a required markdown rubric graded by an isolated grader), read each verdict, decide the next move (sharpen / re-run / promote to schedule), and once a version passes, run held-back eval cases in parallel. Use when the user says "grade my agent", "make it pass the rubric", "iterate until it's good", "is it good enough", or when the orchestrator routes phase=grade-iterate. outcome_builder.py builds the user.define_outcome payload (rubric required, max_iterations clamped 1..20 — never unbounded); verdict_reader.py reads the grader result and recommends the next move; eval_scaffold.py generates held-back cases + a parallel run plan (capped at the 25-thread CMA ceiling). Distinct from stage-launch (first launch) and run-without-you (scheduling).

alirezarezvaniIndexed from GitHubFor Founders

By alirezarezvani on GitHub, licence MIT. Redistributed unchanged with its licence and attribution, from the source at commit 19392f7, the version we scanned. The description is ours.

SecurityReview before userisk 7 of 100 · SkillSpector (static rules)
Rating—0 verified reviews
Uses this month00 installs
Job done—reported after each use

About this skill

Summary from the skill's own text
Use it whenThis is the plugin's **loop**: CMA's `outcome` primitive self-grades the agent's work in an isolated context and feeds failing verdicts back for the next attempt. It is **always bounded** by `max_iterations` (1..20) — never "improve forever".
You provideSee the skill's instructions.
You receiveSee the skill's instructions.
It needsRuns 3 scripts: scripts/eval_scaffold.py, scripts/outcome_builder.py, scripts/verdict_reader.py.
It won'tNot stated by the author.
Try asking“Use the grade-iterate skill.”

What's inside

Every file in version 1.0.0, exactly as you will install it
Instructions 1Scripts 3References 37 files · 18.0 KB
SKILL.mdInstructions · 74 lines · 4.1 KB
medium · Skill declares no tool scope ('permissions' or 'allowed-tools') but code capabilities were detected: file_read, file_write. · line 1
Without declared permissions the skill's intent is opaque and cannot be validated.

Phase 3 — Grade → Iterate (the bounded loop)

This is the plugin's loop: CMA's outcome primitive self-grades the agent's work in an isolated context and feeds failing verdicts back for the next attempt. It is always bounded by max_iterations (1..20) — never "improve forever".

See [../../references/loops-and-workflows.md](../../references/loops-and-workflows.md) and the outcome section of [../../references/cma-primitives.md](../../references/cma-primitives.md).

Workflow

  1. Define the outcome.

``bash python3 scripts/outcome_builder.py \ --sheet ./my-agent/build-sheet.json --max-iterations 5 \ --out ./my-agent/payloads/outcome.json ` The **rubric is required**; max_iterations is clamped to 1..20. Send the payload as a user.define_outcome` event (append to the running session).

  1. Read every verdict first.

``bash python3 scripts/verdict_reader.py --result ./my-agent/last-verdict.json ` Tables the rubric outcome and recommends: **SHIP** (satisfied), **SHARPEN** then re-run (needs_revision), **ESCALATE** (max_iterations_reached / failed), **RESUME** (interrupted`). With ≤1 iteration left it flips to "make the single highest-value fix or escalate now".

  1. Loop invariant. Each iteration must move ≥1 rubric line fail→pass, or the

run halts at the cap and escalates. Don't burn the budget on cosmetic edits.

  1. Once a version passes, run held-back eval.

``bash python3 scripts/eval_scaffold.py \ --sheet ./my-agent/build-sheet.json --out ./my-agent/eval.json --concurrency 5 `` Held-back cases (never seen during iteration) run in parallel, capped at the 25-thread CMA ceiling, each graded against the same rubric.

  1. Decide. SHIP as v0, or promote to a scheduled deployment (Phase 4). Record

the verdict on the goal: goal_state.py set --phase run-without-you.

Hard rules

  • Bounded, always. No outcome without a max_iterations cap.
  • Read the verdict before acting. The grader's explanation drives the next move.
  • Held-back cases are held back. Never grade generalization on cases the agent

already iterated against.

Forcing-question library (recommend + cite)

  1. "What are the 3–5 rubric lines?" *Recommend:* grounded, checkable criteria.

*Cite:* cma-primitives.md (rubric required).

  1. "How many iterations before you'd rather look yourself?" *Recommend:* 3–5.

*Cite:* loops-and-workflows.md (bounded loop).

  1. "On a fail, sharpen the prompt or the tools?" *Recommend:* whichever rubric line

failed points to. *Cite:* verdict_reader next-move table.

  1. "Which cases did the agent NOT see?" *Recommend:* hold back ≥3 for generalization.

*Cite:* this SKILL (held-back eval).

Tools

  • scripts/outcome_builder.py — user.define_outcome payload (rubric required, cap 1..20).
  • scripts/verdict_reader.py — grader result → next move.
  • scripts/eval_scaffold.py — held-back cases + parallel run plan (≤25 threads).

Reviews

Only from people who installed it

No reviews yet.