Claude-Code-Game-Studios/CCGS Skill Testing Framework/skills/utility/skill-test.md

# Skill Test Spec: /skill-test

## Skill Summary

`/skill-test` validates skills (`.claude/skills/*/SKILL.md`) and agents
(`.claude/agents/*.md`). It has four modes:

- **static** `[name | all]`: skills only. Runs 7 structural checks (frontmatter,
  phases, verdict keywords, ask-before-write language, next-step handoff, fork
  complexity, argument hint) and reports each as PASS / WARN / FAIL. Per-skill
  results are COMPLIANT, WARNINGS, NON-COMPLIANT or NOT ASSESSED. Writes nothing.
- **spec** `[skill-or-agent-name]`: resolves the name through
  `CCGS Skill Testing Framework/catalog.yaml`, reads the file and the spec at the
  entry's `spec:` path, plus `CLAUDE.md` and every file it imports with `@path`,
  and marks each assertion — the spec's Static Assertions included, for a skill
  as for an agent — PASS / PARTIAL / FAIL / NOT ASSESSED. Case and Overall
  Verdicts are the worst result beneath them (FAIL, PARTIAL, NOT ASSESSED,
  PASS). It then asks "May I write these results to
  `CCGS Skill Testing Framework/results/…` and update `catalog.yaml`?"
- **category** `[name | all]`: scores the skill or agent against its section of
  `CCGS Skill Testing Framework/quality-rubric.md` (PASS / FAIL / WARN per
  metric) and asks "May I update `catalog.yaml` …?" before recording it.
- **audit** (also the no-argument default): coverage table of every skill and
  every catalogued agent. Writes nothing and issues no verdict.

No director gates apply.

---

## Static Assertions (Structural)

Checked against the SKILL.md by `/skill-test spec` — no fixture needed.

- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
- [ ] Has ≥2 phase headings
- [ ] Contains verdicts: COMPLIANT, WARNINGS, NON-COMPLIANT, NOT ASSESSED (static mode); PASS, PARTIAL, FAIL, NOT ASSESSED (spec mode)
- [ ] Contains "May I write" before writing spec results, and "May I update" before recording a category check in `catalog.yaml`
- [ ] Has a next-step handoff (Phase 3: contextual follow-up after each mode)

---

## Director Gate Checks

None. `/skill-test` is a meta-utility skill. No director gates apply.

---

## Test Cases

### Case 1: Static Mode — Well-formed skill, all 7 checks pass, COMPLIANT

**Fixture:**
- `.claude/skills/brainstorm/SKILL.md` exists and satisfies all 7 checks:
  - Has all 5 required frontmatter fields
  - Has ≥2 phase headings
  - Has verdict keywords
  - Has "May I write" language
  - Ends with a next-step handoff
  - Does not set `context: fork`
  - `argument-hint` matches the modes its Parse Arguments phase documents

**Input:** `/skill-test static brainstorm`

**Expected behavior:**
1. Skill reads `.claude/skills/brainstorm/SKILL.md` fully
2. Skill runs all 7 structural checks
3. Output is the single-skill block `=== Skill Static Check: /brainstorm ===`
   listing Check 1 – Check 7, each PASS
4. Verdict is COMPLIANT
5. Phase 3 follow-up suggests `/skill-test spec brainstorm`

**Assertions:**
- [ ] Exactly 7 structural checks are reported
- [ ] All 7 are marked PASS
- [ ] Verdict is COMPLIANT
- [ ] No files are written and no "May I write" is asked

---

### Case 2: Static Mode — Skill Missing Ask-Before-Write Despite Write Tool in allowed-tools

**Fixture:**
- `.claude/skills/some-skill/SKILL.md` has `Write` in `allowed-tools` frontmatter
- The body has no ask-before-write language in any form Check 4 accepts:
  no "May I write", no "before writing"/"approval" near a write instruction,
  no "ask" and "write" in the same section

**Input:** `/skill-test static some-skill`

**Expected behavior:**
1. Skill reads `some-skill/SKILL.md`
2. Check 4 (collaborative protocol) is FAIL — `Write` is in `allowed-tools` and
   no ask-before-write language is found
3. The other six checks are listed with their own results
4. Verdict is NON-COMPLIANT, with a Recommended line naming the fix

**Assertions:**
- [ ] Check 4 is marked FAIL (not WARN — Write is in allowed-tools)
- [ ] The Check 4 line or the Recommended line names the mismatch (Write tool without ask-before-write language)
- [ ] Verdict is NON-COMPLIANT
- [ ] Other checks are shown (not only the failure)

---

### Case 3: Spec Mode — gate-check Skill Evaluated Against Its Spec

**Fixture:**
- `catalog.yaml` lists `gate-check` under `skills:` with
  `spec: CCGS Skill Testing Framework/skills/gate/gate-check.md`
- That spec has 5 test cases and a Static Assertions list;
  `.claude/skills/gate-check/SKILL.md` exists
- Suppose one of the spec's assertions reads "Balance values link to their
  source formula or rationale" — met only by `.claude/docs/coding-standards.md`
  (imported by `CLAUDE.md`), not by the skill or its gate files themselves
- The user declines the results write

**Input:** `/skill-test spec gate-check`

**Expected behavior:**
1. Skill resolves `gate-check` as a skill via the catalog and takes the spec
   path from its `spec:` field
2. Skill reads the skill, the spec, `CLAUDE.md` and its `@`-imported files
3. The spec's Static Assertions are checked against the SKILL.md in a
   `Static Assertions:` block before Case 1
4. Each assertion is marked PASS / PARTIAL / FAIL / NOT ASSESSED, with a Reason
   for each non-PASS; the balance-values assertion is PASS, citing
   `coding-standards.md`
5. Each case gets a Case Verdict; the report ends with an Overall Verdict, the
   worst result across the static assertions, the cases and Protocol Compliance
6. Skill asks "May I write these results to
   `CCGS Skill Testing Framework/results/skill-test-spec-gate-check-[date].md`
   and update `CCGS Skill Testing Framework/catalog.yaml`?"; on decline nothing is written

**Assertions:**
- [ ] The spec path comes from the catalog `spec:` field
- [ ] The spec's Static Assertions are evaluated against the SKILL.md, not skipped as "static mode's job"
- [ ] All 5 test cases are evaluated, each with its own Case Verdict
- [ ] The balance-values assertion is PASS, and the report cites `coding-standards.md` as the file it came from
- [ ] Overall verdict is the worst result — FAIL, PARTIAL, NOT ASSESSED or PASS; an assertion-level NOT ASSESSED keeps it from PASS
- [ ] No file is written without approval (declined here, so nothing is written)

---

### Case 4: Audit Mode — Coverage Table of All Skills and Agents

**Fixture:**
- `.claude/skills/` contains 74 skill directories; 73 have a spec at the path
  their catalog entry names
- `catalog.yaml` has an `agents:` section with 49 entries, all with specs

**Input:** `/skill-test audit` (and, separately, `/skill-test` with no argument)

**Expected behavior:**
1. Both inputs run audit mode — no argument is the audit default, not a usage
   message
2. Skill globs `.claude/skills/*/SKILL.md` for skills and reads the catalog's
   `agents:` section for agents
3. For each skill: Has Spec (from the catalog `spec:` path, or a glob under
   `CCGS Skill Testing Framework/skills/`), last static/category dates and
   results, category, priority
4. Separate SKILLS and AGENTS tables, a Top 5 Priority Gaps list, and the lines
   `Skill coverage: 73/74 specs (99%)` and `Agent coverage: 49/49 specs (100%)`
5. No verdict line; the output ends with the offer to run `static all`,
   `category all` or `spec [name]`

**Assertions:**
- [ ] `/skill-test` with no argument produces this audit, not usage text
- [ ] All skill directories are enumerated (not just a sample)
- [ ] "Has Spec" column is accurate for each entry
- [ ] Summary coverage counts are correct for skills and for agents
- [ ] No files are written and no verdict is issued in audit mode

---

### Case 5: Category Mode — Gate Skill Evaluated Against Quality Rubric

**Fixture:**
- `CCGS Skill Testing Framework/quality-rubric.md` has a `### gate` section with
  metrics G1–G5
- `catalog.yaml` gives `gate-check` `category: gate`
- `.claude/skills/gate-check/SKILL.md` meets every metric except G2 — Director
  panel width (only CD-PHASE-GATE is spawned at `workflow: full`)
- The user approves the catalog update

**Input:** `/skill-test category gate-check`

**Expected behavior:**
1. Skill reads the `category:` from the catalog, then the `### gate` section of the rubric
2. Skill reads the skill and the `CLAUDE.md` context, and evaluates G1–G5
3. Each metric is marked PASS, FAIL or WARN; G2 is FAIL with a quoted Gap line
4. Report ends `Verdict: FAIL (1 failure, 0 warnings)` and a Fix line
5. Skill asks "May I update `CCGS Skill Testing Framework/catalog.yaml` to record
   this category check (`last_category`, `last_category_result`) for gate-check?"

**Assertions:**
- [ ] All gate metrics (G1–G5) from quality-rubric.md are evaluated
- [ ] Each metric has an individual PASS / FAIL / WARN result, with a Gap line for G2
- [ ] Overall verdict reflects the metric results
- [ ] The catalog is updated only after the "May I update" approval

---

### Case 6: Spec Mode — Agent Name Resolves to the Agent

**Fixture:**
- `catalog.yaml` lists `creative-director` under `agents:` with
  `spec: CCGS Skill Testing Framework/agents/directors/creative-director.md`
- `.claude/agents/creative-director.md` declares `model: opus`; there is no
  `.claude/skills/creative-director/`
- The spec asserts "Model tier is `opus`"
- Suppose one spec case is a redirect: the agent is asked to pick the project's
  engine. The agent's "What This Agent Must NOT Do" list says "Make engine or
  architecture choices (delegate to technical-director)"; it never restates the
  redirect in its own words

**Input:** `/skill-test spec creative-director`

**Expected behavior:**
1. Skill resolves the name through the catalog's `agents:` section — it does not
   report "Skill 'creative-director' not found"
2. Skill reads the agent file and the agent spec
3. Static assertions are compared exactly with the frontmatter: "Model tier is
   `opus`" PASSES against `model: opus`, and would FAIL (not PARTIAL) against
   `model: inherit`
4. Test cases are evaluated against the agent's instructions; Protocol
   Compliance uses the spec's own list
5. The redirect case is met: the loaded `coordination-rules.md` rule 2 covers
   it and the agent's must-not list names technical-director — the report cites
   both
6. Report header is `=== Agent Spec Test: creative-director ===`

**Assertions:**
- [ ] The agent is resolved and evaluated, not refused as a missing skill
- [ ] A literal-value assertion is an exact comparison (near match = FAIL)
- [ ] Every case and the spec's Protocol Compliance list are evaluated
- [ ] The redirect case passes on `coordination-rules.md` plus the agent's own must-not list, both cited — the agent is not required to restate it
- [ ] No files are written without approval

---

### Case 7: Static All — an unreadable skill is NOT ASSESSED and counted

**Fixture:**
- `.claude/skills/` holds 74 skill directories; one `SKILL.md` cannot be
  read (the Read call fails — permission denied), so none of its checks can
  start. (A file that is read but whose frontmatter does not parse is a
  different case: Check 1 fails and it is NON-COMPLIANT.)

**Input:** `/skill-test static all`

**Expected behavior:**
1. The 73 readable skills are checked and listed with their Result
2. The unreadable skill appears in the table as `NOT ASSESSED`, with the
   reason — not omitted, not COMPLIANT
3. The header states the denominator, `73 of 74 skills checked`, and the
   Summary line counts the NOT ASSESSED skill

**Assertions:**
- [ ] The unreadable skill's Result is NOT ASSESSED, with a reason
- [ ] The header or summary reads `73 of 74 skills checked`
- [ ] The Summary counts sum to the skills in the table

---

### Case 8: Spec Mode — an unevaluable assertion makes the run NOT ASSESSED

**Fixture:**
- `.claude/skills/brainstorm/SKILL.md` and its spec exist
- Every assertion in the spec passes except one, which names a fixture file
  the spec never defines
- The user approves the results write

**Input:** `/skill-test spec brainstorm`

**Expected behavior:**
1. That assertion is NOT ASSESSED, with a Reason — never resolved to PASS
   because the skill "probably" handles it
2. Its Case Verdict is NOT ASSESSED; with no PARTIAL or FAIL anywhere, the
   Overall Verdict is NOT ASSESSED
3. On approval, `catalog.yaml` records `last_spec_result: NOT ASSESSED`

**Assertions:**
- [ ] The unevaluable assertion is NOT ASSESSED, not PASS
- [ ] Overall Verdict is NOT ASSESSED, not PASS
- [ ] The catalog records `NOT ASSESSED` as the result

---

### Case 9: Category Mode — no rubric section for the category

**Fixture:**
- `catalog.yaml` gives `some-skill` `category: tooling` — a category added to
  the catalog before its rubric section was written
- `CCGS Skill Testing Framework/quality-rubric.md` has no `tooling` section

**Input:** `/skill-test category some-skill`

**Expected behavior:**
1. Skill reads the category, then looks for its rubric section and finds none
2. The result is NOT ASSESSED, naming the missing `tooling` section
3. No other category's metrics are borrowed to fill the gap — not even
   `utility`'s

**Assertions:**
- [ ] The result is NOT ASSESSED and names the missing rubric section
- [ ] No metric from another category is evaluated in its place

---

## Protocol Compliance

- [ ] Static mode checks exactly 7 structural assertions
- [ ] Spec mode evaluates each test case from the spec file individually, and a skill spec's Static Assertions against the SKILL.md
- [ ] Each mode can report NOT ASSESSED — per skill in static mode, per assertion and overall in spec mode, and for a missing rubric section in category mode
- [ ] Audit mode covers all skills AND agents (not just one category)
- [ ] Spec and category modes accept an agent name as well as a skill name
- [ ] Category mode reads quality-rubric.md to get criteria (not hardcoded)
- [ ] Static and audit modes write nothing; spec and category modes write only after "May I write" / "May I update" approval
- [ ] After a spec FAIL, recommends reviewing the failing assertions and updating the skill or agent, or the spec

---

## Coverage Notes

- The skill-test skill is self-referential (it can test itself). The static
  mode case for skill-test's own SKILL.md is not separately fixture-tested to
  avoid infinite recursion in test design.
- The specific 7 structural checks are defined in the skill body; only Check 4
  (ask-before-write) is individually tested here because it has the most
  nuanced logic. The single-skill NOT ASSESSED verdict follows Case 7's
  per-skill rule; not separately tested.
- Audit mode counts are approximate — the exact number of skills and agents will
  change as the system grows; the fixture's numbers are illustrative.
- A spec case that names a director gate ID has the agent evaluated against
  `.claude/docs/director-gates/[gate-id].md`; not separately fixture-tested.