mirror of
https://github.com/Donchitos/Claude-Code-Game-Studios.git
synced 2026-10-16 01:37:05 +00:00
Release 1.1.2 — the default path works end to end
Fix release for 1.1.1: the minimal path end to end, engine commands that run on Godot, Unity and Unreal, hook warnings you can see, and verdicts that say when they could not check. Details in CHANGELOG.md; upgrade steps in UPGRADING.md (v1.1.1 → v1.1.2). Fixes #67 Fixes #131 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WuJn2FdynEybfL5KC697nJ
This commit is contained in:
Donchitos
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
7ed2c3e9c4
commit
b21fa0f7f2
@@ -8,6 +8,20 @@ skill pipeline. It classifies every gap by severity (BLOCKING / HIGH / MEDIUM /
|
||||
composes a numbered, ordered migration plan, and writes it to `docs/adoption-plan-[date].md`
|
||||
after explicit user approval via `AskUserQuestion`.
|
||||
|
||||
The resolved `workflow` tier scopes the audit: `full` requires all 8 GDD sections,
|
||||
`standard` requires 5 (+ Formulas for numeric systems), and `minimal` — the
|
||||
default, since `modes.rigor` defaults to `minimal` — audits `design/game-brief.md`
|
||||
instead and checks any existing GDD only advisorily, and the plan prescribes no
|
||||
infrastructure bootstrap (`/architecture-review`, `/create-control-manifest`,
|
||||
`/sprint-plan`, `/gate-check` are not on the minimal path). Fixtures that exercise the
|
||||
GDD and ADR audits therefore set `modes.rigor: full` in `project.yaml`.
|
||||
|
||||
A v1.0 project (no `project.yaml`, legacy config with real values) is migrated
|
||||
only by `.claude/scripts/migrate-v1-config.sh` — `--dry-run` first, the real run
|
||||
after an ask naming the files it writes, and `--finalize` left until the user
|
||||
has read the migration report. `/adopt` never writes a review mode; it reports
|
||||
the resolved one.
|
||||
|
||||
This skill is distinct from `/project-stage-detect` (which checks what exists).
|
||||
`/adopt` checks whether what exists will actually work with the template's skills.
|
||||
|
||||
@@ -17,7 +31,7 @@ No director gates apply. The skill does NOT invoke any director agents.
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
@@ -35,43 +49,53 @@ None. `/adopt` is a brownfield audit utility. No director gates apply.
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — All GDDs compliant, no gaps, COMPLIANT
|
||||
### Case 1: Happy Path — All GDDs compliant, no BLOCKING or HIGH gaps
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` sets `modes.rigor: full`, `project.stage`, and `engine.name`,
|
||||
`engine.version`, `engine.language`, `engine.rendering`, `engine.physics`
|
||||
- `design/gdd/` contains 3 GDD files; each has all 8 required sections with content
|
||||
- `docs/architecture/adr-0001.md` exists with `## Status`, `## Engine Compatibility`,
|
||||
and all other required sections
|
||||
- `production/stage.txt` exists
|
||||
and a valid `> **Status**:` field
|
||||
- `docs/architecture/adr-0001.md` exists with `## Status`, `## ADR Dependencies`,
|
||||
`## Engine Compatibility`, `## GDD Requirements Addressed` and `## Performance Implications`
|
||||
- `docs/architecture/tr-registry.yaml` and `docs/architecture/control-manifest.md` exist
|
||||
- Engine configured in `technical-preferences.md`
|
||||
- `docs/engine-reference/[engine]/VERSION.md` exists
|
||||
|
||||
**Input:** `/adopt`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill emits "Scanning project artifacts..." then reads all artifacts silently
|
||||
2. Reports detected phase, GDD count, ADR count, story count
|
||||
3. Phase 2 audit: all 3 GDDs have all 8 sections, Status field present and valid
|
||||
4. ADR audit: all required sections present
|
||||
5. Infrastructure audit: all critical files exist
|
||||
6. Phase 3: zero BLOCKING, zero HIGH, zero MEDIUM, zero LOW gaps
|
||||
7. Summary reports: "No blocking gaps — this project is template-compatible"
|
||||
8. Uses `AskUserQuestion` to ask about writing the plan; user selects write
|
||||
9. Adoption plan is written to `docs/adoption-plan-[date].md`
|
||||
10. Phase 7 offers next action: no blocking gaps, offers options for next steps
|
||||
3. Phase 2 audit: `gdd-structure-check.sh` is run once per GDD path; all 3 GDDs
|
||||
have all 8 sections and a valid Status field
|
||||
4. ADR audit: all critical sections present
|
||||
5. Infrastructure audit: registry, manifest and engine reference all exist
|
||||
6. Phase 3: zero BLOCKING and zero HIGH gaps — the project is reported
|
||||
template-compatible with only advisory improvements remaining (any MEDIUM/LOW
|
||||
item, e.g. a missing `sprint-status.yaml`, is listed as advisory)
|
||||
7. Uses `AskUserQuestion` to ask about writing the plan; user selects write
|
||||
8. Adoption plan is written to `docs/adoption-plan-[date].md`
|
||||
9. Phase 6b reports the resolved review mode ("Review mode resolves to `full` —
|
||||
from `modes.rigor`, unless something pins it") and writes nothing for it
|
||||
10. Phase 7 takes the "no BLOCKING or HIGH" branch: "No blocking gaps — this project
|
||||
is template-compatible. What next?"
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Skill reads silently before presenting any output
|
||||
- [ ] "Scanning project artifacts..." appears before the silent read phase
|
||||
- [ ] Gap counts show 0 BLOCKING, 0 HIGH, 0 MEDIUM (or only LOW)
|
||||
- [ ] `gdd-structure-check.sh` is invoked with an explicit path per GDD (not bare)
|
||||
- [ ] Gap counts show `BLOCKING: 0` and `HIGH: 0`, and the project is reported template-compatible
|
||||
- [ ] `AskUserQuestion` is used before writing the adoption plan
|
||||
- [ ] Adoption plan file is written to `docs/adoption-plan-[date].md`
|
||||
- [ ] Phase 7 offers a specific next action (not just a list)
|
||||
- [ ] `modes.review_mode` is not written to `project.yaml` and `production/review-mode.txt` is not created — the review mode is reported, never pinned
|
||||
- [ ] Phase 7 asks "No blocking gaps — this project is template-compatible. What next?" via `AskUserQuestion`
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Non-Compliant Documents — GDDs missing sections, NEEDS MIGRATION
|
||||
### Case 2: Non-Compliant Documents — GDDs missing sections, BLOCKING and HIGH gaps
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` sets `modes.rigor: full` and configures the engine
|
||||
- `design/gdd/` contains 2 GDD files:
|
||||
- `combat.md` — missing `## Acceptance Criteria` and `## Formulas` sections
|
||||
- `movement.md` — all 8 sections present
|
||||
@@ -112,32 +136,34 @@ None. `/adopt` is a brownfield audit utility. No director gates apply.
|
||||
### Case 3: Mixed State — Some docs compliant, some not, partial report
|
||||
|
||||
**Fixture:**
|
||||
- 4 GDD files: 2 fully compliant, 2 with gaps (one missing Tuning Knobs, one missing Edge Cases)
|
||||
- `project.yaml` sets `modes.rigor: full` and configures engine, naming and performance
|
||||
- 4 GDD files: 2 fully compliant, 2 with gaps (one missing Tuning Knobs, one missing Formulas)
|
||||
- ADRs: 3 files — 2 compliant, 1 missing `## ADR Dependencies`
|
||||
- Stories: 5 files — 3 have TR-ID references, 2 do not
|
||||
- Infrastructure: all critical files present; `technical-preferences.md` fully configured
|
||||
- Infrastructure: all critical files present
|
||||
|
||||
**Input:** `/adopt`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill audits all artifact types
|
||||
2. Audit summary shows totals: "4 GDDs (2 fully compliant, 2 with gaps); 3 ADRs
|
||||
(2 fully compliant, 1 with gaps); 5 stories (3 with TR-IDs, 2 without)"
|
||||
2. Adoption Audit Summary shows `GDDs audited: 4 (2 fully compliant, 2 with gaps)`,
|
||||
`ADRs audited: 3 (2 fully compliant, 1 with gaps)` and `Stories audited: 5`
|
||||
3. Gap classification:
|
||||
- No BLOCKING gaps
|
||||
- HIGH: 1 ADR missing `## ADR Dependencies`
|
||||
- MEDIUM: 2 GDDs with missing sections; 2 stories missing TR-IDs
|
||||
- LOW: none
|
||||
4. Migration plan lists HIGH gap first, then MEDIUM gaps in order
|
||||
5. Note included: "Existing stories continue to work — do not regenerate stories
|
||||
- MEDIUM: 2 GDDs with missing sections (Tuning Knobs, Formulas); 2 stories missing TR-IDs
|
||||
4. Gap Preview has no BLOCKING bullets and shows HIGH / MEDIUM as counts
|
||||
5. Migration plan lists the HIGH gap first, then MEDIUM gaps with GDD gaps before story gaps
|
||||
6. Note included: "Existing stories continue to work — do not regenerate stories
|
||||
that are in progress or done"
|
||||
6. `AskUserQuestion` to write plan; writes after approval
|
||||
7. `AskUserQuestion` to write plan; writes after approval
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Per-artifact compliance tallies are shown (N compliant, M with gaps)
|
||||
- [ ] Summary shows GDD and ADR tallies as `N (X fully compliant, Y with gaps)` and `Stories audited: 5`
|
||||
- [ ] Summary shows `BLOCKING: 0` and the Gap Preview lists no BLOCKING bullets
|
||||
- [ ] The 2 stories without TR-IDs are counted as MEDIUM gaps
|
||||
- [ ] Existing story compatibility note is included in the plan
|
||||
- [ ] No BLOCKING gaps results in no BLOCKING section in migration plan
|
||||
- [ ] HIGH gap precedes MEDIUM gaps in plan ordering
|
||||
- [ ] HIGH gap precedes MEDIUM gaps; MEDIUM GDD gaps precede MEDIUM story gaps
|
||||
- [ ] `AskUserQuestion` is used before writing
|
||||
|
||||
---
|
||||
@@ -146,9 +172,9 @@ None. `/adopt` is a brownfield audit utility. No director gates apply.
|
||||
|
||||
**Fixture:**
|
||||
- Repository has no files in `design/gdd/`, `docs/architecture/`, `production/epics/`
|
||||
- `production/stage.txt` does not exist
|
||||
- `src/` directory does not exist or has fewer than 10 files
|
||||
- No game-concept.md, no systems-index.md
|
||||
- No `project.stage` in `project.yaml` and no `production/stage.txt`
|
||||
- No source code: none of `src/`, `Assets/` or `Source/` exists
|
||||
- No game-concept.md, no game-brief.md, no systems-index.md
|
||||
|
||||
**Input:** `/adopt`
|
||||
|
||||
@@ -169,26 +195,70 @@ None. `/adopt` is a brownfield audit utility. No director gates apply.
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; adopt is a utility audit skill
|
||||
### Case 5: Director Gate Check — No gate; minimal tier scopes the audit to the brief
|
||||
|
||||
**Fixture:**
|
||||
- Project with a mix of compliant and non-compliant GDDs
|
||||
- `project.yaml` configures the engine and has no `modes` block (`workflow`
|
||||
resolves to `minimal`)
|
||||
- `design/game-brief.md` exists
|
||||
- `design/gdd/combat.md` exists and is missing `## Acceptance Criteria`
|
||||
|
||||
**Input:** `/adopt`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill completes full audit and produces migration plan
|
||||
2. No director agents are spawned at any point
|
||||
3. No gate IDs (CD-*, TD-*, AD-*, PR-*) appear in output
|
||||
4. No `/gate-check` is invoked during the skill run
|
||||
1. Phase 2 audits `design/game-brief.md`; GDDs, ADRs and UX specs are not expected
|
||||
2. `combat.md` is checked at the `standard` bar advisorily — its missing Acceptance
|
||||
Criteria is reported as informational, not as a HIGH gap
|
||||
3. The plan does not prescribe the infrastructure bootstrap: its Step 3 says in
|
||||
one line that it is not on the minimal path and names `/create-stories` as
|
||||
the next step (no stories exist)
|
||||
4. No director agents are spawned at any point
|
||||
5. No gate IDs (CD-*, TD-*, AD-*, PR-*) appear in output
|
||||
6. No `/gate-check` is invoked during the skill run
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] `design/game-brief.md` is audited
|
||||
- [ ] Absent ADRs and UX specs are not reported as gaps
|
||||
- [ ] `combat.md`'s missing Acceptance Criteria is not counted in the HIGH total
|
||||
- [ ] The plan prescribes none of `/architecture-review`, `/create-control-manifest`, `/sprint-plan` or `/gate-check` — none is on the minimal path
|
||||
- [ ] No director gate is invoked and no gate skip messages appear
|
||||
- [ ] Skill reaches plan-writing or cancellation without any gate verdict
|
||||
|
||||
---
|
||||
|
||||
### Case 6: v1.0 Project — Converter run, never hand-migrated
|
||||
|
||||
**Fixture:**
|
||||
- No `project.yaml` at the repo root
|
||||
- `production/stage.txt` reads `Production`; `.claude/docs/technical-preferences.md`
|
||||
has `- **Engine**: Godot 4.6` and a filled naming section
|
||||
- `design/gdd/` holds GDDs; no `production/migration-report.md`
|
||||
|
||||
**Input:** `/adopt`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Phase 2g identifies a v1.0 project needing migration (no `project.yaml`,
|
||||
legacy files with real, migratable values)
|
||||
2. It runs `bash .claude/scripts/migrate-v1-config.sh --dry-run` during the
|
||||
audit (the dry run writes nothing) — it does not hand-write `project.yaml`
|
||||
3. The migration is classified BLOCKING in Phase 3 and heads the plan, which
|
||||
reports what the dry run listed
|
||||
4. After the plan, Phase 7 offers the migration first: it asks "May I run the
|
||||
converter? It writes `project.yaml` and `production/migration-report.md`."
|
||||
and runs it without `--dry-run` only after approval
|
||||
5. It tells the user to read `production/migration-report.md` before running
|
||||
`bash .claude/scripts/migrate-v1-config.sh --finalize`, and does not run
|
||||
`--finalize` itself before the user has read the report
|
||||
|
||||
**Assertions:**
|
||||
- [ ] `--dry-run` runs before any real migration, and its output is reported
|
||||
- [ ] `project.yaml` is written only by the converter, after the ask naming `project.yaml` and `production/migration-report.md` is approved
|
||||
- [ ] `--finalize` is not run before the user has read `production/migration-report.md`
|
||||
- [ ] The migration is a BLOCKING item, first in the plan
|
||||
- [ ] No legacy file is deleted during the run
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Emits "Scanning project artifacts..." before silent read phase
|
||||
@@ -210,5 +280,9 @@ None. `/adopt` is a brownfield audit utility. No director gates apply.
|
||||
Not separately fixture-tested here.
|
||||
- The systems-index.md parenthetical status value check (BLOCKING) is a special case
|
||||
that triggers an immediate fix offer before writing the plan; not separately tested.
|
||||
- The review-mode.txt prompt (Phase 6b) runs after plan writing if `production/review-mode.txt`
|
||||
does not exist; not separately tested here.
|
||||
- Phase 6b reports the resolved review mode and writes nothing — `review_mode` is
|
||||
fronted by `modes.rigor`, so pinning it would shadow the rigor expansion; Case 1
|
||||
asserts that nothing is written for it.
|
||||
- The converter's refusals (exit 3 when its values disagree or a migration
|
||||
already ran; `--finalize` exiting 4 on a mismatch) are not separately tested;
|
||||
Case 6 covers the migration itself.
|
||||
|
||||
@@ -2,178 +2,253 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/asset-spec` generates per-asset visual specification documents from design
|
||||
requirements. It reads the relevant GDD, art bible, and design system to produce
|
||||
a structured asset spec sheet that defines: dimensions, animation states (if
|
||||
applicable), color palette reference, style notes, technical constraints
|
||||
(format, file size budget), and deliverable checklist.
|
||||
`/asset-spec` turns one target's visual needs into asset specs with AI generation
|
||||
prompts. The argument names a target — `system:<name>`, `level:<name>` or
|
||||
`character:<name>` (an `entity:` entry from the inventory is read like a
|
||||
character) — plus an optional `--review full|lean|solo`. It reads the art bible
|
||||
at `design/art/art-bible.md` (and stops if it is missing, unless the minimal-path
|
||||
exception applies), `performance.*` and `naming.*` from `project.yaml`, and the
|
||||
target's source doc. It confirms the identified asset list with the user, drafts
|
||||
one `ASSET-NNN` block per asset, and after approval writes **one file per target**
|
||||
to `design/assets/specs/[target-name]-assets.md`, then updates
|
||||
`design/assets/asset-manifest.md`. Asset IDs run sequentially across the whole
|
||||
project. A re-run for a target whose spec file exists updates that file in place
|
||||
(existing blocks and IDs kept, new assets appended) after "May I update".
|
||||
|
||||
Spec sheets are written to `assets/specs/[asset-name]-spec.md` after a "May I write"
|
||||
ask. If a spec already exists, the skill offers to update it. When multiple assets
|
||||
are requested in a single invocation, a "May I write" ask is made per asset. No
|
||||
director gates apply. The verdict is COMPLETE when all requested specs are written.
|
||||
The resolved review mode decides who drafts: `full` spawns `art-director` and
|
||||
`technical-artist` in parallel, `lean` spawns `art-director` only, `solo` (the
|
||||
default, via `modes.rigor: minimal`) spawns no agents; `full` is also what
|
||||
`modes.rigor: full` resolves to. Every skipped agent is announced by name in
|
||||
the output. These are drafting consultations, not gates. The skill produces an
|
||||
artifact, not a judgement, so it has no run verdict; it closes with an
|
||||
`AskUserQuestion` offering the next target or `/asset-audit`.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keyword: COMPLETE
|
||||
- [ ] Contains "May I write" collaborative protocol language (per asset)
|
||||
- [ ] Has a next-step handoff (e.g., assign to an artist, or `/asset-audit` later)
|
||||
- [ ] Check 3 passes on the `BLOCKED` keyword of its Error Recovery Protocol; the skill has no run verdict by design (artifact producer), and the run closes with "Asset specs complete for **[target]**. What's next?"
|
||||
- [ ] Contains "May I write" language for the spec file and "May I update" for the manifest
|
||||
- [ ] Has a next-step handoff (`/asset-spec [next-context]`, `/asset-audit`)
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
None. `/asset-spec` is a design documentation utility. Technical artists may
|
||||
review specs separately but this is not a gate within this skill.
|
||||
No gate IDs and no `/gate-check`. The review mode controls drafting agents only:
|
||||
|
||||
- **Full mode** (what `modes.rigor: full` resolves to): `art-director` and
|
||||
`technical-artist` spawned in parallel
|
||||
- **Lean mode**: `art-director` only; the output says "technical-artist skipped —
|
||||
Lean mode: technical constraints not validated."
|
||||
- **Solo mode** (default at `modes.rigor: minimal`): no agents; the main session
|
||||
derives specs from the art bible, and the output names both skipped agents and
|
||||
says technical constraints were not validated
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — Enemy sprite spec with full GDD and art bible
|
||||
### Case 1: Happy Path — Character spec in lean mode
|
||||
|
||||
**Fixture:**
|
||||
- `design/gdd/enemies.md` exists with enemy variants defined
|
||||
- `design/art-bible.md` exists with color palette and style notes
|
||||
- No existing asset spec for "goblin-enemy"
|
||||
- `project.yaml` sets `modes.rigor: standard` (review mode resolves to `lean`) and
|
||||
has `performance.*` and `naming.*` values
|
||||
- `design/art/art-bible.md` exists with Shape Language, Color System and Section 8 Asset Standards
|
||||
- `design/narrative/characters/goblin-enemy.md` exists with a visual description
|
||||
- No `design/assets/asset-manifest.md`
|
||||
|
||||
**Input:** `/asset-spec goblin-enemy`
|
||||
**Input:** `/asset-spec character:goblin-enemy`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads enemies GDD and art bible
|
||||
2. Skill generates a spec for the goblin enemy sprite:
|
||||
- Dimensions: inferred from engine defaults or explicitly from GDD
|
||||
- Animation states: idle, walk, attack, hurt, death
|
||||
- Color palette reference: links to art-bible palette section
|
||||
- Style notes: from art bible character design rules
|
||||
- Technical constraints: format (PNG), size budget
|
||||
- Deliverable checklist
|
||||
3. Skill asks "May I write to `assets/specs/goblin-enemy-spec.md`?"
|
||||
4. File written on approval; verdict is COMPLETE
|
||||
1. Skill reads the art bible, project config and the character doc before asking anything
|
||||
2. Skill presents the context summary ("Asset Spec: Character — goblin-enemy")
|
||||
3. Phase 2 identifies the character's assets (sprite sheets such as idle, walk,
|
||||
attack, death; portrait; ability VFX; UI icon), shows them grouped by category,
|
||||
and asks via `AskUserQuestion` before speccing
|
||||
4. Lean mode: spawns `art-director` only, and says so: "technical-artist skipped —
|
||||
Lean mode: technical constraints not validated."
|
||||
5. Phase 4 presents one `ASSET-NNN` block per asset with the field table
|
||||
(Category, Dimensions, Format, Naming, Polycount, Texture Res), Visual
|
||||
Description, Art Bible Anchors citing sections, Generation Prompt, and `Status: Needed`
|
||||
6. After approval asks "May I write the spec to `design/assets/specs/goblin-enemy-assets.md`?"
|
||||
7. Creates the manifest after asking "May I update `design/assets/asset-manifest.md`?";
|
||||
IDs start at `ASSET-001`
|
||||
8. Phase 6 `AskUserQuestion` offers another target or `/asset-audit`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All 6 spec components are present (dimensions, animations, palette, style, tech, checklist)
|
||||
- [ ] Color palette reference links to art bible (not duplicated)
|
||||
- [ ] Animation states are drawn from GDD (not invented)
|
||||
- [ ] "May I write" is asked with the correct path
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] Asset list is confirmed with `AskUserQuestion` before any spec is generated
|
||||
- [ ] Only `art-director` is spawned (no `technical-artist` in lean mode)
|
||||
- [ ] The output names `technical-artist` as skipped and says technical constraints were not validated
|
||||
- [ ] Each asset block has the field table, Art Bible Anchors (by section), a Generation Prompt and `Status: Needed`
|
||||
- [ ] "May I write" names `design/assets/specs/goblin-enemy-assets.md`
|
||||
- [ ] Manifest is created with a Progress Summary and IDs starting at `ASSET-001`
|
||||
- [ ] Phase 6 offers `/asset-audit` among the next steps
|
||||
|
||||
---
|
||||
|
||||
### Case 2: No Art Bible Found — Spec with Placeholder Style Notes, Dependency Flagged
|
||||
### Case 2: No Art Bible — Skill stops before speccing
|
||||
|
||||
**Fixture:**
|
||||
- `design/gdd/player.md` exists
|
||||
- `design/art-bible.md` does NOT exist
|
||||
- `project.yaml` sets `modes.rigor: standard`
|
||||
- `design/gdd/combat.md` exists with a Visual/Audio Requirements section
|
||||
- `design/art/art-bible.md` does NOT exist
|
||||
- `design/game-brief.md` does NOT exist
|
||||
|
||||
**Input:** `/asset-spec player-sprite`
|
||||
**Input:** `/asset-spec system:combat`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads player GDD but cannot find the art bible
|
||||
2. Skill generates spec with placeholder style notes: "DEPENDENCY GAP: art bible
|
||||
not found — style notes are placeholders"
|
||||
3. Color palette section uses: "TBD — see art bible when created"
|
||||
4. Skill asks "May I write to `assets/specs/player-sprite-spec.md`?"
|
||||
5. File written with placeholders and dependency flag; verdict is COMPLETE with advisory
|
||||
1. Phase 1 reads for `design/art/art-bible.md` and finds nothing
|
||||
2. The minimal-path exception does not apply (no game brief)
|
||||
3. Skill stops with: "No art bible found. Run `/art-bible` first — asset specs are
|
||||
anchored to the art bible's visual rules and asset standards."
|
||||
4. No asset identification, no agents, no files
|
||||
|
||||
**Assertions:**
|
||||
- [ ] DEPENDENCY GAP is flagged for the missing art bible
|
||||
- [ ] Spec is still generated (not blocked)
|
||||
- [ ] Style notes contain placeholder markers, not invented styles
|
||||
- [ ] Verdict is COMPLETE with advisory note
|
||||
- [ ] The "No art bible found. Run `/art-bible` first" message is shown
|
||||
- [ ] No asset list is presented and no `Agent` call is made
|
||||
- [ ] No spec file or manifest is written
|
||||
- [ ] Specs are not generated with invented style rules
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Asset Spec Already Exists — Offers to Update
|
||||
### Case 3: Manifest Already Exists — IDs continue and shared assets are reused
|
||||
|
||||
**Fixture:**
|
||||
- `assets/specs/goblin-enemy-spec.md` already exists
|
||||
- GDD has been updated since the spec was written (new attack animation added)
|
||||
- `project.yaml` sets `modes.rigor: standard`; art bible exists
|
||||
- `design/assets/asset-manifest.md` exists; the highest ID is `ASSET-014`, and
|
||||
`ASSET-012` (Generic Hit Spark) is specced for the Combat system in
|
||||
`design/assets/specs/combat-assets.md`
|
||||
- `design/gdd/tower-defense.md` Visual/Audio section lists a hit spark and a tower-build VFX
|
||||
|
||||
**Input:** `/asset-spec goblin-enemy`
|
||||
**Input:** `/asset-spec system:tower-defense`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill detects existing spec file
|
||||
2. Skill reports: "Asset spec already exists for goblin-enemy — checking for updates"
|
||||
3. Skill diffs GDD against existing spec and identifies: new "charge-attack" animation
|
||||
state added in GDD but not in spec
|
||||
4. Skill presents the diff: "1 new animation state found — offering to update spec"
|
||||
5. Skill asks "May I update `assets/specs/goblin-enemy-spec.md`?" (not overwrite)
|
||||
6. Spec is updated; verdict is COMPLETE
|
||||
1. Skill reads the manifest and globs `design/assets/specs/*.md` for shareable assets
|
||||
2. Context summary lists the shared asset found in another spec
|
||||
3. The hit spark reuses `ASSET-012` instead of creating a duplicate, with a note like
|
||||
"ASSET-012 (Generic Hit Spark) already specced for Combat system. Reusing for Tower Defense"
|
||||
4. The new tower-build VFX is numbered `ASSET-015`
|
||||
5. After "May I write" for `design/assets/specs/tower-defense-assets.md`, the skill asks
|
||||
"May I update `design/assets/asset-manifest.md`?", appends a Tower Defense context
|
||||
block, and updates the Progress Summary counts
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Existing spec is detected and "update" path is offered
|
||||
- [ ] Diff between GDD and existing spec is shown
|
||||
- [ ] "May I update" language is used (not "May I write")
|
||||
- [ ] Existing spec content is preserved; only the diff is applied
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] The hit spark references `ASSET-012`; no duplicate hit-spark spec is created
|
||||
- [ ] The new asset's ID is `ASSET-015` (project-wide sequence, not restarted per target)
|
||||
- [ ] The manifest is appended to, not recreated, after a "May I update" ask
|
||||
- [ ] Progress Summary counts are updated
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Multiple Assets Requested — May-I-Write Per Asset
|
||||
### Case 4: Several Assets in One Target — One file, per-asset revision before write
|
||||
|
||||
**Fixture:**
|
||||
- GDD and art bible exist
|
||||
- User requests specs for 3 assets: goblin-enemy, orc-enemy, treasure-chest
|
||||
- `project.yaml` sets `modes.rigor: standard`; art bible exists; no manifest
|
||||
- `design/gdd/combat.md` Visual/Audio section names a hit-spark VFX, a floating
|
||||
damage number (UI) and a hit sound (audio)
|
||||
|
||||
**Input:** `/asset-spec goblin-enemy orc-enemy treasure-chest`
|
||||
**Input:** `/asset-spec system:combat`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill generates all 3 specs in sequence
|
||||
2. For each asset, skill shows the draft and asks "May I write to
|
||||
`assets/specs/[name]-spec.md`?" individually
|
||||
3. User can approve all 3 or skip individual assets
|
||||
4. All approved specs are written; verdict is COMPLETE
|
||||
1. Phase 2 lists all three assets grouped under VFX, UI and Audio and asks the user to confirm
|
||||
2. Phase 4 presents `ASSET-001` to `ASSET-003`; the audio asset carries a
|
||||
description of its sonic character and no image generation prompt
|
||||
3. User picks `[B] Revise a specific asset` for the damage number; the skill revises
|
||||
it inline and re-presents without re-spawning agents
|
||||
4. User approves all; the skill asks "May I write" **once**, for
|
||||
`design/assets/specs/combat-assets.md`, which holds all three assets
|
||||
|
||||
**Assertions:**
|
||||
- [ ] "May I write" is asked 3 times (once per asset), not once for all
|
||||
- [ ] User can decline one asset without blocking the others
|
||||
- [ ] All 3 spec files are written for approved assets
|
||||
- [ ] Verdict is COMPLETE when all approved specs are written
|
||||
- [ ] One "May I write" ask covers all three assets in one file (not one ask or file per asset)
|
||||
- [ ] The audio asset has no image generation prompt
|
||||
- [ ] The minor revision does not re-spawn `art-director`
|
||||
- [ ] All three `ASSET-NNN` blocks are in `design/assets/specs/combat-assets.md`
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; asset-spec is a design utility
|
||||
### Case 5: Director Gate Check — Solo default, brief instead of art bible
|
||||
|
||||
**Fixture:**
|
||||
- GDD and art bible exist
|
||||
- `project.yaml` has no `modes` block (review mode resolves to `solo` via `modes.rigor: minimal`)
|
||||
- `design/game-brief.md` exists with an Art & audio direction line
|
||||
- No `design/art/art-bible.md`
|
||||
- No character profile for `hero` in `design/narrative/` or `design/assets/entity-inventory.md`
|
||||
|
||||
**Input:** `/asset-spec goblin-enemy`
|
||||
**Input:** `/asset-spec character:hero`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill generates and writes the asset spec
|
||||
2. No director agents are spawned
|
||||
3. No gate IDs appear in output
|
||||
1. The minimal-path exception applies: the skill does not stop, and states at the top
|
||||
"Anchored to the brief's one-line art direction; there is no art bible, so colour,
|
||||
shape and asset-standard rules below are proposals, not constraints."
|
||||
2. `/art-bible` is offered once, as optional
|
||||
3. No profile found → `AskUserQuestion`: "No profile found for **hero**. Describe it
|
||||
briefly" with Describe / Skip / Stop options
|
||||
4. Solo mode: no `Agent` calls; the output announces "art-director and
|
||||
technical-artist skipped — Solo mode: specs derived in this session; technical
|
||||
constraints not validated."
|
||||
5. No gate IDs appear and `/gate-check` is not invoked
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Verdict is COMPLETE without any gate check
|
||||
- [ ] Skill continues without an art bible and says the specs are anchored to the brief
|
||||
- [ ] Missing profile triggers the "Describe it briefly" question rather than a failure
|
||||
- [ ] No `art-director` or `technical-artist` is spawned
|
||||
- [ ] Both skipped agents are named in the output, which says technical constraints were not validated
|
||||
- [ ] If the spec is written, its header names `design/game-brief.md` as the art direction (no `> **Art Bible**:` line), each asset carries a `**Brief anchor:**` line instead of Art Bible Anchors, and no Art Bible section or §8 tier is cited
|
||||
- [ ] No gate IDs appear — the skip note names agents, not gates
|
||||
|
||||
---
|
||||
|
||||
### Case 6: Re-run on an Existing Spec — Updated in place, not overwritten
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` sets `modes.rigor: standard`; art bible exists
|
||||
- `design/assets/specs/goblin-enemy-assets.md` exists with `ASSET-001` to
|
||||
`ASSET-004` (idle, walk and attack sprites; portrait); the manifest's highest ID
|
||||
is `ASSET-009`
|
||||
- `design/narrative/characters/goblin-enemy.md` has since gained a charge-attack
|
||||
ability
|
||||
|
||||
**Input:** `/asset-spec character:goblin-enemy`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Phase 1 reads the existing spec file for this target as well as the character doc
|
||||
2. Phase 2 proposes only what is new — the charge-attack sprite and its VFX — and
|
||||
confirms the list with the user
|
||||
3. The new assets are numbered `ASSET-010` and `ASSET-011` (project-wide sequence)
|
||||
4. The skill shows what changes, then asks "May I update
|
||||
`design/assets/specs/goblin-enemy-assets.md`?" — not "May I write"
|
||||
5. `ASSET-001` to `ASSET-004` are kept unchanged in the file; the new blocks are appended
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The existing spec file is detected and read before the asset list is proposed
|
||||
- [ ] The ask is "May I update", naming the existing file
|
||||
- [ ] Existing blocks and IDs are preserved; the file is not overwritten wholesale
|
||||
- [ ] New assets continue the project-wide ID sequence (`ASSET-010`, `ASSET-011`)
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Reads GDD, art bible, and design system before generating spec
|
||||
- [ ] Includes all 6 spec components (dimensions, animations, palette, style, tech, checklist)
|
||||
- [ ] Flags missing dependencies (art bible, GDD) with DEPENDENCY GAP notes
|
||||
- [ ] Asks "May I write" (or "May I update") per asset
|
||||
- [ ] Handles multiple assets with individual write confirmations
|
||||
- [ ] Verdict is COMPLETE when all approved specs are written
|
||||
- [ ] Reads the art bible, project config and source doc before asking the user anything
|
||||
- [ ] Never specs assets before the user confirms the asset list
|
||||
- [ ] Spawns agents strictly per the resolved review mode (`--review` overrides for one run), and names every skipped agent in the output
|
||||
- [ ] Surfaces any art-director / technical-artist conflict instead of silently resolving it
|
||||
- [ ] Asks "May I write" before a new spec file, "May I update" before changing an existing one, and "May I update" before the manifest
|
||||
- [ ] Assigns asset IDs sequentially across the project, reading the manifest first
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- Audio asset specs (sound effects, music) follow the same structure with
|
||||
different fields (duration, sample rate, looping) and are not separately tested.
|
||||
- UI asset specs (icons, button states) follow the same flow with interaction
|
||||
state requirements aligned to the UX spec.
|
||||
- The case where GDD is also missing (neither GDD nor art bible exists) is not
|
||||
separately tested; spec would be generated with both dependency gaps flagged.
|
||||
- The no-argument flow (pick the next "Needed" item from
|
||||
`design/assets/entity-inventory.md`, or build the inventory in Phase 0b) is not
|
||||
separately tested.
|
||||
- `level:` targets and full-mode parallel spawning with a surfaced
|
||||
art-director / technical-artist conflict are not separately tested.
|
||||
- Agent BLOCKED handling (Error Recovery Protocol: always produce a partial spec)
|
||||
is not separately tested.
|
||||
|
||||
@@ -2,188 +2,247 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/brainstorm` facilitates guided game concept ideation. It presents 2-4 concept
|
||||
options with pros/cons, lets the user choose and refine a concept, and produces
|
||||
a structured `design/gdd/game-concept.md` document. The skill is collaborative —
|
||||
it asks questions before proposing options and iterates until the user approves
|
||||
a concept direction.
|
||||
`/brainstorm` facilitates guided game concept ideation. It branches on the
|
||||
resolved `workflow` tier **first**:
|
||||
|
||||
In `full` review mode, four director gates spawn in parallel after the concept
|
||||
is drafted: CD-PILLARS (creative-director), AD-CONCEPT-VISUAL (art-director),
|
||||
TD-FEASIBILITY (technical-director), and PR-SCOPE (producer). In `lean` mode,
|
||||
all 4 inline gates are skipped (lean mode only runs PHASE-GATEs, and brainstorm
|
||||
has none). In `solo` mode, all gates are skipped. The skill asks "May I write"
|
||||
before writing `design/gdd/game-concept.md`.
|
||||
- **`minimal`** (the default — `modes.rigor` defaults to `minimal`) runs the
|
||||
**Lean Brief flow**: 2–3 one-line concepts, one pick, the brief fields filled
|
||||
conversationally, and a one-page `design/game-brief.md` written after
|
||||
"Brief is ready. May I write it to `design/game-brief.md`?". No director gates
|
||||
run and no `game-concept.md` is written.
|
||||
- **`standard` / `full`** run the full flow: Creative Discovery, 3 distinct
|
||||
concepts, core loop, pillars, player types, scope, then
|
||||
`design/gdd/game-concept.md` after "Game concept is ready. May I write it to
|
||||
`design/gdd/game-concept.md`?".
|
||||
|
||||
In the full flow, director gates run at three points, not together: CD-PILLARS
|
||||
(creative-director) and AD-CONCEPT-VISUAL (art-director) in parallel once
|
||||
pillars are agreed (Phase 4); TD-FEASIBILITY (technical-director) after technical
|
||||
risks are identified and before scope tiers (Phase 6); PR-SCOPE (producer) after
|
||||
scope tiers are defined. Review mode comes from the resolved config
|
||||
(`--review` overrides it); `lean` and `solo` skip all four with a named note each.
|
||||
Both flows end with `Verdict: **COMPLETE**`.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keywords: APPROVED, REJECTED, CONCERNS
|
||||
- [ ] Contains "May I write" collaborative protocol language (for game-concept.md)
|
||||
- [ ] Has a next-step handoff at the end (`/map-systems`)
|
||||
- [ ] Contains verdict keyword: COMPLETE (in both the Lean Brief flow and the full flow)
|
||||
- [ ] Contains "May I write" language for both `design/game-brief.md` and `design/gdd/game-concept.md`
|
||||
- [ ] Has a next-step handoff at the end (`/map-systems` in the full flow's Path A; `/create-stories` in the Lean Brief flow)
|
||||
- [ ] Documents 4 director gates in full mode: CD-PILLARS, AD-CONCEPT-VISUAL, TD-FEASIBILITY, PR-SCOPE
|
||||
- [ ] Documents that all 4 gates are skipped in lean and solo modes
|
||||
- [ ] Documents a named skip note for each gate in lean and solo modes
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
In `full` mode: CD-PILLARS, AD-CONCEPT-VISUAL, TD-FEASIBILITY, and PR-SCOPE
|
||||
spawn in parallel after the concept draft is approved by the user.
|
||||
Full flow only (`workflow: standard` or `full`):
|
||||
|
||||
In `lean` mode: all 4 inline gates are skipped (brainstorm has no PHASE-GATEs,
|
||||
so lean mode skips everything). Output notes all 4 as: "[GATE-ID] skipped — lean mode".
|
||||
- **Full review mode**: CD-PILLARS + AD-CONCEPT-VISUAL spawn in parallel after
|
||||
pillars are agreed; TD-FEASIBILITY spawns after technical risks are identified;
|
||||
PR-SCOPE spawns after scope tiers are defined.
|
||||
- **Lean review mode**: all 4 skipped (none is a PHASE-GATE). Notes read
|
||||
"CD-PILLARS skipped — Lean mode.", "AD-CONCEPT-VISUAL skipped — Lean mode.",
|
||||
"TD-FEASIBILITY skipped — Lean mode.", "PR-SCOPE skipped — Lean mode."
|
||||
- **Solo review mode**: all 4 skipped, same notes with "Solo mode".
|
||||
|
||||
In `solo` mode: all 4 gates are skipped. Output notes all 4 as: "[GATE-ID] skipped — solo mode".
|
||||
`workflow: minimal` (Lean Brief flow): no director gate is spawned at all.
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — Full mode, 3 concepts, user picks one, all 4 directors approve
|
||||
### Case 1: Happy Path — Full mode, 3 concepts, gates at their three points
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` sets `modes.rigor: full` (`workflow` → `full`, `review_mode` → `full`)
|
||||
- No existing `design/gdd/game-concept.md`
|
||||
- `production/session-state/review-mode.txt` contains `full`
|
||||
- CD-PILLARS returns APPROVE; TD-FEASIBILITY returns VIABLE and PR-SCOPE returns REALISTIC (each gate's own verdict words)
|
||||
|
||||
**Input:** `/brainstorm`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill asks the user questions about genre, scope, and target feeling
|
||||
2. Skill presents 3 concept options with pros/cons each
|
||||
3. User selects one concept
|
||||
4. Skill elaborates the chosen concept into a structured draft
|
||||
5. All 4 director gates spawn in parallel: CD-PILLARS, AD-CONCEPT-VISUAL, TD-FEASIBILITY, PR-SCOPE
|
||||
6. All 4 return APPROVED
|
||||
7. Skill asks "May I write `design/gdd/game-concept.md`?"
|
||||
8. Concept written after approval
|
||||
1. Tier branch: `full` → full flow
|
||||
2. Phase 1 asks discovery questions (favourite games as plain text; the
|
||||
Experience / Timeline / Dev level multi-tab `AskUserQuestion`)
|
||||
3. Phase 2 presents 3 concepts, then a plain-list `AskUserQuestion` (no tabs) to pick
|
||||
4. After pillars are locked, CD-PILLARS and AD-CONCEPT-VISUAL are spawned in
|
||||
parallel; their results are presented in a two-tab (Pillars / Visual anchor) `AskUserQuestion`
|
||||
5. Phase 6 asks for platform and engine (the engine question always prompts);
|
||||
TD-FEASIBILITY spawns after risks are identified, before scope tiers
|
||||
6. PR-SCOPE spawns after scope tiers are defined
|
||||
7. Skill asks "Game concept is ready. May I write it to `design/gdd/game-concept.md`?"
|
||||
8. Written after approval with a Visual Identity Anchor section; Path A next steps include `/map-systems`
|
||||
9. Verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Exactly 3 concept options are presented (not 1, not 5+)
|
||||
- [ ] All 4 director gates spawn in parallel (not sequentially)
|
||||
- [ ] All 4 gates complete before the "May I write" ask
|
||||
- [ ] "May I write `design/gdd/game-concept.md`?" is asked before writing
|
||||
- [ ] Concept file is NOT written without user approval
|
||||
- [ ] Next-step handoff to `/map-systems` is present
|
||||
- [ ] Exactly 3 concept options are presented
|
||||
- [ ] CD-PILLARS and AD-CONCEPT-VISUAL are issued as simultaneous `Agent` calls
|
||||
- [ ] TD-FEASIBILITY spawns before scope tiers are defined; PR-SCOPE after
|
||||
- [ ] "May I write it to `design/gdd/game-concept.md`?" is asked before writing
|
||||
- [ ] The concept document includes a Visual Identity Anchor section
|
||||
- [ ] Next-step handoff lists `/map-systems`
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Failure Path — CD-PILLARS returns REJECT
|
||||
|
||||
**Fixture:**
|
||||
- Concept draft is complete
|
||||
- `production/session-state/review-mode.txt` contains `full`
|
||||
- CD-PILLARS gate returns REJECT: "The concept has no identifiable creative pillar"
|
||||
- `project.yaml` sets `modes.rigor: full`
|
||||
- Pillars and anti-pillars have been agreed with the user
|
||||
- CD-PILLARS returns REJECT: "The pillars do not carry weight — none forces a design choice"
|
||||
- AD-CONCEPT-VISUAL returns 3 named visual directions
|
||||
|
||||
**Input:** `/brainstorm`
|
||||
|
||||
**Expected behavior:**
|
||||
1. CD-PILLARS gate returns REJECT with specific feedback
|
||||
2. Skill surfaces the rejection to the user
|
||||
3. Concept is NOT written to file
|
||||
4. User is asked: rethink the concept direction, or override the rejection
|
||||
5. If rethinking: skill returns to the concept options phase
|
||||
1. Both gates were spawned in parallel; the skill collects both results
|
||||
2. Because CD-PILLARS returned REJECT, the two-tab question is **not** used: the
|
||||
creative-director's feedback is presented in a Pillars question on its own,
|
||||
with options `Lock in as-is` / `Revise [specific pillar]` / `Discuss further`
|
||||
3. Pillar issues are resolved first; only then is the Visual anchor question
|
||||
asked, on its own, with the art-director's 3 directions
|
||||
4. The skill does not move on to document generation or the write ask while the pillar issue is open
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Concept is NOT written when CD-PILLARS returns REJECT
|
||||
- [ ] Rejection feedback is shown to the user verbatim
|
||||
- [ ] User is given the option to rethink or override
|
||||
- [ ] Skill returns to concept ideation phase if user chooses to rethink
|
||||
- [ ] CD-PILLARS feedback is shown to the user
|
||||
- [ ] User is offered `Revise [specific pillar]` and `Discuss further` (and may `Lock in as-is`)
|
||||
- [ ] The first question after the REJECT has no Visual anchor tab
|
||||
- [ ] Visual anchor selection is not requested until the pillar issue is resolved
|
||||
- [ ] `design/gdd/game-concept.md` is not written at this point
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Lean Mode — All 4 gates skipped; concept written after user confirms
|
||||
### Case 3: Lean Mode — All 4 gates skipped with named notes
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` sets `modes.rigor: standard` (`workflow` → `standard`, `review_mode` → `lean`)
|
||||
- No existing game concept
|
||||
- `production/session-state/review-mode.txt` contains `lean`
|
||||
|
||||
**Input:** `/brainstorm`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Concept options are presented and user selects one
|
||||
2. Concept is elaborated into a structured draft
|
||||
3. All 4 director gates are skipped — each noted: "[GATE-ID] skipped — lean mode"
|
||||
4. Skill asks user to confirm the concept is ready to write
|
||||
5. "May I write `design/gdd/game-concept.md`?" asked after confirmation
|
||||
6. Concept written after approval
|
||||
1. Tier branch: `standard` → full flow
|
||||
2. At Phase 4: "CD-PILLARS skipped — Lean mode. AD-CONCEPT-VISUAL skipped — Lean mode."
|
||||
3. At Phase 6: "TD-FEASIBILITY skipped — Lean mode." then "PR-SCOPE skipped — Lean mode."
|
||||
4. No director agents are spawned
|
||||
5. "Game concept is ready. May I write it to `design/gdd/game-concept.md`?" is asked
|
||||
6. Written after approval; Verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All 4 gate skip notes appear: "CD-PILLARS skipped — lean mode", "AD-CONCEPT-VISUAL skipped — lean mode", "TD-FEASIBILITY skipped — lean mode", "PR-SCOPE skipped — lean mode"
|
||||
- [ ] Concept is written after user confirmation only (no director approval needed in lean)
|
||||
- [ ] All 4 skip notes appear, each naming its gate and "Lean mode"
|
||||
- [ ] No `Agent` call is made for any director
|
||||
- [ ] "May I write" is still asked before writing
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Solo Mode — All gates skipped; concept written with only user approval
|
||||
### Case 4: Solo Default — Minimal tier runs the Lean Brief flow
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` has `engine.name` set and no `modes` block (`modes.rigor`
|
||||
defaults to `minimal`: `workflow` → `minimal`, `review_mode` → `solo`)
|
||||
- No `design/game-brief.md`, no `design/gdd/game-concept.md`
|
||||
|
||||
**Input:** `/brainstorm roguelike`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Tier branch: `minimal` → Lean Brief flow; Phases 1–5 are skipped
|
||||
2. From the hint, proposes 2–3 one-line concepts and asks one `AskUserQuestion` to pick or combine
|
||||
3. Fills the brief fields conversationally (pitch, core loop, player goal & fail
|
||||
state, MVP, out of scope, build order) and presents the brief for one confirmation
|
||||
4. Asks "Brief is ready. May I write it to `design/game-brief.md`?" with
|
||||
`[A] Yes — write it` / `[B] Revise a field first`
|
||||
5. Next steps: `/create-stories`, then `/dev-story`; `/setup-engine` is NOT listed
|
||||
because `engine.name` is set
|
||||
6. Verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Output file is `design/game-brief.md`; `design/gdd/game-concept.md` is not written
|
||||
- [ ] 2–3 one-line concepts are proposed (not the full 9-field concept cards)
|
||||
- [ ] No director agent is spawned
|
||||
- [ ] Next steps list `/create-stories` and `/dev-story`, not `/map-systems` or `/setup-engine`
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate — PR-SCOPE returns UNREALISTIC
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` sets `modes.rigor: full`
|
||||
- Concept, pillars and scope tiers are defined
|
||||
- PR-SCOPE returns UNREALISTIC: "The MVP would take 18+ months for a solo developer"
|
||||
- The user cuts the MVP to the adjusted tiers the skill offers, then answers
|
||||
`[A] Yes — write it` to the write ask
|
||||
|
||||
**Input:** `/brainstorm`
|
||||
|
||||
**Expected behavior:**
|
||||
1. PR-SCOPE spawns after scope tiers are defined
|
||||
2. Skill presents the producer's assessment to the user
|
||||
3. Because the verdict is UNREALISTIC, the skill offers to adjust the MVP
|
||||
definition or scope tiers before writing the document
|
||||
4. The write ask comes only after that offer is resolved; the user decides
|
||||
5. After the user approves the write, `design/gdd/game-concept.md` is written
|
||||
|
||||
**Assertions:**
|
||||
- [ ] PR-SCOPE assessment is shown to the user before the "May I write" ask
|
||||
- [ ] Skill offers to adjust the MVP definition or scope tiers
|
||||
- [ ] Skill does NOT discard or auto-reject the concept — the user decides
|
||||
- [ ] The concept is written only after the user approves the write
|
||||
- [ ] "Estimated Scope" in the written concept matches the Scope Tiers timeline (e.g. "Large (X–Y months, solo)")
|
||||
|
||||
---
|
||||
|
||||
### Case 6: Solo Review on the Full Flow — All 4 gates skipped with Solo-mode notes
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` sets `modes.rigor: standard` (`workflow` → `standard`)
|
||||
- No existing game concept
|
||||
- `production/session-state/review-mode.txt` contains `solo`
|
||||
|
||||
**Input:** `/brainstorm`
|
||||
**Input:** `/brainstorm --review solo`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Concept options are presented and user selects one
|
||||
2. Concept draft is shown to user
|
||||
3. All 4 director gates are skipped — each noted with "solo mode"
|
||||
4. "May I write `design/gdd/game-concept.md`?" asked
|
||||
5. Concept written after user approval
|
||||
1. Tier branch: `standard` → full flow; `--review solo` overrides the resolved
|
||||
`review_mode` for this run
|
||||
2. At Phase 4: "CD-PILLARS skipped — Solo mode. AD-CONCEPT-VISUAL skipped — Solo mode."
|
||||
3. At Phase 6: "TD-FEASIBILITY skipped — Solo mode." then "PR-SCOPE skipped — Solo mode."
|
||||
4. No director agents are spawned
|
||||
5. "Game concept is ready. May I write it to `design/gdd/game-concept.md`?" is
|
||||
asked, and the concept is written after approval; Verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All 4 skip notes appear with "solo mode" label
|
||||
- [ ] No director agents are spawned
|
||||
- [ ] Concept is written with only user approval
|
||||
- [ ] Behavior is otherwise equivalent to lean mode for this skill
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate — PR-SCOPE returns CONCERNS (scope too large)
|
||||
|
||||
**Fixture:**
|
||||
- Concept draft is complete
|
||||
- `production/session-state/review-mode.txt` contains `full`
|
||||
- PR-SCOPE gate returns CONCERNS: "The concept scope would require 18+ months for a solo developer"
|
||||
|
||||
**Input:** `/brainstorm`
|
||||
|
||||
**Expected behavior:**
|
||||
1. PR-SCOPE gate returns CONCERNS with specific scope feedback
|
||||
2. Skill surfaces the scope concerns to the user
|
||||
3. Scope concerns are documented in the concept draft before writing
|
||||
4. User is asked: reduce scope, accept concerns and document them, or rethink
|
||||
5. If concerns are accepted: concept is written with a "Scope Risk" note embedded
|
||||
|
||||
**Assertions:**
|
||||
- [ ] PR-SCOPE concerns are shown to the user before the "May I write" ask
|
||||
- [ ] Skill does NOT write concept without surfacing scope concerns
|
||||
- [ ] If user accepts: scope concerns are documented in the concept file
|
||||
- [ ] Skill does NOT auto-reject a concept due to PR-SCOPE CONCERNS (user decides)
|
||||
- [ ] All 4 skip notes appear, each naming its gate and "Solo mode" (not "Lean mode")
|
||||
- [ ] No `Agent` call is made for any director
|
||||
- [ ] The concept is written only after the user's approval
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Presents 2-4 concept options with pros/cons before user commits
|
||||
- [ ] User confirms concept direction before director gates are invoked
|
||||
- [ ] All 4 director gates spawn in parallel in full mode
|
||||
- [ ] All 4 gates skipped in lean AND solo mode — each noted by name
|
||||
- [ ] "May I write `design/gdd/game-concept.md`?" asked before writing
|
||||
- [ ] Ends with next-step handoff: `/map-systems`
|
||||
- [ ] Checks `workflow` before anything else and runs exactly one flow
|
||||
- [ ] Full flow presents 3 concepts before the user commits; Lean Brief flow presents 2–3 one-liners
|
||||
- [ ] Concept selection always asks, regardless of `modes.automation`
|
||||
- [ ] Gates spawn at their own points in full mode (CD-PILLARS + AD-CONCEPT-VISUAL together; TD-FEASIBILITY; PR-SCOPE)
|
||||
- [ ] In the full flow (`workflow: standard` / `full`), all 4 gates are skipped in lean and solo modes, each noted by name; the Lean Brief flow spawns no gate at all
|
||||
- [ ] Asks "May I write" before writing `design/game-brief.md` or `design/gdd/game-concept.md`
|
||||
- [ ] Ends with Verdict COMPLETE and the tier's next-step handoff
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- AD-CONCEPT-VISUAL gate (art director feasibility) is grouped with the other
|
||||
3 gates in the parallel spawn — not independently fixture-tested.
|
||||
- The iterative concept refinement loop (user rejects all options, skill
|
||||
generates new ones) is not fixture-tested — it follows the same pattern as
|
||||
the option selection phase.
|
||||
- The game-concept.md document structure (required sections) is defined in the
|
||||
skill body and not re-enumerated in test assertions.
|
||||
- TD-FEASIBILITY returning HIGH RISK (offer to revisit scope) is not separately tested.
|
||||
- A gate answering NOT ASSESSED (it lacked an input) is never an approval — the
|
||||
missing input is named and the gate re-run, or the result recorded as not
|
||||
assessed; not separately tested.
|
||||
- The concept refinement loop ("Generate fresh directions") is not fixture-tested.
|
||||
- Resuming an existing `design/game-brief.md` or `design/gdd/game-concept.md` is not separately tested.
|
||||
- `guided` / `autonomous` automation behaviour for the pillar and write loops is not tested here.
|
||||
|
||||
@@ -2,29 +2,48 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/bug-report` creates a structured bug report document from a user description.
|
||||
It produces a report with the following required fields: Title, Repro Steps,
|
||||
Expected Behavior, Actual Behavior, Severity (CRITICAL/HIGH/MEDIUM/LOW), Affected
|
||||
System(s), and Build/Version. If the user's initial description is missing any
|
||||
required field, the skill asks follow-up questions to fill the gaps before
|
||||
producing the draft.
|
||||
`/bug-report` has four modes, chosen from the argument: a plain description
|
||||
(**Description Mode**), `analyze [path]` (**Analyze Mode** — read code and
|
||||
report potential bugs), `verify [BUG-ID]` (**Verify Mode**) and `close [BUG-ID]`
|
||||
(**Close Mode**). With no argument it asks for a bug description first.
|
||||
|
||||
The skill checks for possibly duplicate reports (by comparing to existing files
|
||||
in `production/bugs/`) and offers to link rather than create a new report. Each
|
||||
report is written to `production/bugs/bug-[date]-[slug].md` after a "May I write"
|
||||
ask. No director gates are used — bug reporting is an operational utility.
|
||||
Description Mode parses the description, searches the codebase with Grep/Glob
|
||||
for related files, and drafts the report template: Title, ID `BUG-[NNNN]`,
|
||||
Severity on the `S1-Critical / S2-High / S3-Medium / S4-Low` scale, Priority
|
||||
on the `P1-Fix this sprint / P2-Fix soon / P3-Backlog / P4-Won't fix` ladder
|
||||
(which must match `/bug-triage`), Status `Open`, Classification (Category,
|
||||
System, Frequency, Regression — for a bug spanning systems, `System` is the one
|
||||
whose code must change), Environment (Build, Platform, Scene/Level),
|
||||
Reproduction Steps with Expected and Actual Result, and Technical Context
|
||||
(likely affected files, related systems). It presents the report and asks
|
||||
"May I write this to `production/qa/bugs/BUG-[NNNN].md`?" — `Verdict: **COMPLETE**`
|
||||
on write, `Verdict: **BLOCKED**` if declined. The ID is the highest existing
|
||||
number plus one; several reports in one run (Analyze Mode) take consecutive IDs,
|
||||
and one ask names every file. Next steps: `/bug-triage`, and
|
||||
`/hotfix [BUG-ID]` for S1/S2. No director gates are used.
|
||||
|
||||
Verify and Close Mode find the bug file by its number, whatever its padding —
|
||||
an older three-digit file with a slug included. Verify runs a related test with
|
||||
`commands.test` from `project.yaml` (never a runner line from memory). When no
|
||||
test ran — none covers the bug, `commands.test` is unset, or the run did not
|
||||
complete — or the bug is Visual or UI, it asks whether the user played the
|
||||
reproduction steps: `[No longer occurs]` → VERIFIED FIXED, recorded as a manual
|
||||
verification with the build; `[Still occurs]` → STILL PRESENT; `[Not played yet]`
|
||||
→ CANNOT VERIFY. It asks "May I update" before setting the Status; only a
|
||||
`Verified Fixed` bug can be closed, and Close Mode shows the closure record and
|
||||
asks before it writes anything.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keyword: COMPLETE
|
||||
- [ ] Contains "May I write" collaborative protocol language before writing the report
|
||||
- [ ] Has a next-step handoff (e.g., `/bug-triage` to reprioritize, `/hotfix` for critical)
|
||||
- [ ] Contains verdict keywords: COMPLETE, BLOCKED (and VERIFIED FIXED / STILL PRESENT / CANNOT VERIFY for Verify Mode)
|
||||
- [ ] Contains "May I write" language before writing the report and "May I update" before changing an existing bug file
|
||||
- [ ] Has a next-step handoff (`/bug-triage`; `/hotfix [BUG-ID]` for S1/S2; `/bug-report verify` → `close`)
|
||||
|
||||
---
|
||||
|
||||
@@ -39,136 +58,224 @@ None. `/bug-report` is an operational documentation skill. No director gates app
|
||||
### Case 1: Happy Path — User describes a crash, full report produced
|
||||
|
||||
**Fixture:**
|
||||
- `production/bugs/` directory exists and is empty
|
||||
- No similar existing reports
|
||||
- `production/qa/bugs/` does not exist yet
|
||||
- The code root contains a boss-arena scene script
|
||||
|
||||
**Input:** `/bug-report` (user describes: "Game crashes when player enters the boss arena")
|
||||
**Input:** `/bug-report The game crashes to desktop every time the player enters the boss arena — progress is blocked`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill extracts: Title = "Game crashes when entering boss arena"
|
||||
2. Skill recognizes crash reports as CRITICAL severity
|
||||
3. Skill confirms repro steps, expected (no crash), actual (crash), affected system
|
||||
(arena/boss), and build version with the user
|
||||
4. Skill drafts the full structured report
|
||||
5. Skill asks "May I write to `production/bugs/bug-2026-04-06-game-crashes-boss-arena.md`?"
|
||||
6. File is written on approval; verdict is COMPLETE
|
||||
1. Description Mode (no keyword)
|
||||
2. Skill parses the description and greps the codebase for boss-arena files
|
||||
3. Drafts the report template: Category `Crash`, Severity `S1-Critical`,
|
||||
Status `Open`, reproduction steps, Expected Result (no crash) and Actual
|
||||
Result (crash to desktop), likely affected files from the search
|
||||
4. Globs `production/qa/bugs/BUG-*.md`, finds none, allocates `BUG-0001`, presents the
|
||||
report and asks "May I write this to `production/qa/bugs/BUG-0001.md`?"
|
||||
5. On approval writes the file, creating `production/qa/bugs/`; Verdict is COMPLETE
|
||||
6. Next steps: `/bug-triage`, and `/hotfix [BUG-ID]` because the bug is S1
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All 7 required fields are present in the report
|
||||
- [ ] Severity is CRITICAL for a crash report
|
||||
- [ ] Filename follows the `bug-[date]-[slug].md` convention
|
||||
- [ ] "May I write" is asked with the full file path
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] Report contains Title, ID, Severity, Priority, Status, Category, System, Build, Reproduction Steps, Expected Result and Actual Result
|
||||
- [ ] Severity uses the S1–S4 scale (`S1-Critical` here) and Priority uses the P1–P4 labels
|
||||
- [ ] "Likely affected files" is filled from the codebase search
|
||||
- [ ] "May I write" names `production/qa/bugs/BUG-0001.md` — no bug files exist yet
|
||||
- [ ] Variant — with `BUG-0001.md` and `BUG-0003.md` already present, the new ID is
|
||||
`BUG-0004`: the highest existing number plus one, not the file count plus one
|
||||
- [ ] Verdict is COMPLETE after the write
|
||||
- [ ] `/hotfix [BUG-ID]` is suggested because severity is S1
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Minimal Input — Skill asks follow-up questions for missing fields
|
||||
### Case 2: No Argument — Asks for a description before drafting
|
||||
|
||||
**Fixture:**
|
||||
- User provides: "Sometimes the audio cuts out"
|
||||
- No existing reports
|
||||
- No existing bug reports
|
||||
|
||||
**Input:** `/bug-report`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill identifies missing required fields: repro steps, expected vs. actual,
|
||||
severity, affected system, build
|
||||
2. Skill asks targeted follow-up questions for each missing field (one at a time
|
||||
or in a structured prompt)
|
||||
3. User provides answers
|
||||
4. Skill compiles complete report from answers
|
||||
5. Skill asks "May I write?" and writes on approval
|
||||
1. No argument → skill asks the user for a bug description before proceeding
|
||||
2. No template is drafted and nothing is written until a description arrives
|
||||
3. Once the user answers ("Sometimes the audio cuts out"), Description Mode proceeds;
|
||||
unknown values (e.g. Build, Frequency) remain visible in the draft for the user to correct
|
||||
4. "May I write" is asked before the file is created
|
||||
|
||||
**Assertions:**
|
||||
- [ ] At least 3 follow-up questions are asked to fill missing fields
|
||||
- [ ] Each required field is filled before the report is finalized
|
||||
- [ ] Report is not written until all required fields are present
|
||||
- [ ] Verdict is COMPLETE after all fields are filled and file is written
|
||||
- [ ] Skill asks for a description instead of drafting from nothing
|
||||
- [ ] No report is drafted or written before the description is given
|
||||
- [ ] "May I write" precedes the write
|
||||
- [ ] Verdict is COMPLETE only after the file is written
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Possible Duplicate — Offers to link rather than create new
|
||||
### Case 3: Close Mode — Bug not yet verified is refused
|
||||
|
||||
**Fixture:**
|
||||
- `production/bugs/bug-2026-03-20-audio-cut-out.md` already exists with
|
||||
similar title and MEDIUM severity
|
||||
- `production/qa/bugs/BUG-0007.md` exists with `**Status**: Open`
|
||||
|
||||
**Input:** `/bug-report` (user describes: "Audio randomly stops working")
|
||||
**Input:** `/bug-report close BUG-0007`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill scans existing reports and finds the similar audio bug
|
||||
2. Skill reports: "A similar bug report exists: bug-2026-03-20-audio-cut-out.md"
|
||||
3. Skill presents options: link as duplicate (add note to existing), create new anyway
|
||||
4. If user chooses link: skill adds a cross-reference note to the existing file
|
||||
(asks "May I update the existing report?")
|
||||
5. If user chooses create new: normal report creation proceeds
|
||||
1. Close Mode reads `production/qa/bugs/BUG-0007.md`
|
||||
2. Status is not `Verified Fixed`, so the skill stops:
|
||||
"Bug BUG-0007 must be Verified Fixed before it can be closed. Run `/bug-report verify BUG-0007` first."
|
||||
3. No Closure Record is appended and the Status field is not changed
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Existing similar report is surfaced before creating a new one
|
||||
- [ ] User is given the choice (not forced to link or create)
|
||||
- [ ] If linking: "May I update" is asked before modifying the existing file
|
||||
- [ ] Verdict is COMPLETE in either path
|
||||
- [ ] The "must be Verified Fixed before it can be closed" message is shown
|
||||
- [ ] `/bug-report verify BUG-0007` is named as the next step
|
||||
- [ ] No Closure Record is written and `BUG-0007.md` is not edited
|
||||
- [ ] No "May I update" ask is issued (nothing to update)
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Multi-System Bug — Report created with multiple system tags
|
||||
### Case 4: Multi-System Bug — One report, primary and related systems
|
||||
|
||||
**Fixture:**
|
||||
- No existing reports
|
||||
- The code root has a save manager and a level-complete UI screen
|
||||
|
||||
**Input:** `/bug-report` (user describes: "After finishing a level, the save system
|
||||
freezes and the UI doesn't show the completion screen")
|
||||
**Input:** `/bug-report After finishing a level, the save system freezes and the UI doesn't show the completion screen`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill identifies 2 affected systems from the description: Save System and UI
|
||||
2. Report is drafted with both systems listed under Affected System(s)
|
||||
3. Severity is assessed (likely HIGH — data loss risk from save freeze)
|
||||
4. Skill asks "May I write" with the appropriate filename
|
||||
5. Report is written with both systems tagged; verdict is COMPLETE
|
||||
1. Description Mode drafts **one** report
|
||||
2. Classification `System` names the save system — its freeze is what must be
|
||||
fixed, and the missing completion screen follows from it; Technical Context
|
||||
"Related systems" lists the UI; "Likely affected files" lists files for both
|
||||
from the codebase search
|
||||
3. Skill asks "May I write" once for one `BUG-[NNNN].md`
|
||||
4. Verdict is COMPLETE after the write
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Both affected systems are listed in the report
|
||||
- [ ] Single report is created (not one per system)
|
||||
- [ ] Severity reflects the most impactful component (save freeze → HIGH or CRITICAL)
|
||||
- [ ] A single report is created (not one per system)
|
||||
- [ ] `System` is the save system, the one whose code must change; the UI is under Related systems
|
||||
- [ ] Likely affected files cover both systems
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; bug reporting is operational
|
||||
### Case 5: Director Gate Check — Analyze Mode, no gate
|
||||
|
||||
**Fixture:**
|
||||
- Any bug description provided
|
||||
- `src/save/save_manager.gd` exists and contains an unguarded null dereference and an unclosed file handle
|
||||
- `production/qa/bugs/` holds `BUG-0011.md` (the highest existing number)
|
||||
|
||||
**Input:** `/bug-report`
|
||||
**Input:** `/bug-report analyze src/save/save_manager.gd`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill creates and writes the bug report
|
||||
2. No director agents are spawned
|
||||
3. No gate IDs appear in output
|
||||
1. Analyze Mode reads the target file
|
||||
2. Identifies potential bugs (null reference, resource leak) and drafts one report
|
||||
per bug with the trigger scenario and a recommended fix
|
||||
3. Allocates consecutive IDs, `BUG-0012` and `BUG-0013`, one per report
|
||||
4. Presents the reports and asks once, naming both files: "May I write these to
|
||||
`production/qa/bugs/BUG-0012.md`, `production/qa/bugs/BUG-0013.md`?"
|
||||
5. No director agents are spawned; no gate IDs appear
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Skill reaches COMPLETE without any gate check
|
||||
- [ ] One report per potential bug, each with a trigger scenario and recommended fix
|
||||
- [ ] The two reports get different, consecutive IDs (`BUG-0012`, `BUG-0013`) — never the same ID twice
|
||||
- [ ] The single "May I write" ask names every file before any report is saved
|
||||
- [ ] No director gate is invoked and no gate skip messages appear
|
||||
- [ ] If the user declines, Verdict is BLOCKED and nothing is written
|
||||
|
||||
---
|
||||
|
||||
### Case 6: Verify Mode — older three-digit file found by its number, CANNOT VERIFY
|
||||
|
||||
**Fixture:**
|
||||
- `production/qa/bugs/` holds a bug file for bug 42 written in the older form:
|
||||
a three-digit number followed by a slug (`-save-corruption.md`), with
|
||||
`**Status**: Fixed — Pending Verification`
|
||||
- The bug's root-cause code path is gone from `src/save/`, but no test in
|
||||
`tests/` covers the save system, and the reproduction needs a console suspend
|
||||
that no automated check can drive
|
||||
|
||||
**Input:** `/bug-report verify BUG-0042`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Verify Mode globs `production/qa/bugs/BUG-*.md` and takes the file whose
|
||||
number is 42 — the three-digit, slugged file — instead of stopping on a
|
||||
missing `BUG-0042.md`
|
||||
2. It checks the root-cause code path (changed) and looks for a related test
|
||||
(none), then greps for the pattern recurring
|
||||
3. No test ran, so it asks via `AskUserQuestion` whether the user played the
|
||||
reproduction steps on a build with the fix — `[No longer occurs]` /
|
||||
`[Still occurs]` / `[Not played yet]`; the user answers `[Not played yet]`
|
||||
4. The verdict is **CANNOT VERIFY**: it names what is missing (no related test)
|
||||
and says that playing the reproduction steps settles it
|
||||
5. It asks "May I update `production/qa/bugs/[that file]` to set Status: … Cannot
|
||||
Verify?", naming the file it found
|
||||
6. It does not suggest `/bug-report close` — only a `Verified Fixed` bug can be
|
||||
closed
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The older three-digit file is found by its number; the skill does not report "No bug BUG-0042"
|
||||
- [ ] The manual-play question is asked, with the three answers, before a verdict is given
|
||||
- [ ] Verdict is CANNOT VERIFY, not VERIFIED FIXED — the changed code path is not taken as a pass, and "Not played yet" is not a pass
|
||||
- [ ] Playing the reproduction steps is named as what would settle it
|
||||
- [ ] The "May I update" ask names the file actually found, before the Status changes
|
||||
- [ ] Close is not offered while the verdict is CANNOT VERIFY
|
||||
|
||||
---
|
||||
|
||||
### Case 7: Verify then Close — no test covers the bug, the user's play settles it
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` has no `modes` block (so `qa.level: minimal` — tests are waived)
|
||||
and no `commands` block
|
||||
- `production/qa/bugs/BUG-0005.md`: Category `UI`, System `HUD`, "health bar does
|
||||
not update after healing", `**Status**: Fixed — Pending Verification`
|
||||
- No test in the engine's test root covers the HUD
|
||||
|
||||
**Input:** `/bug-report verify BUG-0005`, then `/bug-report close BUG-0005`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Verify finds no related test and runs no runner line of its own — `commands.test`
|
||||
is unset, and it says so
|
||||
2. It asks via `AskUserQuestion` whether the user played the reproduction steps
|
||||
on a build with the fix, and which build — `[No longer occurs]` /
|
||||
`[Still occurs]` / `[Not played yet]`; the user answers `[No longer occurs]`,
|
||||
build `0.3.1`
|
||||
3. Verdict **VERIFIED FIXED**, a manual verification; it asks "May I update
|
||||
`production/qa/bugs/BUG-0005.md` to set Status: Verified Fixed …?" and, on
|
||||
yes, writes the Status with `**Verified by**: [user], manual, build 0.3.1`
|
||||
4. Next step: `/bug-report close BUG-0005`
|
||||
5. Close reads `Verified Fixed`, shows the closure record (`Verified by` from the
|
||||
verify record, `Regression test: Manual verification`) and the Status change,
|
||||
and asks "May I update `production/qa/bugs/BUG-0005.md` to mark it Closed?"
|
||||
before appending anything or changing the Status
|
||||
6. On yes it appends the Closure Record and sets `**Status**: Closed`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No test command is written from memory; the unset `commands.test` is named
|
||||
- [ ] The manual-play question is asked with the three answers and the build is recorded as the user gave it
|
||||
- [ ] Verdict is VERIFIED FIXED, recorded as manual (`[user], manual, build 0.3.1`) — a bug no test covers can reach Verified Fixed
|
||||
- [ ] Close asks "May I update … to mark it Closed?" before the Closure Record is appended or the Status changed
|
||||
- [ ] The Closure Record's Regression test reads "Manual verification"
|
||||
- [ ] Variant — answering `[Still occurs]` gives STILL PRESENT (the bug is reopened and `/hotfix [BUG-ID]` suggested); answering `[Not played yet]` gives CANNOT VERIFY
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Collects all 7 required fields before drafting the report
|
||||
- [ ] Asks follow-up questions for any missing required fields
|
||||
- [ ] Checks for similar existing reports before creating a new one
|
||||
- [ ] Asks "May I write to `production/bugs/bug-[date]-[slug].md`?" before writing
|
||||
- [ ] Verdict is COMPLETE when the report file is written
|
||||
- [ ] Selects the mode from the argument; asks for a description when none is given
|
||||
- [ ] Uses the S1–S4 severity scale and the P1–P4 priority labels shared with `/bug-triage`
|
||||
- [ ] Verify runs a related test with `commands.test` from `project.yaml`, never a runner line written from memory
|
||||
- [ ] Verify says VERIFIED FIXED only when every check that applies passed — the related tests ran and passed, or, with no test run (or for a Visual or UI bug), the user played the reproduction steps on a named build and it no longer occurs, recorded as a manual verification; "Not played yet" with nothing else to settle it is CANNOT VERIFY
|
||||
- [ ] The Reporter field never holds an email address or account ID taken from the session
|
||||
- [ ] Searches the codebase for affected files before drafting
|
||||
- [ ] Asks "May I write this to `production/qa/bugs/BUG-[NNNN].md`?" before writing
|
||||
- [ ] Asks "May I update" before changing an existing bug file (Verify / Close); Close Mode shows the closure record and asks before appending it
|
||||
- [ ] Finds the bug file by its number in Verify and Close Mode, and never closes a bug that is not `Verified Fixed`
|
||||
- [ ] Verdict is COMPLETE on write, BLOCKED when the user declines
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- The case where the user provides a severity that seems too low for the
|
||||
described impact (e.g., LOW for a crash) is not tested; the skill may suggest
|
||||
a higher severity but ultimately respects user input.
|
||||
- Build/version field is required but may be "unknown" if the user doesn't know —
|
||||
this is accepted as a valid value and not tested separately.
|
||||
- Report slug generation (sanitizing the title into a filename) is an
|
||||
implementation detail not assertion-tested here.
|
||||
- Verify Mode on a bug whose related test runs and passes (VERIFIED FIXED with
|
||||
no manual question) is not separately tested; Case 6 covers CANNOT VERIFY,
|
||||
Case 7 a manual VERIFIED FIXED and, as variants, STILL PRESENT.
|
||||
- Close Mode's triage-report reference note is not separately tested.
|
||||
- Verify or close with a number no file carries ("No bug [BUG-ID] in
|
||||
`production/qa/bugs/`", Verdict BLOCKED) is not separately tested.
|
||||
|
||||
@@ -2,173 +2,192 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/bug-triage` reads all open bug reports in `production/bugs/` and produces a
|
||||
prioritized triage table sorted by severity (CRITICAL → HIGH → MEDIUM → LOW).
|
||||
It runs on the Haiku model (read-only, formatting/sorting task) and produces no
|
||||
file writes — the triage output is conversational. The skill flags bugs missing
|
||||
reproduction steps and identifies possible duplicates by comparing titles and
|
||||
affected systems.
|
||||
`/bug-triage` turns the open bug backlog into a prioritised, sprint-assigned
|
||||
action list and writes it to `production/qa/bug-triage-[date].md` after a
|
||||
"May I write" ask. It runs on the Sonnet model. Modes: `sprint` (assign fixable
|
||||
bugs to the current sprint, defer the rest), `full` (P1 → current sprint, P2 →
|
||||
next sprint, P3+ → backlog), `trend` (trend analysis only, no assignment, from
|
||||
header fields alone); no argument runs `sprint` if a sprint file exists, else `full`.
|
||||
|
||||
The verdict is always TRIAGED — the skill is advisory and informational. No
|
||||
director gates apply. The output is intended to help a producer or QA lead
|
||||
prioritize which bugs to address next.
|
||||
Bugs are discovered in `production/qa/bugs/*.md`, falling back to
|
||||
`production/qa/bugs.md`, then a `production/qa/qa-plan-*.md` "Bugs Found" table.
|
||||
Each bug gets a **severity** (S1 Critical – S4 Low, impact) and a **priority**
|
||||
(P1 Fix this sprint – P4 Won't fix / Deferred, urgency) — two separate axes, so
|
||||
the report groups bugs by priority, not severity. A deviation check flags
|
||||
systemic problems (3+ bugs in one system in a sprint; 2+ S1/S2 in one story; a
|
||||
bug filed against a Complete story). The verdict is `COMPLETE` when the report is
|
||||
written, `COMPLETE` with no report when there are no bug files to triage, and
|
||||
`BLOCKED` when the user declines the write. No director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keyword: TRIAGED
|
||||
- [ ] Does NOT contain "May I write" language (skill is read-only)
|
||||
- [ ] Has a next-step handoff (e.g., `/bug-report` to create new reports, `/hotfix` for critical bugs)
|
||||
- [ ] Contains verdict keywords: COMPLETE, BLOCKED
|
||||
- [ ] Contains "May I write" language before writing `production/qa/bug-triage-[date].md`
|
||||
- [ ] Has a next-step handoff (`/sprint-status` for unassigned S1s; `/smoke-check` after regressions)
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
None. `/bug-triage` is a read-only advisory skill. No director gates apply.
|
||||
None. `/bug-triage` is an operational triage skill. No director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — 5 bugs of varying severity, sorted table produced
|
||||
### Case 1: Happy Path — Sprint mode, 5 bugs grouped by priority
|
||||
|
||||
**Fixture:**
|
||||
- `production/bugs/` contains 5 bug report files:
|
||||
- bug-2026-03-10-audio-crash.md (CRITICAL)
|
||||
- bug-2026-03-12-score-overflow.md (HIGH)
|
||||
- bug-2026-03-14-ui-overlap.md (MEDIUM)
|
||||
- bug-2026-03-15-typo-tutorial.md (LOW)
|
||||
- bug-2026-03-16-vfx-flicker.md (HIGH)
|
||||
- `production/sprints/sprint-004.md` is the newest sprint file and notes spare capacity
|
||||
- `production/qa/bugs/` contains 5 bug files:
|
||||
- `BUG-0001.md` — S1, P1, Audio
|
||||
- `BUG-0002.md` — S2, P1, Combat
|
||||
- `BUG-0003.md` — S3, P2, UI
|
||||
- `BUG-0004.md` — S2, P2, VFX
|
||||
- `BUG-0005.md` — S4, P3, Tutorial
|
||||
|
||||
**Input:** `/bug-triage`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads all 5 bug report files
|
||||
2. Skill extracts severity, title, system, and repro status from each
|
||||
3. Skill produces a triage table sorted: CRITICAL first, then HIGH, MEDIUM, LOW
|
||||
4. Within the same severity, bugs are ordered by date (oldest first)
|
||||
5. Verdict is TRIAGED
|
||||
1. No argument and a sprint file exists → `sprint` mode
|
||||
2. Reads the 5 bug files and the newest sprint file
|
||||
3. Classifies severity and priority separately; P1 bugs are assigned to sprint 4
|
||||
4. Report has the Triage Summary table (P1: 2, P2: 2, P3: 1, P4: 0),
|
||||
"Critical (S1/S2) unfixed count: 3", and separate P1, P2 and P3/P4 tables
|
||||
5. Presents the report and asks "May I write this triage report to `production/qa/bug-triage-[date].md`?"
|
||||
6. Writes only after approval
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Triage table has exactly 5 rows
|
||||
- [ ] CRITICAL bug appears before both HIGH bugs
|
||||
- [ ] HIGH bugs appear before MEDIUM and LOW bugs
|
||||
- [ ] Verdict is TRIAGED
|
||||
- [ ] No files are written
|
||||
- [ ] Mode reported as `sprint`
|
||||
- [ ] BUG-0001 and BUG-0002 are in the "P1 Bugs — Fix This Sprint" table, assigned to the current sprint
|
||||
- [ ] BUG-0003 and BUG-0004 are in the P2 table; BUG-0005 is in the P3/P4 table
|
||||
- [ ] Triage Summary counts and the S1/S2 unfixed count (3) are correct
|
||||
- [ ] "May I write" names `production/qa/bug-triage-[date].md`, and nothing is written before approval
|
||||
|
||||
---
|
||||
|
||||
### Case 2: No Bug Reports Found — Guidance to run /bug-report
|
||||
### Case 2: No Bug Files Found — Stops without a report
|
||||
|
||||
**Fixture:**
|
||||
- `production/bugs/` directory exists but is empty (or does not exist)
|
||||
- `production/qa/bugs/` does not exist
|
||||
- No `production/qa/bugs.md` and no `production/qa/qa-plan-*.md`
|
||||
|
||||
**Input:** `/bug-triage`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill scans `production/bugs/` and finds no reports
|
||||
2. Skill outputs: "No open bug reports found in production/bugs/"
|
||||
3. Skill suggests running `/bug-report` to create a bug report
|
||||
4. No triage table is produced
|
||||
1. Step 2a finds no bug source in any of its three locations
|
||||
2. Skill says: "No bug files found in `production/qa/bugs/`. If bugs are tracked in a
|
||||
different location, adjust the glob pattern. If no bugs exist yet, there is nothing to triage."
|
||||
3. Skill stops — no classification, no triage report, no write — and ends
|
||||
`Verdict: COMPLETE — no bug files in production/qa/bugs/; nothing to triage.`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Output explicitly states no bugs were found
|
||||
- [ ] `/bug-report` is suggested as the next step
|
||||
- [ ] Skill does not error out — it handles empty directory gracefully
|
||||
- [ ] Verdict is TRIAGED (with "no bugs found" context)
|
||||
- [ ] Output states that no bug files were found in `production/qa/bugs/`
|
||||
- [ ] Skill stops gracefully rather than erroring
|
||||
- [ ] No triage tables are produced
|
||||
- [ ] No "May I write" ask and no file written
|
||||
- [ ] The stop still ends with a verdict: Verdict: COMPLETE — no bug files, nothing to triage
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Bug Missing Reproduction Steps — Flagged as NEEDS REPRO INFO
|
||||
### Case 3: Systemic Issues — Deviation check flags a hot spot and a regression
|
||||
|
||||
**Fixture:**
|
||||
- `production/bugs/` contains 3 bug reports; one has an empty "Repro Steps" section
|
||||
- `production/sprints/sprint-004.md` exists
|
||||
- `production/qa/bugs/` holds 4 bugs: 3 in the Inventory system filed this sprint,
|
||||
and 1 filed against `production/epics/core/story-003.md`, whose `Status: Complete`
|
||||
- The user approves the write when asked
|
||||
|
||||
**Input:** `/bug-triage`
|
||||
**Input:** `/bug-triage sprint`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads all 3 reports
|
||||
2. Skill detects the report with no repro steps
|
||||
3. That bug appears in the triage table with a `NEEDS REPRO INFO` tag
|
||||
4. Other bugs are triaged normally
|
||||
5. Verdict is TRIAGED
|
||||
1. Classifies all 4 bugs
|
||||
2. "Systemic Issues Flagged" lists "Potential design or implementation quality issue in Inventory"
|
||||
3. It also flags "Regression in completed story — story should be re-opened in sprint tracking"
|
||||
4. Trend Analysis names Inventory as the hot spot and counts 1 regression
|
||||
5. Asks "May I write this triage report to `production/qa/bug-triage-[date].md`?";
|
||||
the user approves and the report is written
|
||||
6. After writing, the skill adds: "Regressions found — consider re-opening the affected
|
||||
stories in sprint tracking and running `/smoke-check` to re-gate." Verdict: COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] `NEEDS REPRO INFO` tag appears next to the bug missing repro steps
|
||||
- [ ] The flagged bug is still included in the table (not excluded)
|
||||
- [ ] Other bugs are unaffected
|
||||
- [ ] Verdict is TRIAGED
|
||||
- [ ] Inventory is flagged as a potential quality issue (3+ bugs in one system)
|
||||
- [ ] The bug against the Complete story is flagged as a regression
|
||||
- [ ] Trend Analysis shows Inventory as the hot spot and "Regressions: 1"
|
||||
- [ ] The report is written only after the "May I write" ask is approved
|
||||
- [ ] The post-write message recommends `/smoke-check`, and the verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Possible Duplicate Bugs — Flagged in triage output
|
||||
### Case 4: Sprint at Capacity — Overflow flagged, Won't Fix asked, write declined
|
||||
|
||||
**Fixture:**
|
||||
- `production/bugs/` contains 2 bug reports with similar titles:
|
||||
- bug-2026-03-18-player-fall-through-floor.md
|
||||
- bug-2026-03-20-player-clips-through-floor.md
|
||||
- Both affect the "Physics" system with identical severity
|
||||
- `production/sprints/sprint-004.md` notes the sprint is at full capacity
|
||||
- `production/qa/bugs/` holds one S2/P1 bug and one S4 bug judged P4 (cosmetic, out of scope)
|
||||
|
||||
**Input:** `/bug-triage`
|
||||
**Input:** `/bug-triage sprint`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads both reports and detects similar title + same system + same severity
|
||||
2. Both bugs are included in the triage table
|
||||
3. Each is tagged with `POSSIBLE DUPLICATE` and cross-references the other report
|
||||
4. No bugs are merged or deleted — flagging is advisory
|
||||
5. Verdict is TRIAGED
|
||||
1. The P1 bug is not auto-assigned to the full sprint; it is flagged
|
||||
`Priority overflow — consider pulling from sprint`
|
||||
2. The S4 bug is surfaced as a P4 candidate with "Are these acceptable as Won't Fix?"
|
||||
— it is not marked Won't Fix without the user's answer
|
||||
3. User declines the report write
|
||||
4. Verdict is BLOCKED — user declined write; no file is created
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Both bugs appear in the table (not merged)
|
||||
- [ ] Both are tagged `POSSIBLE DUPLICATE`
|
||||
- [ ] Each cross-references the other (by filename or title)
|
||||
- [ ] Verdict is TRIAGED
|
||||
- [ ] P1 bug carries the `Priority overflow` flag instead of a sprint assignment
|
||||
- [ ] The Won't Fix question is asked before any bug is dispositioned P4
|
||||
- [ ] Verdict is BLOCKED when the write is declined
|
||||
- [ ] `production/qa/bug-triage-[date].md` is not written
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; triage is advisory
|
||||
### Case 5: Director Gate Check — Full mode, no P1 bugs
|
||||
|
||||
**Fixture:**
|
||||
- `production/bugs/` contains any number of reports
|
||||
- No sprint file in `production/sprints/`
|
||||
- `production/qa/bugs/` holds two bugs, S3/P3 and S4/P3
|
||||
|
||||
**Input:** `/bug-triage`
|
||||
**Input:** `/bug-triage full`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill produces the triage table
|
||||
2. No director agents are spawned
|
||||
3. No gate IDs appear in output
|
||||
4. No write tool is called
|
||||
1. Notes "No sprint plan found — assigning to backlog only."
|
||||
2. Both bugs go to the backlog (P3+)
|
||||
3. Report is written after "May I write" approval
|
||||
4. With no P1 bugs: "No P1 bugs — build is in good shape for QA hand-off." Verdict: COMPLETE
|
||||
5. No director agents are spawned; no gate IDs appear
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No write tool is called
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Verdict is TRIAGED without any gate check
|
||||
- [ ] "No sprint plan found — assigning to backlog only." is stated
|
||||
- [ ] Both bugs are dispositioned Backlog
|
||||
- [ ] "No P1 bugs — build is in good shape for QA hand-off." appears with Verdict COMPLETE
|
||||
- [ ] No director gate is invoked and no gate skip messages appear
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Reads all files in `production/bugs/` before generating the table
|
||||
- [ ] Sorts by severity (CRITICAL → HIGH → MEDIUM → LOW)
|
||||
- [ ] Flags bugs missing repro steps
|
||||
- [ ] Flags possible duplicates by title/system similarity
|
||||
- [ ] Does not write any files
|
||||
- [ ] Verdict is TRIAGED in all cases (even empty)
|
||||
- [ ] Discovers bugs in `production/qa/bugs/*.md` before the fallbacks
|
||||
- [ ] Keeps severity (S1–S4) and priority (P1–P4) as separate classifications
|
||||
- [ ] Never auto-assigns to a sprint at capacity
|
||||
- [ ] Never marks a bug Won't Fix without asking the user
|
||||
- [ ] Asks "May I write" before writing the triage report
|
||||
- [ ] Verdict is COMPLETE when written (or when there are no bug files to triage), BLOCKED when the write is declined
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- The case where a bug report is malformed (missing severity field entirely)
|
||||
is not fixture-tested; skill would flag it as `UNKNOWN SEVERITY` and sort it
|
||||
last in the table.
|
||||
- Status transitions (marking bugs as resolved) are outside this skill's scope —
|
||||
bug-triage is read-only.
|
||||
- The duplicate detection heuristic (title similarity + same system) is
|
||||
approximate; exact matching logic is defined in the skill body.
|
||||
- `trend` mode (header-field Grep only, no bug-body reads, no assignment) is not
|
||||
separately tested.
|
||||
- The consolidated `production/qa/bugs.md` and `qa-plan-*.md` fallbacks are not
|
||||
separately tested.
|
||||
- The "2+ S1/S2 bugs in the same story" deviation flag and the aged-bug
|
||||
warning (>2 sprints) are not separately tested.
|
||||
|
||||
@@ -2,174 +2,217 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/day-one-patch` prepares a day-one patch plan for issues that are known at
|
||||
launch but deferred from the v1.0 release. It reads open bug reports in
|
||||
`production/bugs/`, deferred acceptance criteria from story files (stories
|
||||
marked `Status: Done` but with noted deferred ACs), and produces a prioritized
|
||||
patch plan with estimated fix timelines per issue.
|
||||
`/day-one-patch` runs a fix-only mini-sprint for known issues found after the
|
||||
gold master: scope → rollback plan → fixes → lightweight QA gate → patch record.
|
||||
It runs only when the user types it (`disable-model-invocation: true`).
|
||||
|
||||
The patch plan is written to `production/releases/day-one-patch.md` after a
|
||||
"May I write" ask. If a P0 (critical post-ship) issue is discovered, the skill
|
||||
triggers guidance to run `/hotfix` before the patch. No director gates apply.
|
||||
The verdict is always COMPLETE.
|
||||
Phase 1 reads `project.stage` (`project.yaml`, fallback `production/stage.txt`)
|
||||
and requires `Release` or `Polish`; reads the most recent record in
|
||||
`production/gate-checks/` and **verifies the release gate passed** rather than
|
||||
inferring it from the stage; and loads bugs from `production/qa/bugs/*.md` with
|
||||
Status `Open` or `Fixed — Pending Verification`, plus the latest sprint and
|
||||
security audit. Phase 2 classifies each bug for inclusion (S1/S2, P1, fix under
|
||||
4 hours, data/config-only and cert requirements go in; architecture changes and
|
||||
new code paths are deferred to 1.1; S3/S4 only if a trivial config fix) and asks
|
||||
the user to approve the scope, with `[C] No day-one patch needed` as an exit.
|
||||
|
||||
Before any fix, `release-manager` drafts a rollback plan written to
|
||||
`production/releases/rollback-plan-[version].md`. Code fixes go through
|
||||
`lead-programmer`, which first returns each fix and the files it would change
|
||||
while writing nothing; the skill then asks once for the set — "May I apply these
|
||||
fixes? [BUG-ID → files]" — before any code changes, and `qa-tester` verifies each
|
||||
fix. Targeted and broader tests run with `commands.test` from `project.yaml`
|
||||
(unset → `Tests: NOT RUN` per fix, and the broader regression NOT ASSESSED).
|
||||
Config/data fixes are made directly, after a "May I edit" ask naming the file and
|
||||
the value change. `qa-lead` decides the QA scope (`/smoke-check`, or the affected
|
||||
systems' tests in the engine's test root); the patch QA verdict must be PASS or PASS WITH WARNINGS to
|
||||
proceed — FAIL defers the fix to 1.1 and NOT ASSESSED is never a pass. The patch
|
||||
record is written to `production/releases/day-one-patch-[version].md` after a
|
||||
"May I write" ask. No director gates are spawned.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Frontmatter sets `disable-model-invocation: true`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keyword: COMPLETE
|
||||
- [ ] Contains "May I write" collaborative protocol language before writing the plan
|
||||
- [ ] Has a next-step handoff (e.g., `/hotfix` for P0 issues, `/release-checklist` for follow-up)
|
||||
- [ ] Contains verdict keywords: PASS, PASS WITH WARNINGS, NOT ASSESSED, FAIL
|
||||
- [ ] Contains "May I write" language for both the rollback plan and the patch record, and "May I edit" for a direct config/data fix
|
||||
- [ ] Has a next-step handoff (`/patch-notes`, `/bug-report verify`, `/retrospective [milestone-name]` for the launch milestone)
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
None. `/day-one-patch` is a release planning utility. No director gates apply.
|
||||
None of the director-tier agents (creative-director, technical-director,
|
||||
producer, art-director) is spawned. The skill spawns working agents —
|
||||
`release-manager` (rollback plan), `lead-programmer` and `qa-tester` (fixes),
|
||||
`qa-lead` (patch QA scope) — and checks the existing release-gate record in
|
||||
`production/gate-checks/`; it does not run `/gate-check` itself.
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — 3 Known Issues, Patch Plan With Fix Estimates
|
||||
### Case 1: Happy Path — Scope, rollback plan first, fixes, patch record
|
||||
|
||||
**Fixture:**
|
||||
- `production/bugs/` contains 3 open bugs with severities: 1 MEDIUM, 2 LOW
|
||||
- No deferred ACs in sprint stories
|
||||
- All bugs have repro steps and system identifications
|
||||
- `project.yaml` has `project.stage: Release`
|
||||
- The most recent file in `production/gate-checks/` records a release-gate PASS
|
||||
- `production/qa/bugs/` holds three Open bugs:
|
||||
- `BUG-0010` — S2, P1, code fix estimated at 2 hours
|
||||
- `BUG-0011` — S3, a one-value config/data fix
|
||||
- `BUG-0012` — S2, fix requires an architecture change
|
||||
|
||||
**Input:** `/day-one-patch`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads all 3 open bugs
|
||||
2. Skill assigns fix effort estimates: MEDIUM bug = 1-2 days, LOW bugs = 4 hours each
|
||||
3. Skill produces a patch plan prioritizing MEDIUM bug first
|
||||
4. Plan includes: priority order, estimated timeline, responsible system, fix description
|
||||
5. Skill asks "May I write to `production/releases/day-one-patch.md`?"
|
||||
6. File written; verdict is COMPLETE
|
||||
1. Phase 1 confirms Release stage and a passing release gate
|
||||
2. Phase 2 proposes including BUG-0010 and BUG-0011 and deferring BUG-0012
|
||||
("Fix requires architecture change — defer to 1.1"); `AskUserQuestion` offers
|
||||
Approve / Adjust / No day-one patch needed
|
||||
3. After approval, `release-manager` drafts the rollback plan and the skill asks
|
||||
"May I write this rollback plan to `production/releases/rollback-plan-[version].md`?"
|
||||
— no fix work starts before it is written
|
||||
4. `lead-programmer` returns BUG-0010's minimal fix and the files it would change,
|
||||
writing nothing; the skill asks "May I apply these fixes? [BUG-0010 → files]",
|
||||
and only on yes does `lead-programmer` implement it and run targeted tests with
|
||||
`commands.test`; `qa-tester` verifies it. BUG-0011's config
|
||||
change is made directly without a programmer agent, after the skill asks
|
||||
"May I edit `[config file]` — `[key]`: [old value] → [new value]?"
|
||||
5. `qa-lead` sets the QA scope; the QA verdict is PASS
|
||||
6. Skill asks "May I write this patch record to `production/releases/day-one-patch-[version].md`?"
|
||||
7. Record lists BUG-0010 and BUG-0011 under Bugs Fixed and BUG-0012 under Deferred to 1.1 with its reason
|
||||
8. Next steps offer `/patch-notes`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All 3 bugs appear in the plan
|
||||
- [ ] Bugs are prioritized by severity (MEDIUM before LOW)
|
||||
- [ ] Fix estimates are provided per issue
|
||||
- [ ] "May I write" is asked before writing
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] BUG-0012 is deferred to 1.1 because it needs an architecture change
|
||||
- [ ] Scope is approved via `AskUserQuestion` before any work
|
||||
- [ ] The rollback plan is written before any fix is attempted
|
||||
- [ ] No code file changes before the one "May I apply these fixes?" ask is approved, and `lead-programmer` writes nothing while proposing
|
||||
- [ ] BUG-0011 is fixed without spawning `lead-programmer`
|
||||
- [ ] The config file is not edited before a "May I edit" ask naming the file and the value change is approved
|
||||
- [ ] "May I write" names `production/releases/day-one-patch-[version].md`
|
||||
- [ ] Next steps include `/patch-notes`
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Critical Issue Discovered Post-Ship — P0, Triggers /hotfix Guidance
|
||||
### Case 2: Critical Bug That Cannot Be Fixed Safely — Accepted risk, rollback trigger
|
||||
|
||||
**Fixture:**
|
||||
- A CRITICAL severity bug is found in `production/bugs/` after the v1.0 release
|
||||
- The bug causes data loss for all save files
|
||||
- Release stage; release-gate PASS on record
|
||||
- `production/qa/bugs/` holds `BUG-0020` — S1 (save-file data loss) whose fix needs
|
||||
an architecture change — and `BUG-0021` — S2, P1, 1-hour fix
|
||||
|
||||
**Input:** `/day-one-patch`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads bugs and identifies the CRITICAL severity issue
|
||||
2. Skill escalates: "P0 ISSUE DETECTED — data loss bug requires immediate hotfix
|
||||
before patch planning can proceed"
|
||||
3. Skill does NOT include the P0 issue in the patch plan timeline
|
||||
4. Skill explicitly directs: "Run `/hotfix` to resolve this issue first"
|
||||
5. After P0 guidance is issued: plan for remaining lower-severity bugs is still
|
||||
generated and written; verdict is COMPLETE
|
||||
1. BUG-0020 is S1 but its fix requires an architecture change, so it is proposed
|
||||
for deferral to 1.1; BUG-0021 is included
|
||||
2. The user approves the scope; the patch proceeds for BUG-0021
|
||||
3. After the patch record is written, the skill warns: "S1 bugs remain open and were
|
||||
not patched. These are accepted risks. Document them in the rollback plan trigger
|
||||
conditions — if they occur at scale, rollback may be preferable to a follow-up patch."
|
||||
|
||||
**Assertions:**
|
||||
- [ ] P0 escalation message appears prominently before the patch plan
|
||||
- [ ] `/hotfix` is explicitly directed for the P0 issue
|
||||
- [ ] P0 issue is NOT scheduled in the patch plan timeline (it needs immediate action)
|
||||
- [ ] Non-P0 issues are still planned; verdict is COMPLETE
|
||||
- [ ] BUG-0020 appears in the deferred table with its reason (architecture change)
|
||||
- [ ] BUG-0021 is still patched — the S1 does not block the rest of the patch
|
||||
- [ ] The "S1 bugs remain open" warning is shown after the record is written
|
||||
- [ ] The warning points to the rollback plan's trigger conditions
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Deferred AC From Story-Done — Pulled Into Patch Plan Automatically
|
||||
### Case 3: No Release-Gate Record — NOT ASSESSED, not assumed
|
||||
|
||||
**Fixture:**
|
||||
- `production/sprints/sprint-008.md` has a story with `Status: Done` and a note:
|
||||
"DEFERRED AC: Gamepad vibration on damage — deferred to post-launch patch"
|
||||
- No open bugs for the same system
|
||||
- `project.yaml` has `project.stage: Release`
|
||||
- `production/gate-checks/` is empty (no release-gate record)
|
||||
- Open bugs exist in `production/qa/bugs/`
|
||||
|
||||
**Input:** `/day-one-patch`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads sprint stories and detects the deferred AC note
|
||||
2. Deferred AC is automatically included in the patch plan as a work item
|
||||
3. Plan entry: "Deferred from sprint-008: Gamepad vibration on damage"
|
||||
4. Fix estimate is assigned; patch plan written after "May I write" approval
|
||||
5. Verdict is COMPLETE
|
||||
1. Phase 1 finds no release-gate record
|
||||
2. Reports `Release gate: NOT ASSESSED — no gate-check record found`
|
||||
3. States that the reduced QA scope depends on a gate nobody can find
|
||||
4. Asks whether to run `/gate-check` first or proceed with a full QA pass instead
|
||||
5. Does not proceed silently to scoping or fixes
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Deferred ACs from story files are automatically pulled into the plan
|
||||
- [ ] Deferred items are labeled by their source story (sprint-008)
|
||||
- [ ] Deferred AC gets a fix estimate like bug entries
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] `Release gate: NOT ASSESSED — no gate-check record found` is reported
|
||||
- [ ] `project.stage: Release` alone is not treated as proof the gate passed
|
||||
- [ ] User is offered `/gate-check` first or a full QA pass
|
||||
- [ ] No rollback plan, fix or patch record is produced before the user answers
|
||||
|
||||
---
|
||||
|
||||
### Case 4: No Known Issues — Empty Plan With Template Note
|
||||
### Case 4: Nothing Worth Patching — User selects "No day-one patch needed"
|
||||
|
||||
**Fixture:**
|
||||
- `production/bugs/` is empty
|
||||
- No stories have deferred ACs
|
||||
- Release stage; release-gate PASS on record
|
||||
- `production/qa/bugs/` holds only two S4 cosmetic bugs that need code changes
|
||||
|
||||
**Input:** `/day-one-patch`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads bugs — none found
|
||||
2. Skill reads story deferred ACs — none found
|
||||
3. Skill produces an empty patch plan with a note: "No known issues at launch"
|
||||
4. Template structure is preserved (headers intact) for future use
|
||||
5. Skill asks "May I write to `production/releases/day-one-patch.md`?"
|
||||
6. File written; verdict is COMPLETE
|
||||
1. Both S4 bugs are proposed for deferral (S3/S4 only if a trivial config fix)
|
||||
2. Scope question is asked; user selects `[C] No day-one patch needed`
|
||||
3. Skill outputs "No day-one patch required. Proceed to `/launch-checklist`." and stops
|
||||
4. No rollback plan, no agents, no patch record
|
||||
|
||||
**Assertions:**
|
||||
- [ ] "No known issues at launch" note appears in the written file
|
||||
- [ ] Template headers are present in the empty plan
|
||||
- [ ] Skill does NOT error out when there are no issues to plan
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] Both S4 bugs are listed as deferred, not included
|
||||
- [ ] On `[C]`, "No day-one patch required. Proceed to `/launch-checklist`." is shown
|
||||
- [ ] Skill stops without writing any file
|
||||
- [ ] No `release-manager`, `lead-programmer` or `qa-lead` is spawned
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; day-one-patch is a planning utility
|
||||
### Case 5: Director Gate Check — Patch QA NOT ASSESSED is not a pass
|
||||
|
||||
**Fixture:**
|
||||
- Known issues present in production/bugs/
|
||||
- `project.yaml`: `modes.rigor: standard` (so a project with no game tests gets NOT ASSESSED, not WAIVED)
|
||||
- Release stage; release-gate PASS on record; one included S2/P1 fix implemented
|
||||
- `qa-lead` chooses a targeted smoke check; `/smoke-check` returns
|
||||
`NOT ASSESSED — no game tests found`
|
||||
|
||||
**Input:** `/day-one-patch`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill generates and writes the patch plan
|
||||
2. No director agents are spawned
|
||||
3. No gate IDs appear in output
|
||||
1. The skill does not treat NOT ASSESSED as PASS or PASS WITH WARNINGS
|
||||
2. It either obtains a real result (following what the verdict says would make it
|
||||
runnable) or defers the fix to 1.1
|
||||
3. No director-tier agent is spawned at any point and `/gate-check` is not run
|
||||
4. The patch record, if written, shows the QA verdict and fills "Not assessed" with what could not be checked and why
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Verdict is COMPLETE without any gate check
|
||||
- [ ] NOT ASSESSED does not let the patch proceed as if it passed
|
||||
- [ ] The fix is either re-verified with a real result or deferred to 1.1
|
||||
- [ ] No creative-director, technical-director, producer or art-director is spawned
|
||||
- [ ] Any written record states the QA verdict and what was not assessed
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Reads open bugs from `production/bugs/` before generating the plan
|
||||
- [ ] Scans story files for deferred AC notes
|
||||
- [ ] Escalates CRITICAL (P0) bugs with explicit `/hotfix` guidance
|
||||
- [ ] Produces an empty plan with note when no issues exist (not an error)
|
||||
- [ ] Asks "May I write to `production/releases/day-one-patch.md`?" before writing
|
||||
- [ ] Verdict is COMPLETE in all paths
|
||||
- [ ] Confirms Release/Polish stage and verifies the release-gate record before scoping
|
||||
- [ ] Loads bugs from `production/qa/bugs/*.md`
|
||||
- [ ] Gets user approval of the scope before any work
|
||||
- [ ] Writes the rollback plan before any fix
|
||||
- [ ] Asks once — "May I apply these fixes?" — before `lead-programmer` changes any code
|
||||
- [ ] Runs tests with `commands.test`, never a runner line from memory; with the key unset, says so (the broader regression is NOT ASSESSED)
|
||||
- [ ] Requires PASS or PASS WITH WARNINGS to proceed; FAIL defers the fix; NOT ASSESSED is never a pass
|
||||
- [ ] Asks "May I write" before the rollback plan and before the patch record, and "May I edit" before a direct config/data change
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- The case where multiple CRITICAL bugs exist is handled the same as Case 2;
|
||||
all P0 issues are escalated together.
|
||||
- Timeline estimation for the patch (e.g., "patch available in 3 days")
|
||||
requires manual QA and build time estimates; the skill uses rough estimates
|
||||
based on severity, not actual team velocity.
|
||||
- The patch notes player communication document (`/patch-notes`) is a separate
|
||||
skill invoked after the patch plan is executed.
|
||||
- A stage other than Release/Polish ("Day-one patch prep is for Release-stage
|
||||
projects") is not separately tested.
|
||||
- A most-recent gate record of FAIL, CONCERNS or NOT ASSESSED (name it and stop)
|
||||
is not separately tested.
|
||||
- The >1 day scope warning and the `cert-feedback` scope argument are not separately tested.
|
||||
|
||||
@@ -2,27 +2,42 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/help` analyzes what has been done and what comes next in the project workflow.
|
||||
It runs on the Haiku model (read-only, formatting task) and reads `production/stage.txt`,
|
||||
the active sprint file, and recent session state to produce a concise situational
|
||||
guidance summary. The skill optionally accepts a context query (e.g., `/help testing`)
|
||||
to surface relevant skills for a specific topic.
|
||||
`/help` tells the user where they are in the pipeline and the one thing to do
|
||||
next. It is read-only and declares the Haiku tier (`model: haiku`), which is
|
||||
used only when the user types `/help` in Manual mode — in auto mode, or when
|
||||
Claude starts the skill itself, the session model runs it
|
||||
(`.claude/docs/model-tiers.md`). Before it runs, it is handed the resolved
|
||||
config block (`project.stage`, `workflow`) and the output of
|
||||
`.claude/scripts/story-status.sh`; it reads the session checkpoint itself (Step 3). It then reads
|
||||
`.claude/docs/workflow-catalog.yaml`, resolves step completion for the current
|
||||
phase with one call to `bash .claude/scripts/artifact-check.sh --phase [phase]`,
|
||||
and in Production reads `production/sprint-status.yaml`.
|
||||
|
||||
The output is always informational — no files are written and no director gates
|
||||
are invoked. The verdict is always HELP COMPLETE. The skill serves as a workflow
|
||||
navigator, suggesting 2-3 next skills based on the current project state.
|
||||
When `workflow` resolves to `minimal` (the default, because `modes.rigor`
|
||||
defaults to `minimal`) it skips the phase ladder, reads the session checkpoint
|
||||
(Step 3), and walks the minimal path (Step 4m): engine →
|
||||
`design/game-brief.md` → stories → `/dev-story` ↔ `/story-done`. The optional
|
||||
argument is **what the user just finished or is stuck
|
||||
on** — the skill uses it to advance past the named step, and confusion language
|
||||
in it adds an escalation footer. It is not a topic filter.
|
||||
|
||||
Output is a short "Where You Are" block with `✓ Done`, exactly one `→ Next up`,
|
||||
optional `~ Also available`, and `Coming up after that`. The verdict line is
|
||||
`Verdict: **COMPLETE** — next steps identified.` No files are written and no
|
||||
director gates are invoked.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Frontmatter sets `model: haiku`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keyword: HELP COMPLETE
|
||||
- [ ] Does NOT contain "May I write" language (skill is read-only)
|
||||
- [ ] Has a next-step handoff (suggests 2-3 relevant skills based on state)
|
||||
- [ ] Contains verdict keyword: COMPLETE
|
||||
- [ ] Does NOT contain "May I write" language, and `allowed-tools` has no Write or Edit (skill is read-only)
|
||||
- [ ] Has a next-step handoff (one `→ Next up` recommendation plus escalation paths)
|
||||
|
||||
---
|
||||
|
||||
@@ -37,136 +52,246 @@ None. `/help` is a read-only navigation skill. No director gates apply.
|
||||
### Case 1: Happy Path — Production stage with active sprint
|
||||
|
||||
**Fixture:**
|
||||
- `production/stage.txt` contains `Production`
|
||||
- `production/sprints/sprint-004.md` exists with in-progress stories
|
||||
- `production/session-state/active.md` has a recent checkpoint
|
||||
- `project.yaml` sets `project.stage: Production` and `modes.rigor: standard`
|
||||
(so `workflow` resolves to `standard`)
|
||||
- `production/sprints/sprint-004.md` exists
|
||||
- `production/sprint-status.yaml` lists one story `status: in-progress`, two
|
||||
`status: ready-for-dev`, three `status: done`, and one `status: blocked` with a
|
||||
`blocker` field
|
||||
- `production/session-state/active.md` has a STATUS block naming the in-progress story
|
||||
|
||||
**Input:** `/help`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads stage.txt and active sprint
|
||||
2. Skill identifies current sprint number and in-progress story count
|
||||
3. Skill outputs: current stage, sprint summary, and 3 suggested next skills
|
||||
(e.g., `/sprint-status`, `/dev-story`, `/story-done`)
|
||||
4. Suggestions are ranked by relevance to current sprint state
|
||||
5. Verdict is HELP COMPLETE
|
||||
1. Skill takes `project.stage: Production` and `workflow: standard` from the config block
|
||||
2. Skill reads `.claude/docs/workflow-catalog.yaml` and runs
|
||||
`bash .claude/scripts/artifact-check.sh --phase production`
|
||||
3. Skill reads `production/sprint-status.yaml`: the in-progress story is
|
||||
"currently active", ready-for-dev stories are "next up", done stories count as
|
||||
complete, the blocked story is named with its `blocker` text
|
||||
4. Because `active.md` shows an active task, it is surfaced prominently at the top:
|
||||
"It looks like you were working on [story]"
|
||||
5. Output uses the Step 7 shape headed `## Where You Are: Production`, with a
|
||||
single `→ Next up (REQUIRED)` step
|
||||
6. Verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Current stage is shown (Production)
|
||||
- [ ] Active sprint number and story count are mentioned
|
||||
- [ ] Exactly 2-3 next-skill suggestions are given (not a list of all skills)
|
||||
- [ ] Suggestions are appropriate for Production stage
|
||||
- [ ] Verdict is HELP COMPLETE
|
||||
- [ ] Heading reads `## Where You Are: Production`
|
||||
- [ ] `production/sprint-status.yaml` is read; the in-progress story is surfaced as currently active
|
||||
- [ ] The blocked story appears with its `blocker` field
|
||||
- [ ] Exactly one `→ Next up (REQUIRED)` step is shown (not a list of all skills)
|
||||
- [ ] The in-progress story from `active.md` is surfaced at the top ("It looks like you were working on …")
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] No files are written
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Concept Stage — Shows concept-to-systems-design workflow path
|
||||
### Case 2: Concept Stage — engine configured, no concept document
|
||||
|
||||
**Fixture:**
|
||||
- `production/stage.txt` contains `Concept`
|
||||
- No sprint files, no GDD files
|
||||
- `technical-preferences.md` is configured (engine selected)
|
||||
- `project.yaml` sets `project.stage: Concept`, `modes.rigor: full`, and
|
||||
`engine.name: "Godot"` (`full`, because at `standard` the art bible is
|
||||
required only when visual-asset stories exist — `.claude/docs/workflow-modes.md`
|
||||
— and a Concept-stage project has no stories to decide it by)
|
||||
- No `design/gdd/game-concept.md`, no `design/art/art-bible.md`, no
|
||||
`design/gdd/systems-index.md`, no sprint files
|
||||
|
||||
**Input:** `/help`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads stage.txt — detects Concept stage
|
||||
2. Skill outputs the Concept-stage workflow: brainstorm → map-systems → design-system
|
||||
3. Suggested skills are: `/brainstorm`, `/map-systems` (if concept exists)
|
||||
4. Current progress is noted: "Engine configured, concept not yet created"
|
||||
1. Skill maps `Concept` to the catalog phase `concept`
|
||||
2. `artifact-check.sh --phase concept` reports `engine-setup` PRESENT (matched by
|
||||
`engine.name` in `project.yaml`) and `game-concept`, `art-bible`, `map-systems` ABSENT
|
||||
3. `✓ Done` lists Engine Setup
|
||||
4. `→ Next up (REQUIRED)` is **Game Concept Document** with command `/brainstorm`
|
||||
5. `Coming up after that` lists Art Bible (`/art-bible`) and Systems Map (`/map-systems`)
|
||||
6. Verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Stage is identified as Concept
|
||||
- [ ] Workflow path shows the expected sequence for this stage
|
||||
- [ ] Suggestions do not include Production-stage skills (e.g., `/dev-story`)
|
||||
- [ ] Verdict is HELP COMPLETE
|
||||
- [ ] Heading reads `## Where You Are: Concept`
|
||||
- [ ] Engine Setup is shown under `✓ Done` (not reported missing)
|
||||
- [ ] `→ Next up` is Game Concept Document via `/brainstorm`
|
||||
- [ ] `/art-bible` and `/map-systems` appear under "Coming up after that", in that order
|
||||
- [ ] No Production-stage skill (e.g. `/dev-story`, `/sprint-plan`) is suggested
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 3: No stage.txt — Shows full workflow overview
|
||||
### Case 3: No stage recorded — phase inferred from artifacts
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` sets `modes.rigor: standard` and has no `project.stage`
|
||||
- No `production/stage.txt`
|
||||
- No sprint files
|
||||
- `technical-preferences.md` has placeholders
|
||||
- `design/gdd/game-concept.md` and `design/gdd/systems-index.md` exist
|
||||
- No `docs/architecture/adr-*.md`, no `production/epics/**/story-*.md`, code root empty
|
||||
|
||||
**Input:** `/help`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill cannot determine stage from stage.txt
|
||||
2. Skill runs project-stage-detect logic to infer stage from artifacts
|
||||
3. If stage cannot be inferred: outputs the full workflow overview from
|
||||
Concept through Release as a reference map
|
||||
4. Primary suggestion is `/start` to begin configuration
|
||||
1. Config block reports `project.stage` as not set
|
||||
2. Skill applies the Step 2 artifact inference (most-advanced match wins):
|
||||
`design/gdd/systems-index.md` exists and nothing further → `systems-design`
|
||||
3. Skill reports the inferred phase normally with the Step 7 shape
|
||||
4. `→ Next up (REQUIRED)` is System GDDs via `/design-system`. The catalog's
|
||||
`design-system` step has only a `note:` check, so `artifact-check.sh` reports
|
||||
it `NO_CHECK` with a note — a **MANUAL** step: the skill asks whether any
|
||||
system GDD is already done instead of asserting that none is
|
||||
5. The input has no confusion language, so the Step 9 escalation block is not shown
|
||||
6. Verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Skill does not crash when stage.txt is absent
|
||||
- [ ] Full workflow overview is shown when stage cannot be determined
|
||||
- [ ] `/start` or `/project-stage-detect` is a top suggestion
|
||||
- [ ] Verdict is HELP COMPLETE
|
||||
- [ ] Skill does not stop or error when no stage is recorded
|
||||
- [ ] Inferred phase is Systems Design (heading `## Where You Are: Systems Design`), not Concept
|
||||
- [ ] `→ Next up` is System GDDs via `/design-system`
|
||||
- [ ] The skill asks the user whether any system GDD is already done (MANUAL step), rather than reporting System GDDs as done or as not started
|
||||
- [ ] The "Need more detail?" escalation block (`/project-stage-detect`, `/gate-check`, `/start`) is NOT shown
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Context Query — User asks for help with testing
|
||||
### Case 4: Argument — user names the step they finished and says they are unsure
|
||||
|
||||
**Fixture:**
|
||||
- `production/stage.txt` contains `Production`
|
||||
- Active sprint has a story with `Status: In Review`
|
||||
- `project.yaml` sets `project.stage: Systems Design` and `modes.rigor: standard`
|
||||
- `design/gdd/systems-index.md` lists two MVP systems, both `Status: Approved`,
|
||||
and both system GDDs exist
|
||||
- No `design/gdd/gdd-cross-review-*.md`
|
||||
- The catalog's `design-review` step has no artifact check (`NO_CHECK`)
|
||||
|
||||
**Input:** `/help testing`
|
||||
**Input:** `/help just finished design-review, not sure what's next`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads context query: "testing"
|
||||
2. Skill surfaces skills relevant to testing: `/qa-plan`, `/smoke-check`,
|
||||
`/regression-suite`, `/test-setup`, `/test-evidence-review`
|
||||
3. Output is focused on testing workflow, not general sprint navigation
|
||||
4. Currently in-review story is highlighted as a testing candidate
|
||||
1. The argument is read as the step the user just finished (Step 5), not as a topic filter
|
||||
2. Skill advances past Per-System Design Review even though its artifact check is `NO_CHECK`
|
||||
3. `→ Next up (REQUIRED)` is Cross-GDD Review via `/review-all-gdds`
|
||||
4. "not sure" is confusion language, so the Step 9 "Need more detail?" block is
|
||||
appended with `/project-stage-detect`, `/gate-check` and `/start`
|
||||
5. The `/settings` rigor line is NOT shown: the user does not sound overwhelmed by
|
||||
process and the project has not outgrown `standard` (no `settings-guidance.md § 4` trigger fires)
|
||||
6. Verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Context query is acknowledged in output ("Help topic: testing")
|
||||
- [ ] At least 3 testing-relevant skills are listed
|
||||
- [ ] General sprint skills (e.g., `/sprint-plan`) are not the primary suggestions
|
||||
- [ ] Verdict is HELP COMPLETE
|
||||
- [ ] Per-System Design Review is treated as done because the user named it
|
||||
- [ ] `→ Next up` is Cross-GDD Review via `/review-all-gdds`
|
||||
- [ ] Escalation block lists `/project-stage-detect`, `/gate-check` and `/start`
|
||||
- [ ] No `/settings` rigor-change line appears
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; help is read-only navigation
|
||||
### Case 5: Director Gate Check — minimal path, no gate and no gate-check route
|
||||
|
||||
**Fixture:**
|
||||
- Any project state
|
||||
- `project.yaml` has `engine.name: "Godot"` and no `modes` block (`modes.rigor`
|
||||
defaults to `minimal`, so `workflow` resolves to `minimal`)
|
||||
- `design/game-brief.md` exists
|
||||
- `production/epics/core/story-001.md` is `Status: Complete`,
|
||||
`story-002.md` is `Status: In Progress`, `story-003.md` is `Status: Ready`
|
||||
- Neither `active.md` (its CHECKPOINT names story-002 as the current task, with
|
||||
**Next step:** `/dev-story` on it) nor the git log (no commit mentions
|
||||
story-002) says story-002's implementation is written
|
||||
|
||||
**Input:** `/help`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill produces workflow guidance summary
|
||||
2. No director agents are spawned
|
||||
3. No gate IDs appear in output
|
||||
4. No write tool is called
|
||||
1. `workflow: minimal` sends the skill past the rest of Step 2; `project.stage`
|
||||
is not mapped
|
||||
2. Skill reads `active.md`'s STATUS and CHECKPOINT blocks (Step 3), then goes to
|
||||
Step 4m
|
||||
3. Skill runs `bash .claude/scripts/artifact-check.sh --path minimal`
|
||||
4. The injected `story-status.sh` list has story-002 on the first `IN_PROGRESS`
|
||||
line; the skill does not re-run the script or re-rank the stories
|
||||
5. Output uses the minimal shape: `## Where You Are: Minimal path — 1 of 3 stories complete`,
|
||||
with `✓ Done` giving the count of complete stories, not their titles
|
||||
6. `→ Next up` is story-002 with `/dev-story [story-002 path]`
|
||||
7. No phase label, no gate line; the concept doc, art bible, systems map, GDDs,
|
||||
sprint plan and `/gate-check` are not presented as required
|
||||
8. No director agents are spawned and no write tool is called
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] Heading reads `Minimal path — 1 of 3 stories complete`
|
||||
- [ ] `active.md`'s checkpoint is read before the next step is chosen (Step 3 runs at `minimal`)
|
||||
- [ ] `→ Next up` is story-002 via `/dev-story`, not story-003
|
||||
- [ ] No `/gate-check` line and no "Approaching … gate" line appear
|
||||
- [ ] Game concept, art bible, systems map and GDDs are not listed as next steps
|
||||
- [ ] No director gate is invoked and no gate IDs appear in output
|
||||
- [ ] No write tool is called
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Verdict is HELP COMPLETE without any gate check
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 6: Minimal path — the checkpoint says the work is written
|
||||
|
||||
**Fixture:**
|
||||
- As Case 5, except `active.md`'s CHECKPOINT reads
|
||||
**Next step:** `/story-done production/epics/core/story-002.md` (what
|
||||
`/dev-story` writes when it finishes a story)
|
||||
|
||||
**Input:** `/help`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Step 3 runs at `minimal` and finds the checkpoint's next step
|
||||
2. story-002 is still on the first `IN_PROGRESS` line, but `active.md` says its
|
||||
implementation is written, so `→ Next up` is story-002 with
|
||||
`/story-done [story-002 path]`
|
||||
3. Verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] `→ Next up` is story-002 via `/story-done`, not `/dev-story`
|
||||
- [ ] story-002 is not counted or listed as complete
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 7: Minimal path — only a blocked story and an unknown status remain
|
||||
|
||||
**Fixture:**
|
||||
- As Case 5, except `story-002.md` is `Status: Blocked` (its file says it waits
|
||||
on the art for the player sprite) and `story-003.md` is `Status: Draft`
|
||||
|
||||
**Input:** `/help I'm stuck`
|
||||
|
||||
**Expected behavior:**
|
||||
1. The injected list has no `IN_REVIEW`, `IN_PROGRESS` or `TODO` line: story-002
|
||||
is on a `BLOCKED` line and story-003 on an `OTHER` line (`Draft`)
|
||||
2. The skill names story-002 with its blocker and story-003 with its status as
|
||||
written, and says nothing can be built next
|
||||
3. It does **not** say the build order is done, and does not offer the "three
|
||||
ways on" of the every-story-Complete branch
|
||||
4. `→ Next up` is clearing the first of them — story-002's blocker
|
||||
5. "stuck" is confusion language, so the Step 9 "Need more detail?" block is
|
||||
appended — without its `/gate-check` line, since the minimal path has no
|
||||
phase gates
|
||||
6. Verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] story-002 is named with its blocker; story-003 is named with status `Draft`
|
||||
- [ ] Neither story is recommended to `/dev-story` or `/story-done`
|
||||
- [ ] The output does not say the build order is done
|
||||
- [ ] Heading reads `Minimal path — 1 of 3 stories complete`
|
||||
- [ ] The escalation block appears and has no `/gate-check` line
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Reads stage, sprint, and session state before generating suggestions
|
||||
- [ ] Suggestions are specific to the current project state (not generic)
|
||||
- [ ] Context query (if provided) narrows the suggestion set
|
||||
- [ ] Uses the injected config block for `project.stage` and `workflow` instead of re-reading them
|
||||
- [ ] Resolves step completion with one `artifact-check.sh` call, not a Glob per step
|
||||
- [ ] Gives exactly one primary (`→ Next up`) recommendation
|
||||
- [ ] Never auto-runs the recommended skill
|
||||
- [ ] Asks the user about `MANUAL` steps rather than assuming them done or not done
|
||||
- [ ] Does not write any files
|
||||
- [ ] Verdict is HELP COMPLETE in all cases
|
||||
- [ ] Verdict is COMPLETE in all cases
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- The case where the active sprint is complete (all stories Done) is not
|
||||
separately tested; the skill would suggest `/sprint-plan` for the next sprint.
|
||||
- The `/help` skill does not validate whether suggested skills are available —
|
||||
it assumes standard skill catalog availability.
|
||||
- Stage detection fallback (when stage.txt is absent) delegates to the same
|
||||
logic as `/project-stage-detect` and is not re-tested here in detail.
|
||||
- The case where every story in `sprint-status.yaml` is done is not separately
|
||||
tested; the Step 8 gate warning would then apply.
|
||||
- At `workflow: full` every optional step is listed under `~ Also available`;
|
||||
Case 2 runs at `full` only so that the art bible is required, and does not
|
||||
assert that listing.
|
||||
- The minimal path's "every story Complete" branch (three ways on: play the
|
||||
build, `/create-stories`, `/settings`) is not separately tested.
|
||||
|
||||
@@ -2,172 +2,266 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/hotfix` manages an emergency fix workflow: it creates a hotfix branch from
|
||||
main, applies a targeted fix to the identified file(s), runs `/smoke-check` to
|
||||
validate the fix doesn't introduce regressions, and prompts the user to confirm
|
||||
merge back to main. Each code change requires a "May I write to [filepath]?" ask.
|
||||
Git operations (branch creation, merge) are presented as Bash commands for user
|
||||
confirmation before execution.
|
||||
`/hotfix` runs an emergency fix outside the sprint process, with an audit trail.
|
||||
It runs only when the user types it (`disable-model-invocation: true`). The flow:
|
||||
|
||||
The skill is time-sensitive — director review is optional post-hoc, not a
|
||||
blocking gate. Verdicts: HOTFIX COMPLETE (fix applied, smoke check passed, merged)
|
||||
or HOTFIX BLOCKED (fix introduced regression or user declined).
|
||||
1. **Assess severity** (S1 Critical / S2 High / S3 or lower, on `/bug-triage`'s scale) and confirm it with
|
||||
`AskUserQuestion`; S3 or lower stops with `Verdict: **REDIRECTED**`.
|
||||
2. **Hotfix record** — asks "May I write this to
|
||||
`production/hotfixes/hotfix-[date]-[short-name].md`?"
|
||||
3. **Branch** — checks for a git repo, then asks before running
|
||||
`git checkout -b hotfix/[short-name] [base-ref]`. `[base-ref]` is the
|
||||
release the bug is in: its tag on `main`, or its `release/*` branch if one
|
||||
was cut to stabilise it — never the head of `main`.
|
||||
4. **Investigate and propose** — root cause and minimal fix, then
|
||||
"May I implement this fix?" before any code changes; implement, run targeted tests.
|
||||
5. **Approvals** — `lead-programmer`, `qa-tester` and `producer` spawned in
|
||||
parallel; all three must return APPROVE, and CONCERNS or REJECT blocks deployment.
|
||||
6. **QA re-entry gate** — `qa-lead` picks smoke check, targeted
|
||||
`/team-qa feature: [system]`, or full QA; a smoke-check FAIL sends the fix
|
||||
back to Phase 4. A NOT ASSESSED result is not a pass: deploying anyway takes
|
||||
an explicit producer decision, logged in the hotfix record.
|
||||
7. **Deploy** — an unconditional `AskUserQuestion` before any merge, tag or deploy
|
||||
(even in `autonomous` mode); merge to `main`, and to the release's
|
||||
`release/*` branch only if one exists; tag the patch release and verify the
|
||||
backport; ask before updating the bug file, then set it to
|
||||
`Fixed — Pending Verification`; print "Hotfix Deployed" (on `[B]`,
|
||||
"Hotfix Merged to main — release held", with no Tag); `[C]` stops with no
|
||||
merge, tag or bug-file update; then `/bug-report verify` (a CANNOT VERIFY is
|
||||
settled by playing the reproduction steps and answering its manual-play
|
||||
question) and a post-incident review (recorded with the incident-response
|
||||
template — `/retrospective` has no incident mode).
|
||||
|
||||
Targeted tests run with `commands.test` from `project.yaml`; when it is unset or
|
||||
no test covers the system, the hotfix record says `Tests: NOT RUN — [reason]`.
|
||||
A `/team-qa` pass that returns NOT APPROVED or BLOCKED is handled like a
|
||||
smoke-check FAIL. S1 is `/bug-triage`'s definition: crash, data loss, or
|
||||
complete feature failure.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Frontmatter sets `disable-model-invocation: true`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keywords: HOTFIX COMPLETE, HOTFIX BLOCKED
|
||||
- [ ] Contains "May I write" language for code changes
|
||||
- [ ] Has a next-step handoff (e.g., `/bug-report` to document the issue, or version bump)
|
||||
- [ ] Contains verdict keywords: REDIRECTED, APPROVE / CONCERNS / REJECT (sign-offs), NOT ASSESSED (QA gate)
|
||||
- [ ] Contains "May I write" (hotfix record), "May I implement this fix?" (code) and "May I update" (bug file) language
|
||||
- [ ] Has a next-step handoff (`/bug-report verify [BUG-ID]`, a post-incident review)
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
None. Hotfixes are time-critical. Director review may follow separately as a
|
||||
post-hoc step. No gate is invoked within this skill.
|
||||
No named director-gate ID (CD-*, TD-*, AD-*, PR-*) is used and the review mode
|
||||
does not apply. Phase 5 spawns three **blocking sign-offs** in parallel —
|
||||
`lead-programmer`, `qa-tester` and `producer` (the only director-tier agent) —
|
||||
and all three must return APPROVE before deployment. Phase 5b spawns `qa-lead` to
|
||||
choose the QA scope.
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — Critical crash bug fixed, smoke check passes
|
||||
### Case 1: Happy Path — S1 crash fixed with record, branch, approvals and backport
|
||||
|
||||
**Fixture:**
|
||||
- `main` branch is clean
|
||||
- Bug is identified in `src/gameplay/arena.gd` (crash on boss arena entry)
|
||||
- Repro steps are provided by user
|
||||
- Git repository; the live build is the release tag `v1.2.0` on `main`, and no
|
||||
`release/*` branch exists (the trunk default)
|
||||
- Bug in `src/gameplay/arena.gd` (crash on boss arena entry);
|
||||
`production/qa/bugs/BUG-0031.md` exists with `**Status**: Open`
|
||||
- All three sign-offs return APPROVE; `qa-lead` picks a smoke check, which returns PASS
|
||||
|
||||
**Input:** `/hotfix` (user describes the crash and affected file)
|
||||
**Input:** `/hotfix BUG-0031`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill proposes creating a hotfix branch: `hotfix/boss-arena-crash`
|
||||
2. User confirms; Bash command for branch creation is shown and confirmed
|
||||
3. Skill identifies the fix location in `arena.gd` and drafts the change
|
||||
4. Skill asks "May I write to `src/gameplay/arena.gd`?" and applies fix on approval
|
||||
5. Skill runs `/smoke-check` — PASS
|
||||
6. Skill presents the merge command and asks user to confirm merge to `main`
|
||||
7. User confirms; merge executes; verdict is HOTFIX COMPLETE
|
||||
1. Assesses S1 and confirms via `AskUserQuestion`; user picks `[A] S1 (Critical)`
|
||||
2. Asks "May I write this to `production/hotfixes/hotfix-[date]-boss-arena-crash.md`?" and writes it
|
||||
3. Runs `git rev-parse --is-inside-work-tree`, asks "Ready to create hotfix branch
|
||||
'hotfix/boss-arena-crash' from v1.2.0?", and runs
|
||||
`git checkout -b hotfix/boss-arena-crash v1.2.0` only after `[A] Yes — create branch`
|
||||
4. Presents root cause and minimal fix, asks "May I implement this fix?", then implements and runs targeted tests
|
||||
5. Spawns `lead-programmer`, `qa-tester` and `producer` in parallel; all APPROVE
|
||||
6. `qa-lead` → smoke check PASS
|
||||
7. Asks "Hotfix validated and approved. Merge to `main`, tag v1.2.1 and deploy?"
|
||||
(no release branch is named — none exists); user picks `[A]`
|
||||
8. Merges to `main`, tags `v1.2.1` on the fixed build, and confirms the fix is on `main`
|
||||
9. Asks "May I update `production/qa/bugs/BUG-0031.md` …?" before setting it to
|
||||
`Fixed — Pending Verification`, then prints "Hotfix Deployed: boss-arena-crash"
|
||||
with `Tag: v1.2.1` and `Next: /bug-report verify`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Hotfix branch is created before any code changes
|
||||
- [ ] "May I write" is asked before modifying any source file
|
||||
- [ ] `/smoke-check` runs after the fix is applied
|
||||
- [ ] Merge requires explicit user confirmation (not automatic)
|
||||
- [ ] Verdict is HOTFIX COMPLETE after successful merge
|
||||
- [ ] The hotfix record is written before the branch is created
|
||||
- [ ] The branch starts from the release tag `v1.2.0`, not from the head of `main`
|
||||
- [ ] The branch is created only after the user picks `[A]`
|
||||
- [ ] No code is modified before "May I implement this fix?" is approved
|
||||
- [ ] The three sign-off agents are spawned as simultaneous `Agent` calls
|
||||
- [ ] No merge happens before the Phase 6 deploy question is answered
|
||||
- [ ] No `release/*` branch is created or merged; the patch release is tagged
|
||||
- [ ] The summary's Backport line states presence on `main`
|
||||
- [ ] The bug file is not edited before the user approves the "May I update" question
|
||||
- [ ] `BUG-0031.md` Status becomes `Fixed — Pending Verification`
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Smoke Check Fails — HOTFIX BLOCKED
|
||||
### Case 2: Sign-off REJECT — Deployment blocked
|
||||
|
||||
**Fixture:**
|
||||
- Fix has been applied to `src/gameplay/arena.gd`
|
||||
- Fix implemented for an S1 bug
|
||||
- `qa-tester` returns REJECT: "Player health clamping regression detected";
|
||||
`lead-programmer` and `producer` return APPROVE
|
||||
|
||||
**Input:** `/hotfix`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Phase 5 collects all three responses
|
||||
2. Because one returned REJECT, the skill does not deploy: it surfaces the regression and resolves it first
|
||||
3. The Phase 5b QA gate and the Phase 6 deploy question are not reached while the REJECT stands
|
||||
4. No merge, tag or deploy occurs
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The qa-tester's REJECT and its regression are shown to the user
|
||||
- [ ] Skill does not proceed to deployment with a REJECT outstanding
|
||||
- [ ] The deploy `AskUserQuestion` is not issued while the REJECT stands
|
||||
- [ ] No merge is performed
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Release Branch — Deploy approval holds even in autonomous mode
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` sets `modes.automation: autonomous`
|
||||
- The live release `v1.3.0` has a `release/1.3` branch, cut to stabilise it for
|
||||
certification; the hotfix branch was started from `release/1.3`
|
||||
- An S2 fix has all three APPROVE sign-offs and a smoke check PASS
|
||||
|
||||
**Input:** `/hotfix`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Despite `autonomous`, Phase 6 asks "Hotfix validated and approved. Merge to `main`
|
||||
and `release/1.3`, tag v1.3.1 and deploy?" with `[A] Yes — merge and deploy` /
|
||||
`[B] Merge to main only — hold the release` / `[C] Stop here`
|
||||
2. User picks `[B]`
|
||||
3. The fix is merged to `main` only; nothing is merged to `release/1.3`, tagged or deployed
|
||||
4. The summary is titled "Hotfix Merged to main — release held" and has no Tag line
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The deploy question is asked even though `modes.automation` is `autonomous`
|
||||
- [ ] The question names `release/1.3`, because this release has one
|
||||
- [ ] All three options are offered
|
||||
- [ ] With `[B]`, `release/1.3` is not merged, no tag is created and nothing is deployed
|
||||
- [ ] With `[B]`, `main` receives the fix
|
||||
- [ ] With `[B]`, the summary does not say "Deployed" and names no tag
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Not an Emergency — a cosmetic S4 redirected before any work
|
||||
|
||||
**Fixture:**
|
||||
- User reports a typo on the credits screen
|
||||
|
||||
**Input:** `/hotfix typo in the credits screen`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Phase 1 assesses S4 (Low — cosmetic) and asks the user to confirm severity
|
||||
2. User picks `[C] S3 or lower — redirect to normal bug fix workflow`
|
||||
3. Skill stops: Verdict REDIRECTED — use the normal bug fix workflow for S3 and below
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Severity is confirmed via `AskUserQuestion` with S1 / S2 / S3-or-lower options
|
||||
- [ ] Verdict is REDIRECTED
|
||||
- [ ] No hotfix record is written and no branch is created
|
||||
- [ ] No code is modified and no agents are spawned
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — Producer sign-off, and NOT ASSESSED QA is not a pass
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml`: `modes.rigor: standard` (so a project with no game tests gets NOT ASSESSED, not WAIVED)
|
||||
- S1 fix implemented; all three sign-offs APPROVE
|
||||
- `qa-lead` picks a smoke check; `/smoke-check` returns `NOT ASSESSED — no game tests found`
|
||||
|
||||
**Input:** `/hotfix`
|
||||
|
||||
**Expected behavior:**
|
||||
1. `producer` is spawned as a blocking sign-off, not as a named director gate; no gate IDs appear
|
||||
2. The NOT ASSESSED QA result is treated as unmet, not as a pass
|
||||
3. The skill either obtains the missing result or takes the decision to the producer
|
||||
explicitly as a decision to deploy an unverified hotfix. This escape hatch is
|
||||
deliberate: an emergency fix may have to ship before a suite exists, but only
|
||||
as a named decision, never as a gate that quietly cleared
|
||||
4. If the producer decides to deploy unverified, that decision — who made it and
|
||||
why — is logged in the hotfix record, and the summary's QA gate line reads
|
||||
NOT ASSESSED with why and whose explicit decision it was
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No gate IDs (CD-*, TD-*, AD-*, PR-*) appear in output
|
||||
- [ ] NOT ASSESSED does not advance to Phase 6 as if the QA gate passed
|
||||
- [ ] Deployment goes ahead only with the three APPROVE sign-offs plus an explicit producer decision to deploy unverified
|
||||
- [ ] That decision is logged in the hotfix record
|
||||
- [ ] The summary records the QA gate as NOT ASSESSED with the reason and whose decision it was
|
||||
|
||||
---
|
||||
|
||||
### Case 6: Smoke Check FAIL — back to Phase 4, and a CONCERNS sign-off still blocks
|
||||
|
||||
**Fixture:**
|
||||
- S1 fix implemented; all three sign-offs APPROVE; `qa-lead` picks a smoke check
|
||||
- `/smoke-check` returns FAIL: "Player health clamping regression detected"
|
||||
- On the revised fix, `lead-programmer` returns CONCERNS ("the clamp now runs on
|
||||
the save path too"); `qa-tester` and `producer` return APPROVE
|
||||
|
||||
**Input:** `/hotfix`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill applies the fix and runs `/smoke-check`
|
||||
2. Smoke check returns FAIL with specific regression identified
|
||||
3. Skill reports: "HOTFIX BLOCKED — smoke check failed: [regression detail]"
|
||||
4. Skill presents options: attempt revised fix, revert changes, or merge with
|
||||
known regression (user acknowledges risk)
|
||||
5. No automatic merge occurs when smoke check fails
|
||||
1. The FAIL stops deployment: the skill returns to Phase 4 with the failing check,
|
||||
presents a revised fix and asks "May I implement this fix?" again before
|
||||
changing code
|
||||
2. After implementing it (Phase 4b), it re-runs all three Phase 5 sign-offs —
|
||||
qa-tester's regression run among them — and then the QA step
|
||||
3. `lead-programmer`'s CONCERNS blocks exactly as a REJECT does: it is shown to the
|
||||
user and resolved before the skill continues
|
||||
4. The Phase 6 deploy question is asked only once all three return APPROVE and a
|
||||
re-run smoke check passes
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Verdict is HOTFIX BLOCKED
|
||||
- [ ] Smoke check failure is shown verbatim to user
|
||||
- [ ] Merge is NOT performed automatically when smoke check fails
|
||||
- [ ] User is given explicit options for how to proceed
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Fix to Already-Released Build — Version tag noted, patch bump prompted
|
||||
|
||||
**Fixture:**
|
||||
- Latest git tag is `v1.2.0`
|
||||
- Hotfix targets a bug in the v1.2.0 release
|
||||
|
||||
**Input:** `/hotfix`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill detects that the current HEAD is a tagged release (v1.2.0)
|
||||
2. Skill notes: "Hotfix targeting tagged release v1.2.0"
|
||||
3. After smoke check passes, skill prompts: "Should version be bumped to v1.2.1?"
|
||||
4. If user confirms version bump: skill asks "May I write to VERSION or equivalent?"
|
||||
5. After version update and merge: verdict is HOTFIX COMPLETE with version noted
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Version tag context is detected and surfaced to user
|
||||
- [ ] Patch version bump is suggested (not required) after merge
|
||||
- [ ] Version bump requires its own "May I write" confirmation
|
||||
- [ ] Verdict is HOTFIX COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 4: No Repro Steps — Skill Asks Before Applying Fix
|
||||
|
||||
**Fixture:**
|
||||
- User invokes `/hotfix` with a vague description: "something is broken on level 3"
|
||||
- No repro steps provided
|
||||
|
||||
**Input:** `/hotfix` (vague description)
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill detects insufficient information to identify the fix location
|
||||
2. Skill asks: "Please provide reproduction steps and the affected file or system"
|
||||
3. Skill does NOT create a branch or modify any file until repro steps are provided
|
||||
4. After user provides repro steps: normal hotfix flow begins
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No branch is created without repro steps
|
||||
- [ ] No code changes are made without a clearly identified fix location
|
||||
- [ ] Repro step request is specific (not a generic "please provide more info")
|
||||
- [ ] Normal hotfix flow resumes after user provides repro steps
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; hotfixes are time-critical
|
||||
|
||||
**Fixture:**
|
||||
- Critical bug with repro steps identified
|
||||
|
||||
**Input:** `/hotfix`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill completes the hotfix workflow
|
||||
2. No director agents are spawned during execution
|
||||
3. No gate IDs appear in output
|
||||
4. Post-hoc director review (if needed) is a manual follow-up, not invoked here
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Verdict is HOTFIX COMPLETE or HOTFIX BLOCKED — no gate verdict
|
||||
- [ ] A smoke-check FAIL never reaches the deploy question
|
||||
- [ ] The revised fix gets its own "May I implement this fix?" approval
|
||||
- [ ] All three sign-offs and the QA step are re-run for the revised fix
|
||||
- [ ] CONCERNS from a sign-off blocks deployment until resolved; it is not treated as an approval
|
||||
- [ ] No merge, tag or deploy occurs while the FAIL or the CONCERNS stands
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Creates hotfix branch before making any code changes
|
||||
- [ ] Asks "May I write" before modifying any source files
|
||||
- [ ] Runs `/smoke-check` after applying the fix
|
||||
- [ ] Requires explicit user confirmation before merging
|
||||
- [ ] HOTFIX BLOCKED when smoke check fails — no automatic merge
|
||||
- [ ] Verdict is HOTFIX COMPLETE or HOTFIX BLOCKED
|
||||
- [ ] Confirms severity before any work; S3 or lower is REDIRECTED
|
||||
- [ ] Asks "May I write" before the hotfix record
|
||||
- [ ] Asks before creating the branch
|
||||
- [ ] Asks "May I implement this fix?" before modifying code
|
||||
- [ ] Runs targeted tests with `commands.test`, never a runner line from memory; with the key unset (or no covering test) records `Tests: NOT RUN — [reason]`
|
||||
- [ ] Treats a `/team-qa` NOT APPROVED or BLOCKED like a smoke-check FAIL
|
||||
- [ ] On the deploy question's `[C] Stop here`, merges, tags and updates nothing, and the record stays IN PROGRESS
|
||||
- [ ] Deploys only with all three APPROVE sign-offs and a QA gate that ran — or,
|
||||
when the QA gate was NOT ASSESSED, an explicit producer decision to deploy
|
||||
unverified, logged in the hotfix record
|
||||
- [ ] Asks before any merge, tag or deploy regardless of `modes.automation`
|
||||
- [ ] Starts the hotfix branch from the release it fixes (its tag, or its `release/*` branch if one exists)
|
||||
- [ ] Verifies the backport on `main`, and on the release's `release/*` branch only when one exists
|
||||
- [ ] Asks "May I update" before editing the bug file
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- The case where multiple files need to be modified for one fix follows the same
|
||||
"May I write" per-file pattern and is not separately tested.
|
||||
- The post-hotfix steps (create bug report, update changelog) are suggested in
|
||||
the handoff but not tested as part of this skill's execution.
|
||||
- Conflict resolution during the merge (if main has diverged) is not tested;
|
||||
the skill would surface the conflict and ask the user to resolve it manually.
|
||||
- A non-git working directory ("Not a git repository — create the branch manually.")
|
||||
is not separately tested.
|
||||
- `qa-lead` choosing a targeted `/team-qa feature: [system]` pass (APPROVED /
|
||||
APPROVED WITH CONDITIONS) or full QA is not separately tested.
|
||||
- Phase 7 post-deploy verification (VERIFIED FIXED → `/bug-report close`; STILL
|
||||
PRESENT → re-open and assess rollback; CANNOT VERIFY → the bug stays open until
|
||||
its reproduction steps are played in the deployed build and reported through
|
||||
`/bug-report verify`'s manual-play question) is not separately tested.
|
||||
|
||||
@@ -2,179 +2,265 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/launch-checklist` generates and evaluates a complete launch readiness checklist
|
||||
covering: legal compliance (EULA, privacy policy, ESRB/PEGI ratings), platform
|
||||
certification status, store page completeness (screenshots, description, metadata),
|
||||
build validation (version tag, reproducible build), analytics and crash reporting
|
||||
configuration, and first-run experience verification.
|
||||
`/launch-checklist` generates a launch-readiness checklist across code, content,
|
||||
QA, store, legal, infrastructure, community and operations, scoped to the
|
||||
project. It is handed the resolved `rigor`, `project.stage`, `platform.cert_tier`
|
||||
and `automation`, and scopes by them: out-of-reach phases are declared out of
|
||||
scope, `rigor: minimal` drops process-weight items, and the Platform
|
||||
Certification section is emitted **only** for the `cert_tier` value
|
||||
(`none | itch | steam | console`). An unset `cert_tier` is a question, not a
|
||||
licence to emit every track; if it cannot be determined the section is marked
|
||||
`NOT ASSESSED — cert tier unknown`.
|
||||
|
||||
The skill produces a checklist report written to `production/launch/launch-checklist-[date].md`
|
||||
after a "May I write" ask. If a previous launch checklist exists, it compares the
|
||||
new results against the old to highlight newly resolved and newly blocked items. No
|
||||
director gates apply — `/team-release` orchestrates the full release pipeline. Verdicts:
|
||||
LAUNCH READY, LAUNCH BLOCKED, or CONCERNS.
|
||||
Phase 2 reads `CLAUDE.md`, the latest milestone, any existing checklist in
|
||||
`production/releases/` and the content calendar. Phase 3 scans the code root for
|
||||
TODO/FIXME/HACK, debug output, placeholder assets and hardcoded dev values, and
|
||||
reports every scan with its denominator (`scanned [N] files, [M] hits`) or
|
||||
`NOT ASSESSED — [reason]`; an unassessable item is written `- [?]` with a legend,
|
||||
never a bare box or tick. The bug rows follow `workflow`: S1 and S2 (or a
|
||||
documented exception), and at `full` S1–S3 with no exceptions. Phase 4b asks one
|
||||
`AskUserQuestion` per section about the items no scan or file settled, each
|
||||
answered yes / no / not yet / N/A: yes ticks it, no or not yet is a failed item
|
||||
(Blocking, or Conditional with a workaround or accepted risk), N/A leaves it out;
|
||||
it is skipped, and says so, in `dry-run` and in `autonomous` mode. The checklist
|
||||
ends in a Go / No-Go section. **Blocking Items** are every FAIL or unresolved
|
||||
finding with no documented workaround or accepted risk; **Conditional Items** are
|
||||
shortfalls with a documented workaround or an explicitly accepted risk. **Overall
|
||||
Status** is computed from them, first match wins: `NOT READY` (any blocking item)
|
||||
→ `NOT ASSESSED` (any `[?]` item, `NOT ASSESSED` section or unconfirmed item, each
|
||||
named) → `CONDITIONAL` (any conditional item) → `READY`. Unless the argument is
|
||||
`dry-run`, it asks "May I write this to
|
||||
`production/releases/launch-checklist-[date].md`?" and creates nothing — not
|
||||
even the directory — before the answer. Next steps: `/gate-check release` first
|
||||
if `project.stage` is not `Release` (`Polish`, an earlier stage, or unset), then
|
||||
`/team-release`. No director gates are spawned.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keywords: LAUNCH READY, LAUNCH BLOCKED, CONCERNS
|
||||
- [ ] Contains "May I write" collaborative protocol language before writing the checklist
|
||||
- [ ] Has a next-step handoff (e.g., `/team-release` or `/day-one-patch`)
|
||||
- [ ] Contains verdict keywords: READY, CONDITIONAL, NOT ASSESSED, NOT READY (Overall Status, first match NOT READY → NOT ASSESSED → CONDITIONAL → READY), and NOT ASSESSED for unassessable scans and cert tier
|
||||
- [ ] Contains "May I write" language before writing the checklist, and creates no file or directory before the answer
|
||||
- [ ] Has a Phase 4b that asks one `AskUserQuestion` per section, answers yes / no / not yet / N/A, with no / not yet a failed item and N/A excluded
|
||||
- [ ] Has a next-step handoff (`/gate-check release` only when `project.stage` is not `Release`, and `/team-release`)
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
None. `/launch-checklist` is a readiness audit utility. The full release pipeline
|
||||
is managed by `/team-release`.
|
||||
None. `/launch-checklist` has no `Agent` tool and spawns no director. The
|
||||
"Sign-Offs Required" list (Creative Director, Technical Director, QA Lead,
|
||||
Producer, Release Manager) is checklist text for humans, coordinated later via
|
||||
`/team-release`; the formal gate verdict comes from `/gate-check`.
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — All Checklist Items Verified, LAUNCH READY
|
||||
### Case 1: Happy Path — Steam project, clean scans, every section confirmed, READY
|
||||
|
||||
**Fixture:**
|
||||
- Legal docs present: EULA, privacy policy in `production/legal/`
|
||||
- Platform certification: marked as submitted and approved in production notes
|
||||
- Store page assets: screenshots, description, metadata all present in `production/store/`
|
||||
- Build: version tag `v1.0.0` exists, reproducible build confirmed
|
||||
- Crash reporting: configured in `technical-preferences.md`
|
||||
- `project.yaml`: `project.stage: Release`, `modes.rigor: standard`, `platform.cert_tier: steam`,
|
||||
engine Godot (code root `src/`); a single-player game with no servers
|
||||
- `src/` has 40 source files with no TODO/FIXME/HACK, no debug output, no placeholder or hardcoded dev values
|
||||
- `assets/` exists with final assets
|
||||
- `production/releases/` does not exist yet
|
||||
- In Phase 4b the user answers `Yes — all of them` for every section, except the
|
||||
Infrastructure question, where `Some differ` gives each Servers item N/A and
|
||||
every Analytics and Monitoring item yes
|
||||
|
||||
**Input:** `/launch-checklist`
|
||||
**Input:** `/launch-checklist 2026-11-01`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill checks all checklist categories
|
||||
2. All items pass their verification checks
|
||||
3. Skill produces checklist report with all items marked PASS
|
||||
4. Skill asks "May I write to `production/launch/launch-checklist-2026-04-06.md`?"
|
||||
5. Report written on approval; verdict is LAUNCH READY
|
||||
1. Target Launch reads 2026-11-01
|
||||
2. Phase 3 reports each scan as `scanned [N] files, 0 hits`
|
||||
3. Platform Certification contains only the Steam block (Steamworks SDK, store page,
|
||||
depot build, achievements, common content rules) plus the every-tier accessibility row
|
||||
4. The soak-test item asks for the longest soak `/soak-test` runs (or a longer
|
||||
one run by hand), not a fixed duration `/soak-test` cannot produce
|
||||
5. Phase 4b asks one `AskUserQuestion` per section, listing only the items no
|
||||
scan or file settled (not the Phase 3 counts), each answered yes / no /
|
||||
not yet / N/A
|
||||
6. The confirmed items are ticked `(confirmed by the user)`; the Servers items are
|
||||
written `- N/A — [item]` and left out of the status. Nothing is Blocking,
|
||||
Conditional or unconfirmed, so the first-match rule gives READY
|
||||
7. Skill presents the checklist with a summary (Overall Status, total items, blocking
|
||||
count, conditional count, not-assessed count, departments with incomplete sections)
|
||||
8. Asks "May I write this to `production/releases/launch-checklist-[date].md`?";
|
||||
`production/releases/` is not created before the answer
|
||||
9. Next steps: `/team-release` — the stage already reads `Release`, so `/gate-check`
|
||||
is not offered
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All checklist categories are checked (legal, platform, store, build, analytics, UX)
|
||||
- [ ] All items appear in the report with PASS markers
|
||||
- [ ] Verdict is LAUNCH READY
|
||||
- [ ] "May I write" is asked with the correct dated filename
|
||||
- [ ] Every Phase 3 scan line shows its denominator (`scanned [N] files, [M] hits`)
|
||||
- [ ] No console (TRC/Lotcheck) or itch.io items appear
|
||||
- [ ] The soak-test item does not require a soak longer than `/soak-test` offers
|
||||
- [ ] Phase 4b asks at most one question per section, and no Phase 3 count is asked about
|
||||
- [ ] Overall Status is READY — every item is ticked on evidence or on a Phase 4b answer, or marked N/A
|
||||
- [ ] The summary gives total, blocking, conditional and not-assessed counts
|
||||
- [ ] "May I write" names `production/releases/launch-checklist-[date].md`, and no directory exists before the user answers it
|
||||
- [ ] Next steps name `/team-release` and do not offer `/gate-check`
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Platform Certification Not Submitted — LAUNCH BLOCKED
|
||||
### Case 2: Blocking Findings — FIXMEs and debug output
|
||||
|
||||
**Fixture:**
|
||||
- All other checklist items pass
|
||||
- Platform certification section: "not submitted" (no submission record found)
|
||||
- As Case 1, but `src/` contains 3 `FIXME` comments and one `print()` debug call in production code
|
||||
- `src/` also has one `HACK` comment whose justification is documented beside it
|
||||
|
||||
**Input:** `/launch-checklist`
|
||||
**Input:** `/launch-checklist 2026-11-01`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill checks all items
|
||||
2. Platform certification check fails: no submission record
|
||||
3. Skill reports: "LAUNCH BLOCKED — Platform certification not submitted"
|
||||
4. Specific platform(s) missing certification are named
|
||||
5. Verdict is LAUNCH BLOCKED
|
||||
1. Phase 3 counts the FIXMEs with their locations and finds the debug output
|
||||
2. Code Quality shows "FIXME count: 3 (zero required)" and the "No debug output" item unticked
|
||||
3. These appear under Blocking Items — findings with no workaround or accepted risk
|
||||
4. The justified HACK appears under Conditional Items, with its justification
|
||||
5. Overall Status is NOT READY — a blocking item outranks everything else
|
||||
6. "May I write" is still asked before saving
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Verdict is LAUNCH BLOCKED (not CONCERNS)
|
||||
- [ ] Platform certification is identified as the blocking item
|
||||
- [ ] Missing platform names are specified
|
||||
- [ ] All other passing items are still shown in the report
|
||||
- [ ] FIXME count is 3, with file locations
|
||||
- [ ] The debug `print()` is reported and its item is not ticked
|
||||
- [ ] Blocking Items lists the FIXMEs and the debug output
|
||||
- [ ] The documented HACK is under Conditional Items, not Blocking Items
|
||||
- [ ] Overall Status is NOT READY (not READY, CONDITIONAL or NOT ASSESSED)
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Manual Check Required — CONCERNS Verdict
|
||||
### Case 3: Cert Tier Unset — Ask, do not emit every track
|
||||
|
||||
**Fixture:**
|
||||
- All critical checklist items pass
|
||||
- First-run experience item: "MANUAL CHECK NEEDED — human must play the first 5
|
||||
minutes and verify tutorial completion flow"
|
||||
- Store screenshots item: "MANUAL CHECK NEEDED — art team must verify screenshot
|
||||
quality matches current build"
|
||||
- `project.yaml`: `project.stage: Release`, `modes.rigor: standard`, no `platform.cert_tier`
|
||||
- Config block shows `platform.cert_tier: (unset -- ask which platforms are in scope)`
|
||||
|
||||
**Input:** `/launch-checklist`
|
||||
**Input:** `/launch-checklist 2026-11-01`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill checks all items
|
||||
2. 2 items are flagged as requiring human verification
|
||||
3. Skill reports: "CONCERNS — 2 items require manual verification before launch"
|
||||
4. Both items are listed with instructions for what to manually verify
|
||||
5. Verdict is CONCERNS (not LAUNCH BLOCKED, since these are advisory)
|
||||
1. Skill asks which platforms are in scope instead of emitting all certification tracks
|
||||
2. If the user answers, only the matching tier block is emitted
|
||||
3. If it cannot be determined, the section is marked `NOT ASSESSED — cert tier unknown`
|
||||
4. Unset is not treated as `none` — the section is not silently omitted
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Verdict is CONCERNS (not LAUNCH READY or LAUNCH BLOCKED)
|
||||
- [ ] Both manual check items are listed with verification instructions
|
||||
- [ ] Skill does not auto-block on MANUAL CHECK items
|
||||
- [ ] Skill asks which platforms are in scope
|
||||
- [ ] Console, Steam and itch.io blocks are never all emitted together
|
||||
- [ ] Without an answer, the section reads `NOT ASSESSED — cert tier unknown`
|
||||
- [ ] The "omitted — cert_tier is 'none'" line does not appear
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Previous Checklist Exists — Delta Comparison
|
||||
### Case 4: Nothing to Scan — `[?]` items, not green ticks
|
||||
|
||||
**Fixture:**
|
||||
- `production/launch/launch-checklist-2026-03-25.md` exists with previous results:
|
||||
- 2 items were BLOCKED (platform cert, crash reporting)
|
||||
- 1 item had a MANUAL CHECK
|
||||
- New checklist: platform cert is now PASS, crash reporting is now PASS,
|
||||
manual check still open; 1 new item flagged (EULA last updated date)
|
||||
- `project.yaml`: `project.stage: Release`, `platform.cert_tier: itch`, engine Godot
|
||||
- `src/` exists but is empty; no `assets/` directory
|
||||
|
||||
**Input:** `/launch-checklist`
|
||||
**Input:** `/launch-checklist 2026-11-01`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill finds the previous checklist and loads it for comparison
|
||||
2. Skill produces the new checklist and compares:
|
||||
- Newly resolved: "Platform cert — was BLOCKED, now PASS"
|
||||
- Newly resolved: "Crash reporting — was BLOCKED, now PASS"
|
||||
- Still open: manual check (unchanged)
|
||||
- New issue: EULA last updated date (not in previous checklist)
|
||||
3. Delta is shown prominently in the report
|
||||
4. Verdict is CONCERNS (manual check + new EULA question)
|
||||
1. Phase 3 reports the code scans as `NOT ASSESSED — no source files` (or
|
||||
`directory empty`) and the placeholder-asset scan as
|
||||
`NOT ASSESSED — no asset folder`, instead of zero hits
|
||||
2. The affected items (e.g. "All placeholder art replaced with final assets",
|
||||
"No debug output in production code") are written `- [?]`
|
||||
3. The legend `[?] = not assessed — the input to this check was absent` is included
|
||||
4. Platform Certification contains only the itch.io block
|
||||
5. Nothing was found, so Blocking Items is empty — and the `[?]` items make the
|
||||
Overall Status **NOT ASSESSED**, naming the scans that could not run, never
|
||||
READY
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Delta section shows newly resolved items
|
||||
- [ ] Delta section shows new issues (not present in previous checklist)
|
||||
- [ ] Still-open items from the previous checklist are noted as persistent
|
||||
- [ ] Verdict reflects the current state (not the previous state)
|
||||
- [ ] No scan with no input is reported as zero hits
|
||||
- [ ] "All placeholder art replaced with final assets" is `- [?]`, not ticked and not `- [ ]`
|
||||
- [ ] The `[?]` legend appears
|
||||
- [ ] No Steamworks or console items appear
|
||||
- [ ] Overall Status is NOT ASSESSED and names the `[?]` scans (code and placeholder-asset scans); it is not READY or CONDITIONAL
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; launch-checklist is an audit utility
|
||||
### Case 5: Director Gate Check — Dry run, cert tier `none`
|
||||
|
||||
**Fixture:**
|
||||
- All checklist dependencies present
|
||||
- `project.yaml`: `project.stage: Release`, `platform.cert_tier: none`
|
||||
- Code root with source files
|
||||
|
||||
**Input:** `/launch-checklist`
|
||||
**Input:** `/launch-checklist dry-run`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill runs the full checklist and writes the report
|
||||
2. No director agents are spawned
|
||||
3. No gate IDs appear in output
|
||||
1. Target Launch reads DRY RUN
|
||||
2. Platform Certification is replaced by the one line
|
||||
`Platform Certification: omitted — cert_tier is 'none' (internal/jam release).`
|
||||
3. Phase 4b is skipped and the output says so; the items it would have asked stay
|
||||
unconfirmed, so the Overall Status is NOT ASSESSED
|
||||
4. The checklist is presented but no "May I write" ask is made and no file is written
|
||||
5. No director or other agent is spawned; no gate IDs appear
|
||||
6. Next steps name `/team-release`; the stage reads `Release`, so `/gate-check` is not offered
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Verdict is LAUNCH READY, LAUNCH BLOCKED, or CONCERNS — no gate verdict
|
||||
- [ ] The "omitted — cert_tier is 'none'" line appears and no certification rows are emitted
|
||||
- [ ] No Phase 4b question is asked in dry-run, the skip is stated, and the Overall Status is NOT ASSESSED
|
||||
- [ ] No "May I write" ask and no file written in dry-run mode
|
||||
- [ ] No director gate is invoked and no gate skip messages appear
|
||||
- [ ] Next steps name `/team-release` and do not offer `/gate-check`
|
||||
|
||||
---
|
||||
|
||||
### Case 6: Stage Still Polish — Answers Make It CONDITIONAL, or NOT ASSESSED
|
||||
|
||||
**Fixture:**
|
||||
- As Case 1, but `project.stage: Polish` and `modes.rigor: full` (`workflow: full`)
|
||||
- Variant A: in Phase 4b the Marketing items all get yes except "Press/influencer
|
||||
review keys distributed", answered `not yet` — the user says the keys go out on
|
||||
launch morning and the producer accepts the risk; every other section is `Yes —
|
||||
all of them` (Servers N/A as in Case 1)
|
||||
- Variant B: as Variant A, but the user leaves the Legal question unanswered
|
||||
|
||||
**Input:** `/launch-checklist 2026-11-01`
|
||||
|
||||
**Expected behavior:**
|
||||
1. The Testing bug rows are the `workflow: full` ones: zero S1, and zero S2 and S3
|
||||
with no exceptions
|
||||
2. Variant A: the press-keys item is a failed item with an accepted risk, so it goes
|
||||
under Conditional Items naming who accepted it; nothing is Blocking or
|
||||
unconfirmed; Overall Status is CONDITIONAL
|
||||
3. Variant B: the Legal items stay unconfirmed and are named; Overall Status is
|
||||
NOT ASSESSED (it outranks CONDITIONAL)
|
||||
4. Next steps: run `/gate-check release` first — the stage still reads `Polish` —
|
||||
then `/team-release`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] A `not yet` answer is never ticked; with an accepted risk it is listed under Conditional Items
|
||||
- [ ] Variant A's Overall Status is CONDITIONAL
|
||||
- [ ] Variant B's Overall Status is NOT ASSESSED and names the unanswered Legal items
|
||||
- [ ] No "documented exception" is offered for an S2 or S3 at `workflow: full`
|
||||
- [ ] Next steps name `/gate-check release` before `/team-release`
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Checks all required categories (legal, platform, store, build, analytics, UX)
|
||||
- [ ] LAUNCH BLOCKED for hard failures (uncompleted certifications, missing legal docs)
|
||||
- [ ] CONCERNS for advisory items requiring manual verification
|
||||
- [ ] Compares against previous checklist when one exists
|
||||
- [ ] Asks "May I write" before creating the checklist report
|
||||
- [ ] Verdict is LAUNCH READY, LAUNCH BLOCKED, or CONCERNS
|
||||
- [ ] Scopes the checklist by `project.stage`, `modes.rigor` and `platform.cert_tier`
|
||||
- [ ] Branches certification on the four `cert_tier` values, not on platform names
|
||||
- [ ] Reports every scan with a denominator or `NOT ASSESSED — [reason]`
|
||||
- [ ] Writes unassessable items as `- [?]` with the legend
|
||||
- [ ] Asks "May I write" before saving (skipped entirely in dry-run), and creates no file or directory before the answer
|
||||
- [ ] Asks one Phase 4b question per section for the items only a person can confirm, never answering them itself; skips Phase 4b, and says so, in `dry-run` and `autonomous` mode
|
||||
- [ ] Names `/gate-check release` in next steps whenever `project.stage` is not `Release`
|
||||
- [ ] Puts findings with no workaround or accepted risk under Blocking Items, and shortfalls with a documented workaround or accepted risk under Conditional Items
|
||||
- [ ] Overall Status is READY, CONDITIONAL, NOT ASSESSED or NOT READY, first match NOT READY → NOT ASSESSED → CONDITIONAL → READY
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- Region-specific compliance (GDPR data handling, COPPA for under-13 audiences)
|
||||
is checked but the specific requirements are not enumerated in test assertions.
|
||||
- The store page completeness check (screenshots, description) relies on the
|
||||
presence of files in `production/store/`; it cannot verify visual quality.
|
||||
- Build reproducibility check validates the presence of a version tag and build
|
||||
configuration but does not execute the build process.
|
||||
- `cert_tier: console` (TRC/Lotcheck, save-data, controller-mapping, age ratings)
|
||||
is not separately tested.
|
||||
- Stage scoping (items for phases not yet reached are declared out of scope) and
|
||||
`rigor: minimal` trimming are not separately tested.
|
||||
- An existing checklist in `production/releases/` is read as context only; no
|
||||
comparison against it is tested.
|
||||
|
||||
@@ -2,175 +2,247 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/localize` manages the full localization pipeline: it extracts all player-facing
|
||||
strings from source files, manages translation files in `assets/localization/`,
|
||||
and validates completeness across all locale files. For new languages, it creates
|
||||
a locale file skeleton with all current strings as keys and empty values. For
|
||||
existing locale files, it produces a diff showing additions, removals, and
|
||||
changed keys.
|
||||
`/localize` runs the localization pipeline through one required subcommand:
|
||||
`scan`, `extract`, `validate`, `status`, `brief`, `cultural-review`,
|
||||
`vo-pipeline`, `rtl-check`, `freeze` or `qa`. It works on a single source string
|
||||
table under `assets/data/strings/` (`strings-en.json` is the English source; the
|
||||
other locales sit beside it) and writes reports under `production/localization/`.
|
||||
|
||||
Translation files are written to `assets/localization/[locale-code].csv` (or
|
||||
engine-appropriate format) after a "May I write" ask. No director gates apply.
|
||||
Verdicts: LOCALIZATION COMPLETE (all locales are complete) or GAPS FOUND (at
|
||||
least one locale is missing string keys).
|
||||
`scan`, `validate` and `status` are read-only. `extract` shows a diff of new
|
||||
keyed entries and asks "May I write these new entries to
|
||||
`assets/data/strings/strings-en.json`?" before appending only the diff; its
|
||||
verdict is **COMPLETE**. With no subcommand the skill prints usage and stops with
|
||||
verdict **FAIL** — missing required subcommand. A scan whose code root cannot be
|
||||
resolved reports `NOT ASSESSED — code root unresolved`; a status run with no
|
||||
string table reports `NOT ASSESSED — no string table found`. `cultural-review`
|
||||
and `qa` spawn `localization-lead`; `qa` issues a per-locale
|
||||
PASS / PASS WITH CONDITIONS / NOT ASSESSED / FAIL verdict, first match FAIL →
|
||||
NOT ASSESSED → PASS WITH CONDITIONS → PASS. Its checks are playthrough-based, so a
|
||||
locale nobody has played is NOT ASSESSED, and the release gate does not count it
|
||||
as passed. No director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keywords: LOCALIZATION COMPLETE, GAPS FOUND
|
||||
- [ ] Contains "May I write" collaborative protocol language before writing locale files
|
||||
- [ ] Has a next-step handoff (e.g., send locale skeletons to translators)
|
||||
- [ ] Contains verdict keywords: COMPLETE, FAIL, NOT ASSESSED, PASS WITH CONDITIONS
|
||||
- [ ] Contains "May I write" collaborative protocol language before writing the string table or a report
|
||||
- [ ] Has a next-step handoff (the Recommended Workflow chain ending in `/gate-check release`)
|
||||
- [ ] The QA gate note is tiered: at `workflow: full` the Polish → Release gate requires PASS or PASS WITH CONDITIONS for every translated locale being shipped (recommended at `standard`, dropped at `minimal`)
|
||||
- [ ] The `freeze` question names the file it writes — "This writes `production/localization/freeze-status.md`" — before the user answers
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
None. `/localize` is a pipeline utility. No director gates apply. Localization
|
||||
lead agent may review separately but is not invoked within this skill.
|
||||
None. `/localize` is a pipeline utility and spawns no director. The only agent
|
||||
it spawns is `localization-lead` (an operations-tier agent), and only in the
|
||||
`cultural-review` and `qa` modes.
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: New Language — String Extraction and Locale Skeleton Created
|
||||
### Case 1: Extract — New Strings Keyed and Appended After Approval
|
||||
|
||||
**Fixture:**
|
||||
- Source code in `src/` contains player-facing strings (UI text, tutorial messages)
|
||||
- Existing locale: `assets/localization/en.csv`
|
||||
- No French locale exists
|
||||
- `project.yaml` has `engine.name: Godot`, so the code root is `src/`
|
||||
- `src/ui/hud.gd` wraps three strings in `tr()` that have no entry in
|
||||
`assets/data/strings/strings-en.json`
|
||||
- No `production/localization/freeze-status.md` exists
|
||||
|
||||
**Input:** `/localize fr`
|
||||
**Input:** `/localize extract`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill extracts all player-facing strings from source files
|
||||
2. Skill finds the same strings in `en.csv` as a reference
|
||||
3. Skill generates `fr.csv` skeleton with all string keys and empty values
|
||||
4. Skill asks "May I write to `assets/localization/fr.csv`?"
|
||||
5. File written on approval; verdict is GAPS FOUND (file created but empty values)
|
||||
6. Skill notes: "fr.csv created — send to translator to fill values"
|
||||
1. Skill scans source files for localized string references and compares them
|
||||
against the table in `assets/data/strings/`
|
||||
2. Skill proposes one new entry per unkeyed string, with a key in the form
|
||||
`[category].[subcategory].[description]` (e.g. `ui.hud.health_label`)
|
||||
3. Each new entry carries a `context` field: where it appears, maximum length,
|
||||
placeholder meaning
|
||||
4. Skill presents the diff and asks "May I write these new entries to
|
||||
`assets/data/strings/strings-en.json`?"
|
||||
5. On approval it writes only the new entries (not a full replacement);
|
||||
verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All string keys from `en.csv` are present in `fr.csv`
|
||||
- [ ] All values in `fr.csv` are empty (not copied from English)
|
||||
- [ ] "May I write" is asked before creating the file
|
||||
- [ ] Verdict is GAPS FOUND (file is created but untranslated)
|
||||
- [ ] Proposed keys follow `[category].[subcategory].[description]`
|
||||
- [ ] Every new entry includes a `context` field
|
||||
- [ ] The diff is shown and "May I write" names `assets/data/strings/strings-en.json` before any write
|
||||
- [ ] Only the three new entries are written — existing entries are not rewritten
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Existing Locale Diff — Additions, Removals, and Changes Listed
|
||||
### Case 2: No Subcommand — Usage and FAIL
|
||||
|
||||
**Fixture:**
|
||||
- `assets/localization/fr.csv` exists with 20 string keys translated
|
||||
- Source code has changed: 3 new strings added, 1 string removed, 2 strings
|
||||
with changed English source text
|
||||
|
||||
**Input:** `/localize fr`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill extracts current strings from source
|
||||
2. Skill diffs against existing `fr.csv`
|
||||
3. Skill produces diff report:
|
||||
- 3 new keys (need translation — listed as empty in fr.csv)
|
||||
- 1 removed key (marked as obsolete — suggest removal)
|
||||
- 2 changed keys (English source changed — French may need update, flagged)
|
||||
4. Skill asks "May I update `assets/localization/fr.csv`?"
|
||||
5. File updated with new empty keys added, obsolete keys marked; verdict is GAPS FOUND
|
||||
|
||||
**Assertions:**
|
||||
- [ ] New keys appear as empty in the updated file (not auto-translated)
|
||||
- [ ] Removed keys are flagged as obsolete (not silently deleted)
|
||||
- [ ] Changed source strings are flagged for translator review
|
||||
- [ ] Verdict is GAPS FOUND (new empty keys exist)
|
||||
|
||||
---
|
||||
|
||||
### Case 3: String Missing in One Locale — GAPS FOUND With Missing Key List
|
||||
|
||||
**Fixture:**
|
||||
- 3 locale files exist: `en.csv`, `fr.csv`, `de.csv`
|
||||
- `de.csv` is missing 4 keys that exist in both `en.csv` and `fr.csv`
|
||||
- Any project state
|
||||
|
||||
**Input:** `/localize`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads all 3 locale files and cross-references keys
|
||||
2. `de.csv` is missing 4 keys
|
||||
3. Skill produces GAPS FOUND report listing the 4 missing keys by locale:
|
||||
"de.csv missing: [key1], [key2], [key3], [key4]"
|
||||
4. Skill offers to add the missing keys as empty values to `de.csv`
|
||||
5. After approval: file updated; verdict remains GAPS FOUND (values still empty)
|
||||
1. Skill finds no subcommand in the argument
|
||||
2. Skill outputs usage (the ten modes) and stops
|
||||
3. Verdict is FAIL — missing required subcommand
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Missing keys are listed explicitly (not just a count)
|
||||
- [ ] Missing keys are attributed to the specific locale file
|
||||
- [ ] Verdict is GAPS FOUND (not LOCALIZATION COMPLETE)
|
||||
- [ ] Missing keys are added as empty (not auto-translated from English)
|
||||
- [ ] Usage listing the available modes is printed
|
||||
- [ ] Verdict is FAIL with the reason "missing required subcommand"
|
||||
- [ ] No scan, extract or validation work is performed
|
||||
- [ ] No file is written and no "May I write" ask appears
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Translation File Has Syntax Error — Error With Line Reference
|
||||
### Case 3: Validate — Gaps Reported by Locale, Nothing Written
|
||||
|
||||
**Fixture:**
|
||||
- `assets/localization/fr.csv` has a malformed line at line 47
|
||||
(missing quote closure)
|
||||
- `assets/data/strings/` holds `strings-en.json`, `strings-fr.json` and
|
||||
`strings-de.json`
|
||||
- `strings-de.json` lacks 4 keys present in the English source
|
||||
- One French entry omits the `{playerName}` placeholder its English source has
|
||||
|
||||
**Input:** `/localize fr`
|
||||
**Input:** `/localize validate`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads `fr.csv` and encounters a parse error at line 47
|
||||
2. Skill outputs: "Parse error in fr.csv at line 47: [error detail]"
|
||||
3. Skill cannot diff or validate the file until the error is fixed
|
||||
4. Skill does NOT attempt to overwrite or auto-fix the malformed file
|
||||
5. Skill suggests fixing the file manually and re-running `/localize`
|
||||
1. Skill reads every string table file in `assets/data/strings/`
|
||||
2. For `de` it reports the 4 keys as Completeness gaps
|
||||
3. For `fr` it reports a Placeholder mismatch on the affected key
|
||||
4. Results are grouped by locale and severity
|
||||
5. Skill writes nothing — validate is read-only
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Error message includes line number (line 47)
|
||||
- [ ] Error detail describes the nature of the parse error
|
||||
- [ ] Skill does NOT overwrite or modify the malformed file
|
||||
- [ ] Manual fix + re-run is suggested as remediation
|
||||
- [ ] The 4 missing keys are named and attributed to the `de` locale
|
||||
- [ ] The placeholder mismatch is reported against the `fr` locale and its key
|
||||
- [ ] Output is grouped by locale and severity
|
||||
- [ ] No file is written or modified, and no "May I write" ask appears
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; localization is a pipeline utility
|
||||
### Case 4: Status With No String Table — NOT ASSESSED
|
||||
|
||||
**Fixture:**
|
||||
- Source code with player-facing strings
|
||||
- `assets/data/strings/` does not exist
|
||||
|
||||
**Input:** `/localize fr`
|
||||
**Input:** `/localize status`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill extracts strings and manages locale files
|
||||
2. No director agents are spawned
|
||||
3. No gate IDs appear in output
|
||||
1. Skill looks for the source string table and finds none
|
||||
2. The whole status output is `NOT ASSESSED — no string table found`
|
||||
3. Skill does not produce a coverage matrix of zeros, and does not pre-fill the
|
||||
source row as `100%`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Verdict is LOCALIZATION COMPLETE or GAPS FOUND — no gate verdict
|
||||
- [ ] Output is `NOT ASSESSED — no string table found`
|
||||
- [ ] No coverage matrix with zero counts is produced
|
||||
- [ ] The source locale is not reported as `100%` coverage
|
||||
- [ ] No file is written (status is read-only)
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — None; Cultural Review Uses localization-lead
|
||||
|
||||
**Fixture:**
|
||||
- `assets/data/strings/strings-en.json` exists with player-facing strings
|
||||
- Any review mode
|
||||
|
||||
**Input:** `/localize cultural-review`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill spawns `localization-lead` to audit strings and assets for cultural
|
||||
sensitivity
|
||||
2. Findings are presented as a table with a severity of BLOCKING, ADVISORY or
|
||||
NOTE per finding
|
||||
3. Skill asks "May I write this cultural review report to
|
||||
`production/localization/cultural-review-[date].md`?"
|
||||
4. No director agent is spawned and no gate ID appears in output
|
||||
|
||||
**Assertions:**
|
||||
- [ ] `localization-lead` is the only agent spawned
|
||||
- [ ] No director gate (creative, technical, producer, art) is invoked and no gate skip message appears
|
||||
- [ ] Each finding carries BLOCKING, ADVISORY or NOTE severity
|
||||
- [ ] "May I write" names `production/localization/cultural-review-[date].md` before the report is written
|
||||
|
||||
---
|
||||
|
||||
### Case 6: Scan With an Unresolved Code Root — NOT ASSESSED, never "no hardcoded strings"
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` has no `engine.name`; `.claude/docs/technical-preferences.md`
|
||||
reads `[TO BE CONFIGURED]`
|
||||
- Both `src/` and `Assets/` exist and hold UI code with hardcoded strings
|
||||
(ambiguous tree)
|
||||
|
||||
**Input:** `/localize scan`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill tries to resolve the code root and cannot (no engine, two candidate roots)
|
||||
2. It reports `NOT ASSESSED — code root unresolved` for the hardcoded-string
|
||||
search and says which checks did not run
|
||||
3. It does not guess `src/`, and does not report zero hits or "no hardcoded
|
||||
strings found" — that would read as "nothing to localize"
|
||||
4. Nothing is written — scan is read-only
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Output reads `NOT ASSESSED — code root unresolved` for the code search
|
||||
- [ ] No "no hardcoded strings" or zero-hit result is reported
|
||||
- [ ] The code root is not assumed to be `src/`
|
||||
- [ ] No file is written and no "May I write" ask appears
|
||||
|
||||
---
|
||||
|
||||
### Case 7: QA on a Locale Nobody Has Played — NOT ASSESSED
|
||||
|
||||
**Fixture:**
|
||||
- `assets/data/strings/strings-en.json` and `strings-fr.json` exist and validate
|
||||
cleanly
|
||||
- No one has played the French build: there are no in-game observations for the
|
||||
functional, overflow, contextual-accuracy or VO checks
|
||||
|
||||
**Input:** `/localize qa`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill spawns `localization-lead` for the QA plan
|
||||
2. The playthrough-based checks have no results, and no BLOCKING finding is open,
|
||||
so the French verdict is **NOT ASSESSED**, naming each check that did not run
|
||||
3. The verdict is not PASS or PASS WITH CONDITIONS on the strength of the string
|
||||
tables alone
|
||||
4. Skill asks "May I write this localization QA report to
|
||||
`production/localization/loc-qa-fr-[date].md`?"; the report says the locale
|
||||
has not passed QA for the release gate
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The French Status is NOT ASSESSED, with the unrun checks named
|
||||
- [ ] No PASS or PASS WITH CONDITIONS is issued for an unplayed locale
|
||||
- [ ] "May I write" names the loc-qa report path before it is written
|
||||
- [ ] The report does not present the locale as cleared for the Polish → Release gate
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Extracts strings from source before operating on locale files
|
||||
- [ ] Creates new locale files with all keys as empty values (not auto-translated)
|
||||
- [ ] Diffs existing locale files against current source strings
|
||||
- [ ] Flags missing keys by locale and by key name
|
||||
- [ ] Asks "May I write" before creating or updating any locale file
|
||||
- [ ] Verdict is LOCALIZATION COMPLETE (all locales fully translated) or GAPS FOUND
|
||||
- [ ] Stops with usage and verdict FAIL when no subcommand is given
|
||||
- [ ] `scan`, `validate` and `status` write no files
|
||||
- [ ] Asks "May I write" with the target path before writing the string table (extract) or any report (brief, cultural-review, vo-pipeline script, rtl-check, qa)
|
||||
- [ ] Extract writes only the diff of new entries, never a full replacement of the table
|
||||
- [ ] Reports NOT ASSESSED instead of zero results when the code root or the string table is missing
|
||||
- [ ] `qa` issues PASS, PASS WITH CONDITIONS, NOT ASSESSED or FAIL per locale, first match FAIL → NOT ASSESSED → PASS WITH CONDITIONS → PASS
|
||||
- [ ] Ends with the Recommended Workflow chain / `/gate-check release` handoff
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- LOCALIZATION COMPLETE is only achievable when all locale files have all keys
|
||||
with non-empty values; new-language skeleton creation always results in GAPS FOUND.
|
||||
- Engine-specific locale formats (Godot `.translation`, Unity `.po` files) are
|
||||
handled by the skill body; `.csv` is used as the canonical format in tests.
|
||||
- The case where source strings change at a very high rate (continuous integration
|
||||
of new UI text) is not tested; the diff logic handles this case.
|
||||
- `scan` with a resolved code root (findings with file paths and line numbers)
|
||||
is not separately tested; Case 6 covers the unresolved root.
|
||||
- `extract` while `freeze-status.md` shows Status: ACTIVE (new keys appended to
|
||||
`## Post-Freeze Changes` with a freeze-violation warning) is not tested.
|
||||
- `freeze` (pre-freeze checklist and `AskUserQuestion` [A]/[B]), `brief`,
|
||||
`vo-pipeline` and `rtl-check` are not exercised by a dedicated case; `qa` is
|
||||
covered only for its NOT ASSESSED branch (Case 7).
|
||||
- The `qa` gate note — a FAIL blocks release for that locale only — needs a live
|
||||
run with multiple locales to verify.
|
||||
|
||||
@@ -2,178 +2,200 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/onboard` generates a contextual project onboarding summary tailored for a new
|
||||
team member. It reads CLAUDE.md, `technical-preferences.md`, the active sprint
|
||||
file, recent git commits, and `production/stage.txt` to produce a structured
|
||||
orientation document. The skill runs on the Haiku model (read-only, formatting
|
||||
task) and produces no file writes — all output is conversational.
|
||||
`/onboard [role|area]` generates an onboarding document for a new contributor or
|
||||
agent. It reads `CLAUDE.md`, the matching agent definition in `.claude/agents/`
|
||||
when a role is given, scans the area that role works in (the code root for
|
||||
programmers, `design/` for designers, `design/narrative/` for narrative, `tests/`
|
||||
for QA, `production/` for production) and reads recent git history.
|
||||
|
||||
The skill optionally accepts a role argument (e.g., `/onboard artist`) to tailor
|
||||
the summary to a specific discipline. When the project is in an early stage or
|
||||
unconfigured, the output adapts to reflect what little is known. The verdict is
|
||||
always ONBOARDING COMPLETE — the skill is purely informational.
|
||||
Before producing anything it lists its inputs as FOUND or ABSENT. A section whose
|
||||
input is ABSENT is written as `NOT ASSESSED — NO DATA`, and one fed by an
|
||||
unresolved code root as `NOT ASSESSED — code root unresolved`. The current-state
|
||||
section names what Phase 2 found by path (for a designer, each existing GDD and
|
||||
design doc). If every input is ABSENT
|
||||
the whole verdict is `NOT ASSESSED — NO DATA`, naming what was missing. Otherwise
|
||||
it fills the `# Onboarding: [Role/Area]` template, presents it, and asks "May I
|
||||
write this to `production/onboarding/onboard-[role]-[date].md`?". The verdict is
|
||||
**COMPLETE**, followed by next steps pointing to `/sprint-status` and `/help`. No
|
||||
director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keyword: ONBOARDING COMPLETE
|
||||
- [ ] Does NOT contain "May I write" language (skill is read-only)
|
||||
- [ ] Has a next-step handoff suggesting a relevant follow-on skill
|
||||
- [ ] Contains verdict keywords: COMPLETE, NOT ASSESSED
|
||||
- [ ] Contains "May I write" language before saving the onboarding document
|
||||
- [ ] Has a next-step handoff suggesting `/sprint-status` and `/help`
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
None. `/onboard` is a read-only orientation skill. No director gates apply.
|
||||
None. `/onboard` is an orientation utility. It spawns no agents (`Agent` is not
|
||||
in its allowed tools) and no director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — Configured project in Production stage with active sprint
|
||||
### Case 1: Happy Path — Programmer Role on a Godot Project
|
||||
|
||||
**Fixture:**
|
||||
- `production/stage.txt` contains `Production`
|
||||
- `technical-preferences.md` has engine, language, and specialists populated
|
||||
- `production/sprints/sprint-005.md` exists with stories in progress
|
||||
- Git log contains 5 recent commits
|
||||
- `CLAUDE.md` exists
|
||||
- `project.yaml` has `engine.name: Godot`, so the code root is `src/`
|
||||
- `src/` holds game code; `.claude/agents/gameplay-programmer.md` exists
|
||||
- `production/sprints/sprint-005.md` exists; git log has recent commits
|
||||
- `modes.automation` is unset (defaults to `collaborative`)
|
||||
|
||||
**Input:** `/onboard`
|
||||
**Input:** `/onboard gameplay-programmer`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads stage.txt, technical-preferences.md, active sprint, and git log
|
||||
2. Skill produces an onboarding summary with sections: Project Overview, Tech Stack,
|
||||
Current Stage, Active Sprint Summary, Recent Activity
|
||||
3. Summary is formatted for readability (headers, bullet points)
|
||||
4. Next-step suggestions are appropriate for Production stage (e.g., `/sprint-status`,
|
||||
`/dev-story`)
|
||||
5. Verdict ONBOARDING COMPLETE is stated
|
||||
1. Skill reads `CLAUDE.md` and `.claude/agents/gameplay-programmer.md`
|
||||
2. Skill scans the code root `src/` for architecture, patterns and key files, and
|
||||
reads recent git history
|
||||
3. Skill generates a document headed `# Onboarding: gameplay-programmer` with the
|
||||
template sections (Project Summary, Your Role, Project Architecture, Current
|
||||
Standards and Conventions, Current State of Your Area, Current Sprint Context,
|
||||
Key Dependencies, Common Pitfalls, First Tasks, Questions to Ask)
|
||||
4. Skill presents the document, then asks "May I write this to
|
||||
`production/onboarding/onboard-gameplay-programmer-[date].md`?"
|
||||
5. On approval it writes the file (creating the directory if needed); verdict is
|
||||
COMPLETE, with `/sprint-status` and `/help` as next steps
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Output includes current stage name from stage.txt
|
||||
- [ ] Output includes engine and language from technical-preferences.md
|
||||
- [ ] Active sprint stories are summarized (not just the sprint file name)
|
||||
- [ ] Recent commit context is present
|
||||
- [ ] Verdict is ONBOARDING COMPLETE
|
||||
- [ ] No files are written
|
||||
- [ ] The agent definition for the named role is read
|
||||
- [ ] The code root `src/` is scanned for this programmer role
|
||||
- [ ] The document uses the `# Onboarding: [Role/Area]` template sections
|
||||
- [ ] The document is presented before the "May I write" ask naming `production/onboarding/onboard-gameplay-programmer-[date].md`
|
||||
- [ ] Verdict is COMPLETE and next steps name `/sprint-status` and `/help`
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Fresh Project — No engine, no sprint, suggests /start
|
||||
### Case 2: Designer Role — Scans design/, Not the Code Root
|
||||
|
||||
**Fixture:**
|
||||
- `technical-preferences.md` contains only placeholders (`[TO BE CONFIGURED]`)
|
||||
- No `production/stage.txt`
|
||||
- No sprint files
|
||||
- No CLAUDE.md overrides beyond defaults
|
||||
- `CLAUDE.md` exists; `.claude/agents/game-designer.md` exists
|
||||
- `design/gdd/` holds three system GDDs
|
||||
- `src/` also holds code
|
||||
|
||||
**Input:** `/onboard`
|
||||
**Input:** `/onboard game-designer`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads all config files and detects unconfigured state
|
||||
2. Skill produces a minimal summary: "This project has not been configured yet"
|
||||
3. Output explains the onboarding workflow: `/start` → `/setup-engine` → `/brainstorm`
|
||||
4. Skill suggests running `/start` as the immediate next step
|
||||
5. Verdict is ONBOARDING COMPLETE (informational, not a failure)
|
||||
1. Skill reads `CLAUDE.md` and `.claude/agents/game-designer.md`
|
||||
2. Phase 2 takes the designers branch and scans `design/` for existing design
|
||||
documents
|
||||
3. "Current State of Your Area" reflects the three GDDs found
|
||||
4. Document is presented, "May I write" asked for
|
||||
`production/onboarding/onboard-game-designer-[date].md`; verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Output explicitly mentions the project is not yet configured
|
||||
- [ ] `/start` is recommended as the next step
|
||||
- [ ] Skill does NOT error out — it gracefully handles an empty project state
|
||||
- [ ] Verdict is still ONBOARDING COMPLETE
|
||||
- [ ] `design/` is the area scanned for this role
|
||||
- [ ] The existing GDDs are named in the document's current-state section
|
||||
- [ ] The file path in the "May I write" ask uses the role name
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 3: No CLAUDE.md Found — Error with remediation
|
||||
### Case 3: Every Input Absent — NOT ASSESSED, No Document
|
||||
|
||||
**Fixture:**
|
||||
- `CLAUDE.md` file does not exist (deleted or never created)
|
||||
- All other files may or may not exist
|
||||
- `CLAUDE.md` does not exist
|
||||
- `.claude/agents/` does not exist, so no agent definition can be read
|
||||
- `tests/` does not exist; the directory is not a git repository
|
||||
|
||||
**Input:** `/onboard`
|
||||
**Input:** `/onboard qa-tester`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill attempts to read CLAUDE.md and fails
|
||||
2. Skill outputs an error: "CLAUDE.md not found — cannot generate onboarding summary"
|
||||
3. Skill provides remediation: "Run `/start` to initialize the project configuration"
|
||||
4. No partial summary is generated
|
||||
1. Skill lists its inputs and records each as ABSENT
|
||||
2. Because every required input is ABSENT, it stops and reports
|
||||
`NOT ASSESSED — NO DATA` as the whole verdict
|
||||
3. The report names what was missing and what produces it: `/test-setup` for
|
||||
`tests/`; `CLAUDE.md` and `.claude/agents/` come with the framework itself —
|
||||
no skill creates them — so they are reported as missing from the install,
|
||||
not attributed to a skill
|
||||
4. No onboarding document is generated and no file is written
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Error message clearly identifies the missing file as CLAUDE.md
|
||||
- [ ] Remediation step (`/start`) is explicitly named
|
||||
- [ ] Skill does NOT produce a partial output when the root config is missing
|
||||
- [ ] Verdict is ONBOARDING COMPLETE (with error context, not a crash)
|
||||
- [ ] Inputs are recorded as FOUND or ABSENT before any report is produced
|
||||
- [ ] Verdict is `NOT ASSESSED — NO DATA`, not COMPLETE
|
||||
- [ ] The missing inputs are named; `tests/` points to `/test-setup`
|
||||
- [ ] No skill is named as the producer of `CLAUDE.md` or `.claude/agents/` — none creates them
|
||||
- [ ] No template is filled in and no "May I write" ask appears
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Role-Specific Onboarding — User specifies "artist" role
|
||||
### Case 4: Edge Case — Programmer Role With an Unresolved Code Root
|
||||
|
||||
**Fixture:**
|
||||
- Fully configured project in Production stage
|
||||
- `art-bible.md` exists in `design/`
|
||||
- Active sprint has visual story types (animation, VFX)
|
||||
- `CLAUDE.md` exists; `.claude/agents/gameplay-programmer.md` exists
|
||||
- `project.yaml` has no `engine.name`, technical-preferences has
|
||||
`[TO BE CONFIGURED]`, and both `src/` and `Assets/` exist (ambiguous tree)
|
||||
|
||||
**Input:** `/onboard artist`
|
||||
**Input:** `/onboard gameplay-programmer`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads all standard files plus any art-relevant docs (art bible, asset specs)
|
||||
2. Summary is tailored to the artist role: art bible overview, asset pipeline,
|
||||
current visual stories in the active sprint
|
||||
3. Technical architecture details (code structure, ADRs) are de-emphasized
|
||||
4. Specialist agents for art/audio are highlighted in the summary
|
||||
5. Verdict is ONBOARDING COMPLETE
|
||||
1. Skill tries to resolve the code root and finds it unresolved (two candidate
|
||||
roots, no engine set)
|
||||
2. It does not default to `src/`
|
||||
3. The code-dependent sections (Project Architecture, Key Files) are written as
|
||||
`NOT ASSESSED — code root unresolved` rather than filled from a guess — the
|
||||
code exists, so this is not a missing-data case
|
||||
4. Sections backed by found inputs (Project Summary, Your Role, Current
|
||||
Standards and Conventions) are still produced; the document is presented
|
||||
before the "May I write" ask
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Role argument is acknowledged in the output ("Onboarding for: Artist")
|
||||
- [ ] Art bible summary is included if the file exists
|
||||
- [ ] Current visual stories from the active sprint are shown
|
||||
- [ ] Technical implementation details are not the primary focus
|
||||
- [ ] Verdict is ONBOARDING COMPLETE
|
||||
- [ ] The code root is not assumed to be `src/`
|
||||
- [ ] Architecture/key-file content is marked `NOT ASSESSED — code root unresolved`, not `NO DATA`
|
||||
- [ ] Sections with found inputs are still generated
|
||||
- [ ] "May I write" is asked before any file is written
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; onboard is read-only orientation
|
||||
### Case 5: Director Gate Check — None; No Agents Spawned
|
||||
|
||||
**Fixture:**
|
||||
- Any configured project state
|
||||
- Any project state with `CLAUDE.md` present
|
||||
- Any review mode
|
||||
|
||||
**Input:** `/onboard`
|
||||
**Input:** `/onboard producer`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill completes the full onboarding summary
|
||||
2. No director agents are spawned at any point
|
||||
3. No gate IDs appear in the output
|
||||
4. No "May I write" prompts appear
|
||||
1. Skill scans `production/` for the current sprint and milestone
|
||||
2. No agent of any kind is spawned; no gate IDs appear in output
|
||||
3. Document is presented, then "May I write this to
|
||||
`production/onboarding/onboard-producer-[date].md`?" is asked
|
||||
4. Verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No write tool is called
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Verdict is ONBOARDING COMPLETE without any gate check
|
||||
- [ ] No director gate is invoked and no gate skip message appears
|
||||
- [ ] No subagent is spawned
|
||||
- [ ] `production/` is the area scanned for this role
|
||||
- [ ] Verdict is COMPLETE after the write ask
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Reads all source files before generating output (no hallucinated project state)
|
||||
- [ ] Adapts output to project stage (Production ≠ Concept)
|
||||
- [ ] Respects role argument when provided
|
||||
- [ ] Does not write any files
|
||||
- [ ] Ends with ONBOARDING COMPLETE verdict in all paths
|
||||
- [ ] Reads `CLAUDE.md` and, when a role is given, its agent definition before generating
|
||||
- [ ] Scans only the area that matches the role
|
||||
- [ ] Records each input as FOUND or ABSENT; ABSENT-backed sections read `NOT ASSESSED — NO DATA`
|
||||
- [ ] Presents the document before asking "May I write" to `production/onboarding/onboard-[role]-[date].md`
|
||||
- [ ] Ends with verdict COMPLETE and the `/sprint-status` / `/help` next steps — or, when every input is absent, stops at `NOT ASSESSED — NO DATA` naming what produces each missing input
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- The case where `technical-preferences.md` is missing entirely (as opposed to
|
||||
having placeholders) is not separately tested; behavior follows the graceful
|
||||
error pattern of Case 3.
|
||||
- Git history reading is assumed available; offline/no-git scenarios are not
|
||||
- A call with no role argument (`/onboard`) takes Phase 2's general-orientation
|
||||
branch (the top level of `design/`, `docs/architecture/`, `production/` and
|
||||
the code root, plus `active.md`, and a note that a role would focus the next
|
||||
pass); it is not separately tested.
|
||||
- `narrative` (`design/narrative/`) and QA with an existing `tests/` directory
|
||||
follow the same branch pattern as Cases 1 and 2 and are not separately tested.
|
||||
- `modes.automation: autonomous` (log and proceed instead of asking) is not
|
||||
tested here.
|
||||
- Discipline roles beyond "artist" (e.g., programmer, designer, producer) follow
|
||||
the same tailoring pattern as Case 4 and are not separately tested.
|
||||
|
||||
@@ -2,177 +2,205 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/playtest-report` generates a structured playtest report from session notes or
|
||||
user input. The report is organized into four sections: Feel/Accessibility,
|
||||
Bugs Observed, Design Feedback, and Next Steps. When multiple testers participated,
|
||||
the skill aggregates feedback and distinguishes majority opinions from minority
|
||||
ones. The skill links to existing bug reports when a reported bug matches a file
|
||||
in `production/bugs/`.
|
||||
`/playtest-report` has two modes. `new` outputs a blank playtest report template
|
||||
and stops with Verdict COMPLETE, saving nothing;
|
||||
`analyze [path]` reads raw notes from a file, cross-references the design
|
||||
documents, fills the template with structured findings and flags observations
|
||||
that conflict with design intent. Defects (crashes, framerate drops) go in the
|
||||
Bugs Encountered table, not under Gameplay Flow → Pain points. The template has these sections: Session Info,
|
||||
Test Focus, First Impressions, Gameplay Flow (what worked, pain points,
|
||||
confusion, delight), Bugs Encountered, Feature-Specific Feedback, Quantitative
|
||||
Data, Overall Assessment, and Top 3 Priorities.
|
||||
|
||||
Reports are written to `production/qa/playtest-[date].md` after a "May I write"
|
||||
ask. No director gates apply here — the CD-PLAYTEST director gate (if needed) is
|
||||
a separate invocation. The verdict is COMPLETE when the report is written.
|
||||
Findings are sorted into four buckets and routed: design changes to
|
||||
`/propagate-design-change [path]`, balance adjustments to `/balance-check
|
||||
[system]`, bugs to `/bug-report`, polish items to the `production/` backlog. In
|
||||
`full` review mode the skill spawns `creative-director` for gate **CD-PLAYTEST**;
|
||||
in `lean` and `solo` it skips the gate with a note. It then asks "May I write
|
||||
this playtest report to `production/qa/playtests/playtest-[date]-[tester].md`?".
|
||||
Verdict is **COMPLETE**.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keyword: COMPLETE
|
||||
- [ ] Contains "May I write" collaborative protocol language before writing the report
|
||||
- [ ] Has a next-step handoff (e.g., `/bug-report` for new issues found, `/design-review` for feedback)
|
||||
- [ ] Has a next-step handoff (`/design-review` after design changes, `/bug-triage` after bug fixes)
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
None. `/playtest-report` is a documentation utility. The CD-PLAYTEST gate is a
|
||||
separate invocation and not part of this skill.
|
||||
- **Full mode**: CD-PLAYTEST — `creative-director` reviews the report against the
|
||||
game pillars and core fantasy (APPROVE / CONCERNS / REJECT, or NOT ASSESSED
|
||||
when it lacked an input), after findings are categorised and before the report
|
||||
is saved
|
||||
- **Lean mode**: CD-PLAYTEST skipped (not a PHASE-GATE); note
|
||||
"CD-PLAYTEST skipped — Lean mode."
|
||||
- **Solo mode**: CD-PLAYTEST skipped; note "CD-PLAYTEST skipped — Solo mode."
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — User provides playtest notes, structured report produced
|
||||
### Case 1: Happy Path — Analyze Notes, Findings Routed by Bucket
|
||||
|
||||
**Fixture:**
|
||||
- User provides typed playtest notes from a single session
|
||||
- Notes cover: game feel, one bug (framerate drop), and a design concern
|
||||
(tutorial too long)
|
||||
- `production/bugs/` exists but is empty (bug not yet reported)
|
||||
- `production/qa/notes/session-01.md` holds one tester's notes: "framerate
|
||||
drops every time the forest area loads", "I could not work out how to open
|
||||
the first door", and "the second boss hits far too hard"
|
||||
- `design/gdd/` holds the tutorial and combat GDDs; the tutorial GDD says the
|
||||
first door teaches the interact key
|
||||
- Review mode: `solo`
|
||||
|
||||
**Input:** `/playtest-report` (user pastes session notes)
|
||||
**Input:** `/playtest-report analyze production/qa/notes/session-01.md`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads the provided notes and structures them into the 4-section template
|
||||
2. Feel/Accessibility: extracts feel observations
|
||||
3. Bugs: notes the framerate drop with available repro details
|
||||
4. Design Feedback: notes the tutorial length concern
|
||||
5. Next Steps: suggests `/bug-report` for the framerate issue and `/design-review`
|
||||
for the tutorial feedback
|
||||
6. Skill asks "May I write to `production/qa/playtest-2026-04-06.md`?"
|
||||
7. Report is written on approval; verdict is COMPLETE
|
||||
1. Skill reads the notes and cross-references the design documents
|
||||
2. Skill fills the template; the framerate drop goes in the Bugs Encountered table
|
||||
3. Phase 3 presents findings in four buckets: the door confusion as a design
|
||||
change, the boss damage as a balance adjustment, the framerate drop as a bug
|
||||
4. Routing: `/propagate-design-change` on the tutorial GDD, `/balance-check` for
|
||||
combat, `/bug-report` for the framerate drop
|
||||
5. Skill notes "CD-PLAYTEST skipped — Solo mode." and asks "May I write this
|
||||
playtest report to `production/qa/playtests/playtest-[date]-[tester].md`?"
|
||||
6. On approval the file is written; verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All 4 sections are present in the report
|
||||
- [ ] Bug is listed in the Bugs section (not the Design Feedback section)
|
||||
- [ ] Next Steps are appropriate (bug report for crash, design review for feedback)
|
||||
- [ ] "May I write" is asked before writing
|
||||
- [ ] The framerate drop appears in the Bugs Encountered table, not under Gameplay Flow
|
||||
- [ ] Findings are presented in the four buckets (design, balance, bug, polish)
|
||||
- [ ] Each bucket is routed to its named skill (`/propagate-design-change`, `/balance-check`, `/bug-report`)
|
||||
- [ ] "May I write" names `production/qa/playtests/playtest-[date]-[tester].md` before writing
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Empty Input — Guided prompting through each section
|
||||
### Case 2: New Mode — Blank Template Output
|
||||
|
||||
**Fixture:**
|
||||
- No notes provided by user at invocation
|
||||
- Any project state
|
||||
|
||||
**Input:** `/playtest-report`
|
||||
**Input:** `/playtest-report new`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill detects empty input
|
||||
2. Skill prompts through each section:
|
||||
a. "Describe the overall feel and any accessibility observations"
|
||||
b. "Were any bugs observed? Describe them"
|
||||
c. "What design feedback did testers provide?"
|
||||
3. User answers each prompt
|
||||
4. Skill compiles report from answers and asks "May I write"
|
||||
5. Report written on approval; verdict is COMPLETE
|
||||
1. Skill selects `new` mode
|
||||
2. Skill outputs the blank template with every section heading and its
|
||||
bracketed placeholders unfilled
|
||||
3. No notes file is read and no findings are invented
|
||||
4. The mode ends after the template: no routing, no director gate, no save
|
||||
ask, and the verdict is COMPLETE — blank template output; nothing saved
|
||||
|
||||
**Assertions:**
|
||||
- [ ] At least 3 guiding questions are asked (one per main section)
|
||||
- [ ] Report is not created until all sections have input (or user explicitly skips one)
|
||||
- [ ] Verdict is COMPLETE after file is written
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Multiple Testers — Aggregated feedback with majority/minority notes
|
||||
|
||||
**Fixture:**
|
||||
- User provides notes from 3 testers
|
||||
- 2/3 testers found the controls "intuitive"
|
||||
- 1/3 tester found the UI font too small
|
||||
- All 3 noted the same bug (player stuck on ledge)
|
||||
|
||||
**Input:** `/playtest-report` (3-tester session)
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill identifies 3 distinct tester perspectives in the input
|
||||
2. Control intuitiveness → noted as "Majority (2/3): controls intuitive"
|
||||
3. Font size → noted as "Minority (1/3): UI font size concern"
|
||||
4. Stuck-on-ledge bug → noted as "All testers: player stuck on ledge (confirmed)"
|
||||
5. Skill generates aggregated report with majority/minority labels
|
||||
6. Report written after "May I write" approval; verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Majority opinion (2/3) is labeled as majority
|
||||
- [ ] Minority opinion (1/3) is labeled as minority
|
||||
- [ ] Unanimously reported bug is noted as confirmed by all testers
|
||||
- [ ] Output contains the headings Session Info, Test Focus, First Impressions, Gameplay Flow, Bugs Encountered, Feature-Specific Feedback, Quantitative Data, Overall Assessment and Top 3 Priorities
|
||||
- [ ] Placeholders (e.g. `[Date]`, `[Name/ID]`) are left unfilled
|
||||
- [ ] No notes file is read and no findings are fabricated
|
||||
- [ ] No "May I write" ask and no file written
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Bug Matches Existing Report — Links to existing file
|
||||
### Case 3: Observation Conflicts With Design Intent
|
||||
|
||||
**Fixture:**
|
||||
- `production/bugs/bug-2026-03-30-player-stuck-ledge.md` exists
|
||||
- User's playtest notes describe "player gets stuck on ledges near walls"
|
||||
- `design/gdd/crafting.md` states crafting is a core loop players engage with
|
||||
every session
|
||||
- `production/qa/notes/session-02.md` records that the tester never opened the
|
||||
crafting menu in 40 minutes
|
||||
- Review mode: `solo`
|
||||
|
||||
**Input:** `/playtest-report`
|
||||
**Input:** `/playtest-report analyze production/qa/notes/session-02.md`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill structures the report and identifies the stuck-on-ledge bug
|
||||
2. Skill scans `production/bugs/` and finds `bug-2026-03-30-player-stuck-ledge.md`
|
||||
3. In the Bugs section, the report includes: "See existing report:
|
||||
production/bugs/bug-2026-03-30-player-stuck-ledge.md"
|
||||
4. Skill does NOT suggest creating a new bug report for this issue
|
||||
5. Report written; verdict is COMPLETE
|
||||
1. Skill reads the notes and cross-references `design/gdd/crafting.md`
|
||||
2. Skill flags the observation as conflicting with design intent
|
||||
3. Phase 3 puts it in the "Design changes needed" bucket
|
||||
4. Routing names `/propagate-design-change` on `design/gdd/crafting.md`
|
||||
5. Report saved after the "May I write" ask; verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Existing bug report is found and linked in the playtest report
|
||||
- [ ] `/bug-report` is NOT suggested for the already-reported issue
|
||||
- [ ] Cross-reference to existing file appears in the Bugs section
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] The crafting observation is explicitly flagged as conflicting with the GDD's intent
|
||||
- [ ] It is categorised as a design change, not a polish item
|
||||
- [ ] `/propagate-design-change` is suggested on the affected GDD
|
||||
- [ ] Verdict is COMPLETE after the write
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; CD-PLAYTEST is a separate invocation
|
||||
### Case 4: Edge Case — Full Review Mode With No Pillars Available
|
||||
|
||||
**Fixture:**
|
||||
- Playtest notes provided
|
||||
- Review mode: `full`
|
||||
- Neither `design/gdd/game-concept.md` nor `design/game-brief.md` exists
|
||||
- `production/qa/notes/session-03.md` holds playtest notes
|
||||
|
||||
**Input:** `/playtest-report`
|
||||
**Input:** `/playtest-report analyze production/qa/notes/session-03.md`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill generates and writes the playtest report
|
||||
2. No director agents are spawned (CD-PLAYTEST is not invoked here)
|
||||
3. No gate IDs appear in output
|
||||
1. Skill structures and categorises the findings
|
||||
2. Skill spawns `creative-director` for CD-PLAYTEST with the report content and
|
||||
the hypothesis being tested
|
||||
3. Because neither pillars source exists, the prompt states "No pillars
|
||||
available — assess against the hypothesis alone." rather than omitting pillars
|
||||
4. The director's assessment is presented before the save ask
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No CD-PLAYTEST gate skip message appears
|
||||
- [ ] Verdict is COMPLETE without any gate check
|
||||
- [ ] CD-PLAYTEST is spawned (review mode is `full`)
|
||||
- [ ] The gate prompt says no pillars are available instead of passing none silently
|
||||
- [ ] Pillars are not invented by the skill
|
||||
- [ ] The assessment is shown before "May I write"
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate — CD-PLAYTEST by Review Mode
|
||||
|
||||
**Fixture:**
|
||||
- `design/gdd/game-concept.md` exists with pillars
|
||||
- `production/qa/notes/session-04.md` holds playtest notes
|
||||
- Run three times: `--review full`, `--review lean`, `--review solo`
|
||||
- In the full run, `creative-director` returns CONCERNS
|
||||
|
||||
**Input:** `/playtest-report analyze production/qa/notes/session-04.md --review [mode]`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Full: after Phase 3 categorisation, `creative-director` is spawned for
|
||||
CD-PLAYTEST with the report content, pillars from `game-concept.md` and the
|
||||
hypothesis; its CONCERNS assessment is presented and a
|
||||
`## Creative Director Assessment` section is added to the report
|
||||
2. Lean: no director spawned; note "CD-PLAYTEST skipped — Lean mode."
|
||||
3. Solo: no director spawned; note "CD-PLAYTEST skipped — Solo mode."
|
||||
4. All three runs then ask "May I write" and end with verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] In full mode, CD-PLAYTEST spawns `creative-director` before the report is saved
|
||||
- [ ] On CONCERNS, the report gains a `## Creative Director Assessment` section with the verdict and feedback
|
||||
- [ ] In lean mode, the skip note names CD-PLAYTEST and Lean mode
|
||||
- [ ] In solo mode, the skip note names CD-PLAYTEST and Solo mode
|
||||
- [ ] No other director gate is spawned in any mode
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Structures output into all 4 sections (Feel, Bugs, Design Feedback, Next Steps)
|
||||
- [ ] Labels majority vs. minority opinions when multiple testers are involved
|
||||
- [ ] Cross-references existing bug reports when bugs match
|
||||
- [ ] Asks "May I write to `production/qa/playtest-[date].md`?" before writing
|
||||
- [ ] Verdict is COMPLETE when report is written
|
||||
- [ ] `analyze` reads the notes file and cross-references design documents before filling the template
|
||||
- [ ] Findings are categorised into design / balance / bug / polish and routed to the named follow-up
|
||||
- [ ] Applies the review mode before spawning CD-PLAYTEST and prints the skip note when skipped
|
||||
- [ ] Presents the director's assessment (full mode) before asking to save
|
||||
- [ ] Asks "May I write this playtest report to `production/qa/playtests/playtest-[date]-[tester].md`?" before writing
|
||||
- [ ] Ends with verdict COMPLETE and next steps (`/design-review`, `/bug-triage`)
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- The CD-PLAYTEST director gate (creative director reviews playtest insights
|
||||
for design implications) is a separate invocation and is not tested here.
|
||||
- Video recording or screenshot attachments are not tested; the report is a
|
||||
text-only document.
|
||||
- The case where a tester's identity is unknown (anonymous feedback) follows
|
||||
the same aggregation pattern as Case 3 without tester labels.
|
||||
- A REJECT verdict from CD-PLAYTEST follows the same path as CONCERNS (an
|
||||
assessment section is added) and is not separately tested; APPROVE only adds
|
||||
an approval note. NOT ASSESSED (the director lacked an input) adds the section
|
||||
naming what was missing and is never read as an approval; not separately tested.
|
||||
- At `modes.workflow: minimal`, where `design/game-brief.md` replaces
|
||||
`game-concept.md`, the brief has no pillars: its one-sentence pitch and "Who
|
||||
it's for / what they feel" line are passed in their place; not separately tested.
|
||||
- Video or screenshot attachments are not tested; the report is text only.
|
||||
|
||||
@@ -2,182 +2,253 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/project-stage-detect` automatically analyzes project artifacts to determine
|
||||
the current development stage. It runs on the Haiku model (read-only) and
|
||||
examines `production/stage.txt` (if present), design documents in `design/`,
|
||||
source code in `src/`, sprint and milestone files in `production/`, and the
|
||||
presence of engine configuration to classify the project into one of seven
|
||||
stages: Concept, Systems Design, Technical Setup, Pre-Production, Production,
|
||||
Polish, or Release.
|
||||
`/project-stage-detect [role]` determines the project's development stage and
|
||||
its gaps. It starts with a deterministic pass (`bash
|
||||
.claude/scripts/artifact-check.sh`, which reports PRESENT / ABSENT / SHORT /
|
||||
PATTERN_MISS / NO_CHECK per artifact), then scans `design/`, the code root,
|
||||
`production/`, `prototypes/`, `docs/architecture/` and `tests/`.
|
||||
|
||||
The skill is advisory — it never writes `stage.txt`. That file is only updated
|
||||
when `/gate-check` passes and the user confirms advancement. The skill reports
|
||||
its confidence level (HIGH if stage.txt was read directly, MEDIUM if inferred
|
||||
from artifacts, LOW if conflicting signals were found).
|
||||
A configured stage (`project.stage` in `project.yaml`, else legacy
|
||||
`production/stage.txt`) is authoritative for what the stage is, but the skill
|
||||
always runs its heuristics too and states explicitly when the artifacts
|
||||
disagree. Heuristics, checked from most advanced backward: Production = code
|
||||
root has 10+ source files; Pre-Production = engine configured, <10 source files;
|
||||
Technical Setup = systems index, no engine; Systems Design = concept, no
|
||||
complete systems index; Concept = no concept doc; Polish and Release are
|
||||
explicit only. The resolved workflow tier decides which absent documents count
|
||||
as gaps (none of GDDs/ADRs/sprints at `minimal`). An unresolved code root means
|
||||
the source files were not counted: `NOT ASSESSED — code root unresolved`, never
|
||||
zero files.
|
||||
|
||||
The report carries **Stage Confidence**, first match wins: FAIL (critical gaps
|
||||
block progress) → CONCERNS (ambiguous signals — always when the configured and
|
||||
observed stages disagree) → NOT ASSESSED (a check the stage depends on did not
|
||||
run, named) → PASS (clearly detected). At `workflow: minimal` the configured
|
||||
stage is never advanced (no `/gate-check` on that path), so its lag behind the
|
||||
observed stage is reported but is not a disagreement. The skill shows the
|
||||
summary, gaps and next steps, then asks "May I write the full stage analysis to
|
||||
production/project-stage-report.md?". It never writes the stage itself. No
|
||||
director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains all seven stage names: Concept, Systems Design, Technical Setup, Pre-Production, Production, Polish, Release
|
||||
- [ ] Does NOT contain "May I write" language (skill is detection-only)
|
||||
- [ ] Has a next-step handoff (e.g., `/gate-check` to formally advance stage)
|
||||
- [ ] Contains the Stage Confidence values PASS, CONCERNS, NOT ASSESSED and FAIL, with a first-match order
|
||||
- [ ] Contains "May I write" language before writing `production/project-stage-report.md`
|
||||
- [ ] Has a next-step handoff (Follow-Up Actions naming `/map-systems`, `/reverse-document`, `/architecture-decision`, `/sprint-plan`, `/milestone-review`)
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
None. `/project-stage-detect` is a read-only detection utility. No director
|
||||
gates apply.
|
||||
None. `/project-stage-detect` is a diagnostic utility. It spawns no agents
|
||||
(`Agent` is not in its allowed tools) and no director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: stage.txt Exists — Reads directly and cross-checks artifacts
|
||||
### Case 1: Configured Stage Matches the Artifacts
|
||||
|
||||
**Fixture:**
|
||||
- `production/stage.txt` contains `Production`
|
||||
- `design/gdd/` has 4 GDD files
|
||||
- `src/` has source code files
|
||||
- `project.yaml` has `project.stage: Production` and `engine.name: Godot`
|
||||
- `src/` has 24 source files; `design/gdd/` has 4 GDDs and `systems-index.md`,
|
||||
one GDD per indexed system
|
||||
- `docs/architecture/` has `architecture.md` and 5 ADRs; `design/art/art-bible.md`
|
||||
and the UX specs exist; `tests/unit/` holds tests for each system — no doc type
|
||||
the `full` tier flags is missing, so there is no gap to weigh
|
||||
- `production/sprints/sprint-002.md` exists
|
||||
- Resolved workflow tier: `full`
|
||||
|
||||
**Input:** `/project-stage-detect`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads `production/stage.txt` — detects stage `Production`
|
||||
2. Skill cross-checks artifacts: GDDs present, source code present, sprint present
|
||||
3. Artifacts are consistent with Production stage
|
||||
4. Skill reports: Stage = Production, Confidence = HIGH (from stage.txt, confirmed by artifacts)
|
||||
5. Next step: continue with `/sprint-plan` or `/dev-story`
|
||||
1. Skill runs `bash .claude/scripts/artifact-check.sh` before hand-scanning
|
||||
2. Skill reads the configured stage `Production` from `project.yaml`
|
||||
3. Skill still runs the heuristics; 24 source files (10+) indicate Production
|
||||
4. Report states the configured and observed stages agree; Stage Confidence is PASS
|
||||
5. Skill shows the summary, gaps and next steps, then asks "May I write the full
|
||||
stage analysis to production/project-stage-report.md?"
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Detected stage is Production
|
||||
- [ ] Confidence is reported as HIGH when stage.txt is present
|
||||
- [ ] Cross-check result (consistent vs. discrepant) is noted
|
||||
- [ ] No files are written
|
||||
- [ ] Verdict clearly states the detected stage
|
||||
- [ ] `artifact-check.sh` is run as the first scan step
|
||||
- [ ] Detected stage is Production, and the heuristic result is reported alongside the configured one
|
||||
- [ ] Stage Confidence is PASS
|
||||
- [ ] "May I write" names `production/project-stage-report.md` and nothing is written before approval
|
||||
- [ ] Neither `project.yaml` nor `production/stage.txt` is modified
|
||||
|
||||
---
|
||||
|
||||
### Case 2: No stage.txt but GDDs and Epics Exist — Infers Production
|
||||
### Case 2: No Configured Stage — Inferred From Artifacts, Role Filter Applied
|
||||
|
||||
**Fixture:**
|
||||
- No `production/stage.txt`
|
||||
- `design/gdd/` has 3 GDD files
|
||||
- `production/epics/` has 2 epic files
|
||||
- `src/` has source code files
|
||||
- `production/sprints/sprint-001.md` exists
|
||||
- No `project.stage` in `project.yaml` and no `production/stage.txt`
|
||||
- `engine.name: Godot`; `src/` has 14 source files
|
||||
- `design/gdd/` has 3 GDDs; `docs/architecture/` has no ADRs
|
||||
- Resolved workflow tier: `full`
|
||||
|
||||
**Input:** `/project-stage-detect`
|
||||
**Input:** `/project-stage-detect programmer`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill finds no stage.txt — switches to artifact inference mode
|
||||
2. Skill finds GDDs (Systems Design complete), epics (Pre-Production complete),
|
||||
source code and sprints (Production active)
|
||||
3. Skill infers: Stage = Production
|
||||
4. Confidence is MEDIUM (inferred from artifacts, not from stage.txt)
|
||||
5. Skill recommends running `/gate-check` to formalize and write stage.txt
|
||||
1. Skill finds no configured stage and auto-detects from the heuristics
|
||||
2. 14 source files (10+) → stage Production
|
||||
3. The programmer role focuses recommendations on architecture docs, test
|
||||
coverage and missing ADRs
|
||||
4. The missing ADRs are surfaced as a gap with a clarifying question, and
|
||||
`/architecture-decision` or `/reverse-document architecture` is suggested
|
||||
5. Skill asks before writing the report; no stage value is written anywhere
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Inferred stage is Production
|
||||
- [ ] Confidence is MEDIUM (not HIGH, since stage.txt is absent)
|
||||
- [ ] Recommendation to run `/gate-check` is present
|
||||
- [ ] No stage.txt is written by this skill
|
||||
- [ ] Inferred stage is Production, based on the 10+ source-file rule
|
||||
- [ ] Recommendations focus on architecture docs, tests and ADRs for the programmer role
|
||||
- [ ] The ADR gap is phrased with a clarifying question, not only listed
|
||||
- [ ] No stage value is written to `project.yaml` or `production/stage.txt`
|
||||
|
||||
---
|
||||
|
||||
### Case 3: No stage.txt, No Docs, No Source — Infers Concept
|
||||
### Case 3: Configured Stage Contradicted by the Artifacts
|
||||
|
||||
**Fixture:**
|
||||
- No `production/stage.txt`
|
||||
- `design/` directory exists but is empty
|
||||
- `src/` exists but contains no code files
|
||||
- `technical-preferences.md` has placeholders only
|
||||
- `project.yaml` has `project.stage: Release`
|
||||
- `engine.name: Godot`; `src/` has 2 source files; no ADRs, no architecture
|
||||
doc, no epics
|
||||
- Resolved workflow tier: `full`
|
||||
|
||||
**Input:** `/project-stage-detect`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill finds no stage.txt
|
||||
2. Artifact scan: no GDDs, no source, no epics, no sprints, engine unconfigured
|
||||
3. Skill infers: Stage = Concept
|
||||
4. Confidence is MEDIUM
|
||||
5. Skill suggests `/start` to begin the onboarding workflow
|
||||
1. Skill reads the configured stage `Release`
|
||||
2. Heuristics indicate Pre-Production (engine configured, fewer than 10 source
|
||||
files)
|
||||
3. Skill reports both and says explicitly that they disagree — the configured
|
||||
stage may be stale, or work exists outside this repo
|
||||
4. The configured value is not silently replaced, and the disagreement is not
|
||||
hidden behind a confident result
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Inferred stage is Concept
|
||||
- [ ] Output lists the artifacts that were checked (and found absent)
|
||||
- [ ] `/start` is suggested as the next step
|
||||
- [ ] No files are written
|
||||
- [ ] Output names both the configured stage (Release) and the observed stage (Pre-Production)
|
||||
- [ ] The disagreement is stated explicitly
|
||||
- [ ] Stage Confidence is CONCERNS (ambiguous signals), or FAIL if critical gaps are also found — never PASS
|
||||
- [ ] The configured stage is neither silently overridden nor rewritten in any file
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Discrepancy — stage.txt says Production but no source code
|
||||
### Case 4: Minimal Workflow Tier — Absent Docs Are Not Gaps
|
||||
|
||||
**Fixture:**
|
||||
- `production/stage.txt` contains `Production`
|
||||
- `design/gdd/` has GDD files
|
||||
- `src/` directory exists but contains no source code files
|
||||
- No sprint files exist
|
||||
- Resolved workflow tier: `minimal`
|
||||
- `project.yaml` has `project.stage: Concept`, as `/start` wrote it — nothing on
|
||||
the minimal path runs `/gate-check`, so it has never advanced
|
||||
- `design/game-brief.md` exists; `engine.name: Godot`; `src/` has 5 source files
|
||||
- No GDDs, no art bible, no ADRs, no epics, no sprint plans
|
||||
|
||||
**Input:** `/project-stage-detect`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads stage.txt — detects `Production`
|
||||
2. Cross-check finds: no source code, no sprints — inconsistent with Production
|
||||
3. Skill flags discrepancy: "stage.txt says Production but no source code or sprints found"
|
||||
4. Skill reports detected stage as Production (honoring stage.txt) but
|
||||
confidence drops to LOW due to artifact mismatch
|
||||
5. Skill suggests reviewing stage.txt manually or running `/gate-check`
|
||||
1. Skill uses the resolved tier `minimal` rather than assuming one
|
||||
2. It treats a brief plus an engine as the normal state
|
||||
3. It does NOT flag absent GDDs, art bible, UX specs, ADRs, epics or sprint
|
||||
plans as gaps
|
||||
4. It does not suggest `/reverse-document` for an absent GDD or `/sprint-plan`
|
||||
5. The Production line reports stories through the brief's build order, never
|
||||
"no sprint plan"
|
||||
6. The heuristics observe Pre-Production (engine configured, fewer than 10 source
|
||||
files); the configured Concept is reported as not advanced at this tier — an
|
||||
expected lag, not a disagreement — so it does not by itself make Stage
|
||||
Confidence CONCERNS
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Discrepancy is flagged explicitly in the output
|
||||
- [ ] Confidence is LOW when artifacts contradict stage.txt
|
||||
- [ ] stage.txt value is not silently overridden
|
||||
- [ ] User is advised to verify the discrepancy manually
|
||||
- [ ] No absent GDD, art bible, ADR, epic or sprint plan is listed as a gap
|
||||
- [ ] `/reverse-document` and `/sprint-plan` are not suggested
|
||||
- [ ] The Production completeness line refers to the brief's build order
|
||||
- [ ] The configured Concept stage is not reported as a disagreement with the observed stage
|
||||
- [ ] Report is still offered behind the "May I write" ask
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; detection is advisory
|
||||
### Case 5: Director Gate Check — None; Declined Write Leaves No File
|
||||
|
||||
**Fixture:**
|
||||
- Any project state with or without stage.txt
|
||||
- Any project state
|
||||
- Any review mode
|
||||
- User declines the "May I write" ask
|
||||
|
||||
**Input:** `/project-stage-detect`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill completes full stage detection
|
||||
2. No director agents are spawned at any point
|
||||
3. No gate IDs appear in output
|
||||
4. No write tool is called
|
||||
1. Skill completes detection and shows the summary
|
||||
2. No director or other agent is spawned; no gate IDs appear
|
||||
3. Skill asks "May I write the full stage analysis to
|
||||
production/project-stage-report.md?"
|
||||
4. User declines; no file is created
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No write tool is called
|
||||
- [ ] Detection output is purely advisory
|
||||
- [ ] Verdict names the detected stage without triggering any gate
|
||||
- [ ] No director gate is invoked and no gate skip message appears
|
||||
- [ ] No subagent is spawned
|
||||
- [ ] The summary is shown before the write ask
|
||||
- [ ] After a decline, `production/project-stage-report.md` is not created
|
||||
|
||||
---
|
||||
|
||||
### Case 6: Unresolved Code Root — NOT ASSESSED, not a greenfield project
|
||||
|
||||
**Fixture:**
|
||||
- No `project.stage`, no `production/stage.txt`
|
||||
- `project.yaml` has no `engine.name`; `.claude/docs/technical-preferences.md`
|
||||
reads `[TO BE CONFIGURED]`
|
||||
- Both `src/` and `Assets/` exist, each holding source files (ambiguous tree)
|
||||
- `design/gdd/game-concept.md` and `design/gdd/systems-index.md` exist, with a
|
||||
GDD for every indexed system; `docs/architecture/` holds the Foundation-layer
|
||||
ADRs — nothing the `standard` tier flags is missing
|
||||
- Resolved workflow tier: `standard`
|
||||
|
||||
**Input:** `/project-stage-detect`
|
||||
|
||||
**Expected behavior:**
|
||||
1. The code root cannot be resolved (no engine, two candidate roots), so the
|
||||
source-file count does not run
|
||||
2. The Source Code section reads `NOT ASSESSED — code root unresolved` — not
|
||||
"0 source files", and the project is not called greenfield
|
||||
3. Without the count the Production row (10+ source files) can be neither ruled
|
||||
in nor out — the rows below it would say Technical Setup (systems index, no
|
||||
engine) — so the report says which check did not run instead of settling
|
||||
silently on the lower stage
|
||||
4. No stage is configured and there are no gaps, so nothing ranks above it:
|
||||
Stage Confidence is **NOT ASSESSED**, naming the unresolved code root. The
|
||||
two candidate roots are the reason the check could not run, not an
|
||||
ambiguous signal (CONCERNS)
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No source-file count of zero is reported; the Source Code line is `NOT ASSESSED — code root unresolved`
|
||||
- [ ] The code root is not assumed to be `src/`
|
||||
- [ ] Stage Confidence is NOT ASSESSED and names the check that did not run — not PASS
|
||||
- [ ] The report is still offered behind the "May I write" ask
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Reads stage.txt if present; falls back to artifact inference if absent
|
||||
- [ ] Always reports a confidence level (HIGH / MEDIUM / LOW)
|
||||
- [ ] Cross-checks stage.txt against artifacts and flags discrepancies
|
||||
- [ ] Does not write stage.txt (that is `/gate-check`'s responsibility)
|
||||
- [ ] Ends with a next-step recommendation appropriate to the detected stage
|
||||
- [ ] Runs `artifact-check.sh` before hand-scanning artifacts
|
||||
- [ ] Reads `project.stage` (then `production/stage.txt`) when present, and always runs the heuristics as well
|
||||
- [ ] States explicitly when configured and observed stages disagree
|
||||
- [ ] Surfaces gaps only as the resolved workflow tier allows, each with a clarifying question
|
||||
- [ ] Reports Stage Confidence as PASS / CONCERNS / NOT ASSESSED / FAIL, first match FAIL → CONCERNS → NOT ASSESSED → PASS; a configured/observed disagreement is CONCERNS
|
||||
- [ ] Reports an unresolved code root as `NOT ASSESSED — code root unresolved`, never as zero source files
|
||||
- [ ] Asks "May I write" before writing `production/project-stage-report.md`; never writes a stage value
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- The Technical Setup stage (engine configured, no GDDs yet) and Pre-Production
|
||||
stage (GDDs complete, no epics yet) follow the same artifact-inference pattern
|
||||
as Cases 2 and 3 and are not separately fixture-tested.
|
||||
- The Polish and Release stages are not fixture-tested here; they follow the
|
||||
same high-confidence (stage.txt present) or inference logic.
|
||||
- Confidence levels are advisory — the skill does not gate any actions on them.
|
||||
- Concept (no concept doc) and Technical Setup (systems index, no engine) follow
|
||||
the same heuristic table and are not separately fixture-tested.
|
||||
- Polish and Release are explicit-only stages; they can only come from a
|
||||
configured value, which Case 3 covers.
|
||||
- The `standard` tier (flag missing GDDs for built systems and critical ADRs
|
||||
only) is not separately tested.
|
||||
|
||||
@@ -2,177 +2,275 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/prototype` manages a rapid prototyping workflow for validating a game mechanic
|
||||
before committing to full production implementation. Prototypes are created in
|
||||
`prototypes/[mechanic-name]/` and are intentionally disposable — coding standards
|
||||
are relaxed (no ADR required, AC can be minimal, hardcoded values acceptable).
|
||||
After implementation, the skill produces a findings document summarizing what
|
||||
was learned and recommending next steps.
|
||||
`/prototype [concept] [--path html|engine|paper] [--review full|lean|solo]
|
||||
[--spike]` runs a throwaway concept prototype before GDDs are written. It first
|
||||
confirms intent with `AskUserQuestion` (Prototype this concept / Skip — concept
|
||||
already proven / Mid-production spike), forms a falsifiable hypothesis ("If the
|
||||
player [does X], they will feel [Y] — we will know this is true if [Z]") and asks
|
||||
for the riskiest assumption. It picks an HTML, Engine or Paper path (genre table,
|
||||
recommendation pre-stated), presents a 3–5 bullet plan, asks to record it as a
|
||||
checkpoint in `production/session-state/active.md`, then asks "May I create the prototype
|
||||
directory at `prototypes/[concept-name]-concept/` and begin implementation?".
|
||||
Every prototype file starts with a `PROTOTYPE - NOT FOR PRODUCTION` header in
|
||||
that file's own comment syntax (`#` in GDScript and Python, `//` in C#, C++ and
|
||||
JavaScript, `<!-- -->` in HTML and Markdown), and standards are intentionally
|
||||
relaxed.
|
||||
|
||||
The skill asks "May I write to `prototypes/[name]/`?" before creating files. If a
|
||||
prototype already exists, the skill offers to extend, replace, or archive. No
|
||||
director gates apply. Verdicts: PROTOTYPE COMPLETE (prototype built and findings
|
||||
documented) or PROTOTYPE ABANDONED (mechanic found to be unworkable).
|
||||
After a one-question-at-a-time debrief, it fills
|
||||
`.claude/docs/templates/prototype-report.md`, asks to write
|
||||
`prototypes/[concept-name]-concept/REPORT.md` and add its row to
|
||||
`prototypes/index.md` (one ask naming both). Verdicts are **PROCEED / PIVOT / KILL**. PIVOT writes a
|
||||
`PIVOT-NOTE.md`; KILL runs a five-item soundness check and appends to
|
||||
`prototypes/GRAVEYARD.md`. In `full` review mode `creative-director` reviews the
|
||||
result via **CD-PLAYTEST** and answers APPROVE / CONCERNS / REJECT: APPROVE keeps
|
||||
the recommendation; CONCERNS go to the user, who decides (Revise / Accept /
|
||||
Discuss); REJECT means a PROCEED cannot stand — the user chooses PIVOT or KILL,
|
||||
and REPORT.md records the director's reason. A NOT ASSESSED answer (the director
|
||||
lacked an input) is never an approval. `--spike` replaces the
|
||||
whole flow with a ~4-hour spike that asks before creating its folder, asks
|
||||
before writing `SPIKE-NOTE.md` and clearing the spike from `active.md`, and has
|
||||
no CD gate and no PROCEED/PIVOT/KILL.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keywords: PROTOTYPE COMPLETE, PROTOTYPE ABANDONED
|
||||
- [ ] Contains "May I write" language before creating prototype files
|
||||
- [ ] Has a next-step handoff (e.g., `/design-system` to formalize, or archive)
|
||||
- [ ] Contains verdict keywords: PROCEED, PIVOT, KILL, NOT ASSESSED (a prototype nobody has played)
|
||||
- [ ] Contains "May I create" / "May I write" language before creating the prototype directory and the report
|
||||
- [ ] Has a next-step handoff per verdict (PROCEED → `/design-review`, `/gate-check`, `/map-systems`, `/design-system`; PIVOT → `/prototype` or `/brainstorm`; KILL → `/brainstorm`)
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
None. Prototypes are throwaway validation artifacts. No director gates apply.
|
||||
- **Full mode**: CD-PLAYTEST — `creative-director` receives the REPORT.md
|
||||
content, the hypothesis and the pillars (from `design/gdd/game-concept.md`, or
|
||||
the pitch and "what they feel" line from `design/game-brief.md`) and returns
|
||||
APPROVE / CONCERNS / REJECT, or NOT ASSESSED when it lacked an input.
|
||||
APPROVE → the recommendation stands; CONCERNS → shown with the recommendation,
|
||||
and the user decides (Revise / Accept / Discuss); REJECT → a PROCEED cannot
|
||||
stand: the user chooses PIVOT or KILL, and REPORT.md records the director's
|
||||
reason; NOT ASSESSED → never an approval: the missing input is named, and the
|
||||
gate is re-run or the review recorded as NOT ASSESSED. If neither document
|
||||
exists: "CD-PLAYTEST skipped — game pillars not yet defined at concept
|
||||
prototype stage."
|
||||
- **Lean mode**: skipped — "CD-PLAYTEST skipped — Lean mode."
|
||||
- **Solo mode**: skipped — "CD-PLAYTEST skipped — Solo mode."
|
||||
- **Spike mode**: no CD gate and no phase gate, in any review mode.
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — Mechanic concept prototyped, findings documented
|
||||
### Case 1: Happy Path — Engine Prototype, PROCEED
|
||||
|
||||
**Fixture:**
|
||||
- `prototypes/` directory exists
|
||||
- No existing prototype for "grapple-hook"
|
||||
- `design/gdd/game-concept.md` describes a 2D platformer; `project.yaml` has
|
||||
`engine.name: Godot` and `engine.language: GDScript`
|
||||
- `prototypes/` has no `grapple-hook-concept/`
|
||||
- Review mode: `solo`
|
||||
- In the debrief the user reports the hypothesis CONFIRMED and answers PROCEED
|
||||
|
||||
**Input:** `/prototype grapple-hook`
|
||||
**Input:** `/prototype grapple-hook traversal`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill asks "May I write to `prototypes/grapple-hook/`?"
|
||||
2. After approval: creates `prototypes/grapple-hook/` directory and basic
|
||||
implementation skeleton (main scene, player controller extension)
|
||||
3. Skill implements a minimal grapple hook mechanic (intentionally rough — no
|
||||
polish, hardcoded values acceptable)
|
||||
4. Skill produces `prototypes/grapple-hook/findings.md` with:
|
||||
- What was tested
|
||||
- What worked
|
||||
- What didn't work
|
||||
- Recommendation (proceed / abandon / revise concept)
|
||||
5. Verdict is PROTOTYPE COMPLETE
|
||||
1. `AskUserQuestion` confirms intent; user picks "Prototype this concept"
|
||||
2. Skill states a falsifiable hypothesis in the If / feel / know-if form and asks
|
||||
for the riskiest assumption
|
||||
3. Path widget pre-states an Engine recommendation (traversal feel is the
|
||||
hypothesis)
|
||||
4. Plan is presented and confirmed, with the ask naming
|
||||
`production/session-state/active.md`; the checkpoint is written there
|
||||
5. Skill asks "May I create the prototype directory at
|
||||
`prototypes/grapple-hook-concept/` and begin implementation?"; every GDScript
|
||||
file it writes starts with `# PROTOTYPE - NOT FOR PRODUCTION` (GDScript
|
||||
comments use `#`; a `//` header would not parse)
|
||||
6. Debrief questions are asked one at a time; report drafted from
|
||||
`.claude/docs/templates/prototype-report.md`; "May I write this report to
|
||||
`prototypes/grapple-hook-concept/REPORT.md` and add its row to
|
||||
`prototypes/index.md`?"; a row is appended to `prototypes/index.md`
|
||||
7. Note "CD-PLAYTEST skipped — Solo mode."; PROCEED routes to `/design-review`,
|
||||
`/gate-check`, `/art-bible`, `/map-systems`, `/design-system`, `/review-all-gdds`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] "May I write to `prototypes/grapple-hook/`?" is asked before any files are created
|
||||
- [ ] Implementation is isolated to `prototypes/` (not `src/`)
|
||||
- [ ] `findings.md` is created with at minimum: tested/worked/didn't-work/recommendation
|
||||
- [ ] Verdict is PROTOTYPE COMPLETE
|
||||
- [ ] The hypothesis is falsifiable, with a measurable signal
|
||||
- [ ] Engine is recommended in the path prompt before the user chooses
|
||||
- [ ] The directory ask names `prototypes/grapple-hook-concept/` and comes before any prototype file is written
|
||||
- [ ] Every prototype script carries the `PROTOTYPE - NOT FOR PRODUCTION` header as a `#` comment, never `//`; no file is written under the code root
|
||||
- [ ] REPORT.md is written only after its ask, that ask also names `prototypes/index.md`, and the index gains a row with verdict PROCEED
|
||||
- [ ] PROCEED next steps include `/map-systems` and `/design-system`
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Prototype Already Exists — Offers Extend, Replace, or Archive
|
||||
### Case 2: PIVOT — Carry-Forward Note Written
|
||||
|
||||
**Fixture:**
|
||||
- `prototypes/grapple-hook/` already exists from a previous prototype session
|
||||
- It contains a basic implementation and a findings.md
|
||||
- A paper prototype of `trade-routes` has just been debriefed
|
||||
- User answers PIVOT: the route planning was engaging, the upkeep maths was not
|
||||
- Review mode: `solo`
|
||||
|
||||
**Input:** `/prototype grapple-hook`
|
||||
**Input:** `/prototype trade-routes --path paper`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill detects existing `prototypes/grapple-hook/` directory
|
||||
2. Skill reports: "Prototype already exists for grapple-hook"
|
||||
3. Skill presents 3 options:
|
||||
- Extend: add new features to the existing prototype
|
||||
- Replace: start fresh (asks "May I replace `prototypes/grapple-hook/`?")
|
||||
- Archive: move to `prototypes/archive/grapple-hook/` and start fresh
|
||||
4. User selects; skill proceeds accordingly
|
||||
1. Paper path writes `prototypes/trade-routes-concept/rules.md` and
|
||||
`play-log.md` after the directory ask
|
||||
2. REPORT.md is written after its ask with verdict PIVOT
|
||||
3. Skill asks, one at a time: "What specifically worked in this prototype that
|
||||
we should preserve…?" and "What is the single most important thing to change?"
|
||||
4. Skill asks "May I write this to `prototypes/trade-routes-concept/PIVOT-NOTE.md`?"
|
||||
5. The note holds the original hypothesis, what to keep, what to change and the
|
||||
revised hypothesis; next steps are `/prototype [revised-concept]` or
|
||||
`/brainstorm [hint]`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Existing prototype is detected and reported
|
||||
- [ ] Exactly 3 options are presented (extend, replace, archive)
|
||||
- [ ] Replace path includes a "May I replace" confirmation
|
||||
- [ ] Archive path moves (not deletes) the existing prototype
|
||||
- [ ] `--path paper` is honoured and produces `rules.md` and `play-log.md`
|
||||
- [ ] The two carry-forward questions are asked separately
|
||||
- [ ] "May I write" names `PIVOT-NOTE.md` before it is written
|
||||
- [ ] The note contains a revised hypothesis
|
||||
- [ ] Next steps offer `/prototype` for the revised concept or `/brainstorm`
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Prototype Validates Mechanic — Recommends Proceeding to Production
|
||||
### Case 3: KILL — Soundness Check and Graveyard Entry
|
||||
|
||||
**Fixture:**
|
||||
- Prototype implementation complete
|
||||
- Findings: grapple hook mechanic is fun and technically feasible
|
||||
- An HTML prototype of `procedural-dialogue` has been debriefed
|
||||
- User answers KILL; the debrief shows testers never understood the core
|
||||
action after 2 playtests, no fun moment was observed, and the concept only
|
||||
worked when the developer explained it
|
||||
|
||||
**Input:** `/prototype grapple-hook` (prototype session complete)
|
||||
**Input:** `/prototype procedural-dialogue --path html`
|
||||
|
||||
**Expected behavior:**
|
||||
1. After prototype is built and tested, findings are summarized
|
||||
2. Recommendation in findings.md: "Mechanic validated — recommend proceeding
|
||||
to `/design-system` for full specification"
|
||||
3. Skill handoff message explicitly suggests `/design-system grapple-hook`
|
||||
4. Verdict is PROTOTYPE COMPLETE
|
||||
1. REPORT.md is written after its ask with verdict KILL
|
||||
2. Skill runs the five-item KILL checklist; 3 items apply, so the verdict is sound
|
||||
3. Skill asks "May I append this concept to `prototypes/GRAVEYARD.md`?"
|
||||
4. The entry has Kill reason, What worked, What failed and Next time, with a
|
||||
specific kill reason (not "it was boring")
|
||||
5. Next step: `/brainstorm open` or `/brainstorm [new-hint]`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] `findings.md` contains an explicit recommendation
|
||||
- [ ] Recommendation references `/design-system` when mechanic is validated
|
||||
- [ ] Handoff message echoes the recommendation
|
||||
- [ ] Verdict is PROTOTYPE COMPLETE (not PROTOTYPE ABANDONED)
|
||||
- [ ] The KILL checklist is applied, with the 2+ rule deciding soundness
|
||||
- [ ] "May I append" names `prototypes/GRAVEYARD.md` before writing
|
||||
- [ ] The graveyard entry has all four fields and a specific kill reason
|
||||
- [ ] Next steps route to `/brainstorm`
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Prototype Reveals Mechanic is Unworkable — PROTOTYPE ABANDONED
|
||||
### Case 4: Spike Mode — No Debrief, No Gate, No PROCEED/PIVOT/KILL
|
||||
|
||||
**Fixture:**
|
||||
- Prototype implemented for "procedural-dialogue"
|
||||
- After testing: the mechanic creates incoherent dialogue trees and is
|
||||
frustrating to play
|
||||
- Project is in Production
|
||||
- Review mode: `full`
|
||||
|
||||
**Input:** `/prototype procedural-dialogue`
|
||||
**Input:** `/prototype rope-physics --spike`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Prototype is built
|
||||
2. Findings document the failure: incoherent output, player confusion, technical complexity
|
||||
3. Recommendation in findings.md: "Mechanic not viable — abandoning"
|
||||
4. `findings.md` documents the specific reasons the mechanic failed
|
||||
5. Skill suggests alternatives in the handoff (e.g., curated dialogue instead)
|
||||
6. Verdict is PROTOTYPE ABANDONED
|
||||
1. Skill skips Phases 1–9 and runs Spike Mode
|
||||
2. Asks for the spike question in plain text: "Can we [do X] using [approach Y]?"
|
||||
3. Path widget as in Phase 3; scope limited to 2–3 bullets; ~4-hour cap
|
||||
4. Asks "May I create `prototypes/rope-physics-spike-[date]/` and build the
|
||||
spike there?" before writing any spike file
|
||||
5. Asks "Did the spike answer the question? YES or NO, and why in one sentence."
|
||||
6. Asks once, naming both files, "May I write
|
||||
`prototypes/rope-physics-spike-[date]/SPIKE-NOTE.md` and clear the spike from
|
||||
`production/session-state/active.md`?"; then writes the note (question, result
|
||||
YES / NO / PARTIAL, what to do next) and updates `active.md`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Verdict is PROTOTYPE ABANDONED (not PROTOTYPE COMPLETE)
|
||||
- [ ] `findings.md` documents specific failure reasons (not vague)
|
||||
- [ ] Alternative approaches are suggested in the handoff
|
||||
- [ ] Prototype files are retained (not deleted) for reference
|
||||
- [ ] The intent widget and hypothesis/debrief phases are skipped
|
||||
- [ ] The spike folder is created only after its ask
|
||||
- [ ] `SPIKE-NOTE.md` and `active.md` are written only after an ask that names both
|
||||
- [ ] Output is `SPIKE-NOTE.md` in a `-spike-[date]` folder, not a REPORT.md
|
||||
- [ ] No CD-PLAYTEST spawn or skip note, even though review mode is `full`
|
||||
- [ ] No PROCEED / PIVOT / KILL verdict is issued
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; prototypes are validation artifacts
|
||||
### Case 5: Director Gate — CD-PLAYTEST by Review Mode and Pillars
|
||||
|
||||
**Fixture:**
|
||||
- Mechanic concept provided
|
||||
- A concept prototype has been built and REPORT.md written with a PROCEED
|
||||
recommendation
|
||||
- Run A: `--review full`, `design/gdd/game-concept.md` exists with pillars;
|
||||
`creative-director` returns REJECT — the core fantasy is not present; the user,
|
||||
asked PIVOT or KILL, picks PIVOT
|
||||
- Run B: `--review full`, neither `game-concept.md` nor `design/game-brief.md` exists
|
||||
- Run C: `--review lean`
|
||||
|
||||
**Input:** `/prototype wall-jump`
|
||||
**Input:** `/prototype [concept] --review [mode]`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill creates and documents the prototype
|
||||
2. No director agents are spawned
|
||||
3. No gate IDs appear in output
|
||||
1. Run A: `creative-director` is spawned for CD-PLAYTEST with the REPORT.md
|
||||
content, hypothesis and pillars. Its REJECT means the PROCEED recommendation
|
||||
cannot stand: the skill asks the user to choose PIVOT or KILL (it does not pick
|
||||
for them), the user picks PIVOT, and after an ask naming REPORT.md and
|
||||
`prototypes/index.md` the report records the director's verdict and reason and
|
||||
the final PIVOT recommendation; Phase 9 then runs the PIVOT route
|
||||
2. Run B: no spawn; note "CD-PLAYTEST skipped — game pillars not yet defined at
|
||||
concept prototype stage."
|
||||
3. Run C: no spawn; note "CD-PLAYTEST skipped — Lean mode."
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Verdict is PROTOTYPE COMPLETE or PROTOTYPE ABANDONED — no gate verdict
|
||||
- [ ] Run A spawns `creative-director` for CD-PLAYTEST after REPORT.md is written
|
||||
- [ ] Run A: the director's verdict is read as one of APPROVE / CONCERNS / REJECT, not as a PROCEED / PIVOT / KILL of its own
|
||||
- [ ] Run A: after the REJECT, PROCEED is not kept; the user is asked to choose PIVOT or KILL
|
||||
- [ ] Run A's final recommendation is the user's PIVOT, and REPORT.md records it with the director's reason
|
||||
- [ ] Run B skips with the "game pillars not yet defined" note instead of spawning without pillars
|
||||
- [ ] Run C skips with the Lean-mode note
|
||||
- [ ] No other director gate is spawned
|
||||
|
||||
---
|
||||
|
||||
### Case 6: NOT ASSESSED (Unplayed) and the `workflow: minimal` PROCEED Route
|
||||
|
||||
**Fixture:**
|
||||
- Run A: the prototype was built this session but nobody has played it yet
|
||||
- Run B: a PROCEED recommendation, `project.yaml` has `modes.workflow: minimal`
|
||||
|
||||
**Input:** `/prototype [concept]`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Run A: with no fun evidence yet, PROCEED/PIVOT/KILL are all unreachable; the
|
||||
skill reports NOT ASSESSED and stops rather than guessing, naming "play it,
|
||||
then run `/prototype [same concept]` again" as the next step
|
||||
2. Run B: Phase 9's PROCEED route offers `/create-stories` (from the brief) then
|
||||
`/dev-story` on the first story, not the GDD/`/map-systems`/`/art-bible` path
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Run A's verdict is NOT ASSESSED, not a guessed PROCEED/PIVOT/KILL
|
||||
- [ ] Run A's report and `prototypes/index.md` row both read NOT ASSESSED until a playtest happens
|
||||
- [ ] Run B's recommended path is the `workflow: minimal` route (`/create-stories`, `/dev-story`), not the `standard`/`full` GDD pipeline
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Asks "May I write to `prototypes/[name]/`?" before creating any files
|
||||
- [ ] Creates all files under `prototypes/` (not `src/`)
|
||||
- [ ] Produces `findings.md` with tested/worked/didn't-work/recommendation
|
||||
- [ ] Notes that production coding standards are intentionally relaxed
|
||||
- [ ] Offers extend/replace/archive when prototype already exists
|
||||
- [ ] Verdict is PROTOTYPE COMPLETE or PROTOTYPE ABANDONED
|
||||
- [ ] Confirms intent with `AskUserQuestion` before building (unless `--spike`)
|
||||
- [ ] Forms a falsifiable hypothesis and stops if the concept is too vague to form one
|
||||
- [ ] Presents the plan and gets confirmation before building
|
||||
- [ ] Asks before creating `prototypes/[concept-name]-concept/` and before writing REPORT.md (with its `prototypes/index.md` row), PIVOT-NOTE.md or GRAVEYARD.md
|
||||
- [ ] In spike mode, asks before creating the spike folder and before writing `SPIKE-NOTE.md` and `active.md`
|
||||
- [ ] Keeps prototype files under `prototypes/`, each with the `PROTOTYPE - NOT FOR PRODUCTION` header in its own comment syntax
|
||||
- [ ] Applies the review mode before CD-PLAYTEST and prints the skip note when skipped
|
||||
- [ ] Ends with the per-verdict next steps
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- Prototype implementation quality (code style) is intentionally not tested —
|
||||
prototypes are throwaway artifacts and quality standards do not apply.
|
||||
- The archiving mechanism is mentioned in Case 2 but the archive format is
|
||||
not assertion-tested in detail.
|
||||
- Engine-specific prototype scaffolding (GDScript scenes vs. C# MonoBehaviour)
|
||||
follows the same flow with engine-appropriate file types.
|
||||
- "Skip — concept already proven" (records the evidence, suggests `/map-systems`
|
||||
or `/design-system`, and stops) is not tested.
|
||||
- A concept too vague to form a hypothesis (skill stops and asks the user to
|
||||
narrow it) is covered only by protocol compliance.
|
||||
- Reading a prior `PIVOT-NOTE.md` on the next run to seed the revised hypothesis
|
||||
is not tested.
|
||||
- The engine-path multi-turn fix loop and the 2-hour sunk-cost rule need a live
|
||||
run to verify.
|
||||
- CD-PLAYTEST APPROVE (the recommendation stands), CONCERNS (the user decides:
|
||||
Revise / Accept / Discuss), REJECT on a PIVOT or KILL recommendation (it
|
||||
stands) and NOT ASSESSED (never an approval) are not separately tested.
|
||||
|
||||
@@ -2,174 +2,201 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/qa-plan` generates a structured QA test plan for a feature or sprint milestone.
|
||||
It reads story files for the specified sprint, extracts acceptance criteria from
|
||||
each story, cross-references test standards from `coding-standards.md` to assign
|
||||
the appropriate test type (unit, integration, visual, UI, or config/data), and
|
||||
produces a prioritized QA plan document.
|
||||
`/qa-plan [sprint | feature: system-name | story: path]` generates a QA plan
|
||||
before implementation. `sprint` reads the most recent file in
|
||||
`production/sprints/` (or `production/sprint-status.yaml` when present) for its
|
||||
story list; `feature:` globs `production/epics/*/story-*.md` by system name;
|
||||
`story:` loads one file; no argument asks for the scope with `AskUserQuestion`.
|
||||
A referenced story file that does not exist is noted as MISSING and the plan
|
||||
continues; a scope with zero stories stops with `NOT ASSESSED — no stories in
|
||||
scope` and a route (e.g. "No sprint plan found. Run `/sprint-plan new`.").
|
||||
|
||||
The skill asks "May I write to `production/qa/qa-plan-sprint-NNN.md`?" before
|
||||
persisting the output. If an existing test plan for the same sprint is found, the
|
||||
skill offers to update rather than replace. The verdict is COMPLETE when the plan
|
||||
is written. No director gates are used — gate-level story readiness is handled by
|
||||
`/story-readiness`.
|
||||
Story fields are collected with targeted greps. A declared `Type:` is accepted
|
||||
as-is; a missing one is inferred from the acceptance criteria and flagged as a
|
||||
gap. A story with no `## Acceptance Criteria` is full-read and reported as a QA
|
||||
finding, never skipped. After a classification summary table, the plan
|
||||
(Test Summary, Automated Tests Required, Manual QA Checklist, Smoke Test Scope,
|
||||
Playtest Requirements, Definition of Done) is shown, then one `AskUserQuestion`
|
||||
(multiSelect) asks whether to write `production/qa/qa-plan-[sprint-slug]-[date].md`
|
||||
and whether to back-fill `## QA Test Cases` in the story files. `qa.level`
|
||||
changes the plan's depth, but no level drops the Visual/Feel and UI screenshot
|
||||
rows. No director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keyword: COMPLETE
|
||||
- [ ] Contains "May I write" collaborative protocol language before writing the plan
|
||||
- [ ] Has a next-step handoff (e.g., `/smoke-check` or `/story-readiness`)
|
||||
- [ ] Contains the verdict keyword `NOT ASSESSED` for an empty scope
|
||||
- [ ] Asks for approval before writing: an `AskUserQuestion` option naming `production/qa/qa-plan-[sprint-slug]-[date].md`, and the rule "Never write the plan without asking"
|
||||
- [ ] Has a next-step handoff (`/smoke-check sprint` after implementation, `/story-done` checks for the test files)
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
None. `/qa-plan` is a planning utility. Story readiness gates are separate.
|
||||
None. `/qa-plan` is a planning utility. It spawns no agents (`Agent` is not in
|
||||
its allowed tools) and no director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — Sprint with 4 stories generates full test plan
|
||||
### Case 1: Happy Path — Sprint With Four Typed Stories
|
||||
|
||||
**Fixture:**
|
||||
- `production/sprints/sprint-003.md` lists 4 stories with defined acceptance criteria
|
||||
- Stories span types: 1 logic (formula), 1 integration, 1 visual, 1 UI
|
||||
- `coding-standards.md` is present with test evidence table
|
||||
- `production/sprints/sprint-003.md` is the most recent sprint and references 4
|
||||
story files, each with acceptance criteria and a declared Type: Logic,
|
||||
Integration, Visual/Feel, UI
|
||||
- The stories reference GDDs in `design/gdd/`; `engine.name: Godot`
|
||||
- `qa.level: standard`; `modes.automation: collaborative`
|
||||
|
||||
**Input:** `/qa-plan sprint-003`
|
||||
**Input:** `/qa-plan sprint`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads sprint-003.md and identifies 4 stories
|
||||
2. Skill reads each story's acceptance criteria
|
||||
3. Skill assigns test types per coding-standards.md table:
|
||||
- Logic story → Unit test (BLOCKING)
|
||||
- Integration story → Integration test (BLOCKING)
|
||||
- Visual story → Retained screenshot + lead sign-off (BLOCKING)
|
||||
- UI story → Retained screenshot of each screen (BLOCKING)
|
||||
4. Skill drafts QA plan with story-by-story test type breakdown
|
||||
5. Skill asks "May I write to `production/qa/qa-plan-sprint-003.md`?"
|
||||
6. File is written on approval; verdict is COMPLETE
|
||||
1. Skill reads the most recent sprint file and reports "Building QA plan for 4
|
||||
stories in [scope]."
|
||||
2. Story fields are collected with section greps; each declared Type is accepted
|
||||
without re-classification
|
||||
3. A classification summary table is shown before the plan is generated
|
||||
4. The plan's Test Summary maps Logic → unit test under `tests/unit/[system]/`,
|
||||
Integration → integration test under `tests/integration/[system]/`,
|
||||
Visual/Feel → screenshot + lead sign-off, UI → retained screenshot of each
|
||||
screen touched
|
||||
5. The plan is shown, then one multiSelect `AskUserQuestion` offers "Write QA
|
||||
plan to production/qa/qa-plan-[sprint-slug]-[date].md" and the story
|
||||
back-fill option
|
||||
6. After writing, next steps name `/smoke-check sprint` (after implementation)
|
||||
and `/story-done`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All 4 stories are included in the plan
|
||||
- [ ] Test type is assigned per coding-standards.md (not guessed)
|
||||
- [ ] Gate level (BLOCKING vs ADVISORY) is noted for each story
|
||||
- [ ] "May I write" is asked with the correct file path
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] All 4 stories appear in the plan, each with its declared Type unchanged
|
||||
- [ ] The classification summary table appears before the plan
|
||||
- [ ] Logic and Integration rows name test paths under `tests/unit/` and `tests/integration/`
|
||||
- [ ] The UI row's manual verification is the retained screenshot of each screen touched, not a step-through
|
||||
- [ ] Nothing is written before the multiSelect approval, and the written path is `production/qa/qa-plan-[sprint-slug]-[date].md`
|
||||
- [ ] Next steps name `/smoke-check sprint` and `/story-done`
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Story With No Acceptance Criteria — Flagged as UNTESTABLE
|
||||
### Case 2: Story Without Acceptance Criteria and a Missing Story File
|
||||
|
||||
**Fixture:**
|
||||
- `production/sprints/sprint-004.md` lists 3 stories; one story has empty
|
||||
acceptance criteria section
|
||||
- The most recent sprint file references 4 story paths
|
||||
- One story has no `## Acceptance Criteria` section
|
||||
- One referenced story path does not exist on disk
|
||||
- The other 2 stories are complete
|
||||
|
||||
**Input:** `/qa-plan sprint-004`
|
||||
**Input:** `/qa-plan sprint`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads all 3 stories
|
||||
2. Skill detects the story with no AC
|
||||
3. Story is flagged as `UNTESTABLE — Acceptance Criteria required` in the plan
|
||||
4. Other 2 stories receive normal test type assignments
|
||||
5. Plan is written with the UNTESTABLE story flagged; verdict is COMPLETE
|
||||
1. The missing path is noted as MISSING and the plan continues
|
||||
2. The story whose AC grep matched nothing is full-read, and the plan says it
|
||||
has no testable criteria — a QA finding, not a skipped story
|
||||
3. The 2 complete stories are classified and planned normally
|
||||
4. Approval is asked before writing
|
||||
|
||||
**Assertions:**
|
||||
- [ ] UNTESTABLE label appears for the story with no AC
|
||||
- [ ] Plan is not blocked — the other stories are still planned
|
||||
- [ ] Output suggests adding AC to the flagged story (next step)
|
||||
- [ ] Verdict is COMPLETE (the plan is still generated)
|
||||
- [ ] The missing file is reported as MISSING without failing the whole plan
|
||||
- [ ] The story with no acceptance criteria is full-read and reported as a QA finding
|
||||
- [ ] That story is not silently dropped from the plan
|
||||
- [ ] The two complete stories receive normal test assignments
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Existing Test Plan Found — Offers update rather than replace
|
||||
### Case 3: Mode Variant — qa.level minimal
|
||||
|
||||
**Fixture:**
|
||||
- `production/qa/qa-plan-sprint-003.md` already exists from a previous run
|
||||
- Sprint-003 has 2 new stories added since the last plan
|
||||
- Same 4-story sprint as Case 1
|
||||
- `qa.level: minimal`
|
||||
|
||||
**Input:** `/qa-plan sprint-003`
|
||||
**Input:** `/qa-plan sprint`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads sprint-003.md and detects 2 stories not in the existing plan
|
||||
2. Skill reports: "Existing QA plan found for sprint-003 — offering to update"
|
||||
3. Skill presents the 2 new stories and their proposed test assignments
|
||||
4. Skill asks "May I update `production/qa/qa-plan-sprint-003.md`?" (not overwrite)
|
||||
5. Updated plan is written on approval
|
||||
1. Skill produces only a minimal smoke plan
|
||||
2. The "Automated Tests Required" section is omitted and the Test Summary's
|
||||
"Automated Test Required" column is blanked
|
||||
3. The Definition of Done drops the test-file and smoke rows only; it still
|
||||
requires a retained screenshot of each screen touched for the Visual/Feel
|
||||
and UI stories, and a signed-off evidence doc for the Visual/Feel story —
|
||||
`qa.level` waives tests, never the screenshot
|
||||
4. Approval is asked before writing
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Skill detects the existing plan file
|
||||
- [ ] "update" language is used (not "overwrite")
|
||||
- [ ] Only new stories are proposed for addition — existing entries preserved
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] No "Automated Tests Required" section appears
|
||||
- [ ] The Test Summary's automated-test column is blank
|
||||
- [ ] The Definition of Done has no test-file or smoke-check rows
|
||||
- [ ] The Definition of Done still requires the retained screenshot for the Visual/Feel and UI stories, and the signed-off evidence doc for the Visual/Feel story
|
||||
- [ ] Approval is still asked before the plan is written
|
||||
|
||||
---
|
||||
|
||||
### Case 4: No Stories Found for Sprint — Error with guidance
|
||||
### Case 4: No Sprint Plan — NOT ASSESSED
|
||||
|
||||
**Fixture:**
|
||||
- `production/sprints/sprint-007.md` does not exist
|
||||
- No other sprint file matching sprint-007
|
||||
- `production/sprints/` contains no files
|
||||
- No `production/sprint-status.yaml`
|
||||
- `modes.rigor: standard` (so sprints are expected)
|
||||
|
||||
**Input:** `/qa-plan sprint-007`
|
||||
**Input:** `/qa-plan sprint`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill attempts to read sprint-007.md — file not found
|
||||
2. Skill outputs: "No sprint file found for sprint-007"
|
||||
3. Skill suggests running `/sprint-plan` to create the sprint first
|
||||
4. No plan is written; no "May I write" is asked
|
||||
1. The resolved scope contains zero stories
|
||||
2. Skill stops before Phase 2 and reports `NOT ASSESSED — no stories in scope`,
|
||||
naming the scope searched and the empty path
|
||||
3. Skill routes: "No sprint plan found. Run `/sprint-plan new`."
|
||||
4. No plan is generated and no write approval is asked
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Error message names the missing sprint file
|
||||
- [ ] `/sprint-plan` is suggested as the remediation step
|
||||
- [ ] No write tool is called
|
||||
- [ ] Verdict is not COMPLETE (error state)
|
||||
- [ ] Verdict is `NOT ASSESSED — no stories in scope`
|
||||
- [ ] The empty path `production/sprints/` is named
|
||||
- [ ] `/sprint-plan new` is suggested
|
||||
- [ ] No plan document with empty tables is produced, and nothing is written
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; QA planning is a utility
|
||||
### Case 5: Director Gate Check — None
|
||||
|
||||
**Fixture:**
|
||||
- Sprint with valid stories and AC
|
||||
- A sprint with typed stories and acceptance criteria
|
||||
- Any review mode
|
||||
|
||||
**Input:** `/qa-plan sprint-003`
|
||||
**Input:** `/qa-plan sprint`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill generates and writes QA plan
|
||||
2. No director agents are spawned
|
||||
3. No gate IDs appear in output
|
||||
1. Skill classifies stories and generates the plan
|
||||
2. No director or other agent is spawned; no gate IDs appear
|
||||
3. The only interactive step after scope resolution is the single write approval
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Skill reaches COMPLETE without any gate check
|
||||
- [ ] No director gate is invoked and no gate skip message appears
|
||||
- [ ] No subagent is spawned
|
||||
- [ ] Phases 2–4 run without asking the user anything
|
||||
- [ ] Exactly one approval prompt precedes the write
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Reads coding-standards.md test evidence table before assigning test types
|
||||
- [ ] Assigns BLOCKING or ADVISORY gate level per story type
|
||||
- [ ] Flags stories with no AC as UNTESTABLE (does not silently skip them)
|
||||
- [ ] Detects existing plan and offers update path
|
||||
- [ ] Asks "May I write" before creating or updating the plan file
|
||||
- [ ] Verdict is COMPLETE when plan is written
|
||||
- [ ] Stops with `NOT ASSESSED — no stories in scope` when the scope is empty, and routes to the producing skill
|
||||
- [ ] Notes missing story files as MISSING and continues
|
||||
- [ ] Accepts declared `Type:` values; infers and flags a missing Type
|
||||
- [ ] Never skips a story because its acceptance-criteria section is absent
|
||||
- [ ] Shows the classification table and the plan before asking to write
|
||||
- [ ] Writes only after the `AskUserQuestion` approval; back-fills story files only when that option is selected
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- The case where `coding-standards.md` is missing (skill cannot assign test types)
|
||||
is not fixture-tested; behavior would follow the BLOCKED pattern with a note
|
||||
to restore the standards file.
|
||||
- Multi-sprint planning (spanning 2 sprints) is not tested; the skill is designed
|
||||
for one sprint at a time.
|
||||
- Config/data story type (balance tuning → smoke check) follows the same
|
||||
assignment pattern as other types in Case 1 and is not separately tested.
|
||||
- `feature: [system-name]` and `story: [path]` scopes, and the no-argument
|
||||
scope question, are not separately tested.
|
||||
- A story with no `Type:` field (inferred from acceptance criteria and flagged
|
||||
as a gap) is covered only by protocol compliance.
|
||||
- The back-fill option (editing `## QA Test Cases` in each story file) is not
|
||||
tested in detail.
|
||||
- At `rigor: minimal`, the empty-sprint route points to `/qa-plan feature:` or
|
||||
`/qa-plan story:` instead of `/sprint-plan new`; not separately tested.
|
||||
|
||||
@@ -2,171 +2,209 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/regression-suite` maps test coverage to GDD requirements: it reads the
|
||||
acceptance criteria from story files in the current sprint (or a specified epic),
|
||||
then scans `tests/` for corresponding test files and checks whether each AC has
|
||||
a matching assertion. It produces a coverage report identifying which ACs are
|
||||
fully covered, partially covered, or untested, and which test files have no
|
||||
matching AC (orphan tests).
|
||||
`/regression-suite [update | audit | report]` maintains `tests/regression-suite.md`,
|
||||
a curated manifest of existing tests that guard critical paths and fixed bugs.
|
||||
`audit` maps each GDD acceptance criterion (scoped by workflow tier) to tests in
|
||||
`tests/unit/[system]/` and `tests/integration/[system]/` as COVERED / PARTIAL /
|
||||
MISSING / EXEMPT, raising MISSING formula or state-machine criteria to HIGH
|
||||
PRIORITY. `update` checks bugs in `production/qa/bugs/` with `Status: Closed` or
|
||||
`Status: Fixed` for a regression test (HAS / MISSING REGRESSION TEST) and appends
|
||||
new manifest entries without removing any. `report` is read-only. All modes
|
||||
check for coverage drift.
|
||||
|
||||
The skill may write a coverage report to `production/qa/` after a "May I write"
|
||||
ask. No director gates apply. Verdicts: FULL COVERAGE (all ACs have tests),
|
||||
GAPS FOUND (some ACs are untested), or CRITICAL GAPS (a critical-priority AC
|
||||
has no test).
|
||||
At `qa.level: minimal` the skill stops before any scan with "Regression suite
|
||||
not generated at qa.level minimal". With zero critical paths or zero test files
|
||||
it reports `Coverage: NOT ASSESSED — [no GDDs found | no test files found]`
|
||||
instead of a percentage. Before writing it asks "May I write/update
|
||||
`tests/regression-suite.md` with the current regression suite manifest?".
|
||||
Verdict, first match: **NOT ASSESSED** at the `qa.level: minimal` stop or when
|
||||
coverage was not computed (never COMPLETE over a missing figure, even after a
|
||||
write), **BLOCKED** if the user declines the write, **COMPLETE** after the
|
||||
write — or, in read-only `report` mode, once the report is shown. `qa.level`
|
||||
also sets timing — the suite is due at Polish-stage entry at `standard` and at
|
||||
Production-stage entry at `full` — so each fixture pins `project.stage`. No
|
||||
director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keywords: FULL COVERAGE, GAPS FOUND, CRITICAL GAPS
|
||||
- [ ] Contains "May I write" language (skill may write coverage report)
|
||||
- [ ] Has a next-step handoff (e.g., `/test-setup` if framework missing, `/qa-plan` if plan missing)
|
||||
- [ ] Contains verdict keywords: COMPLETE, BLOCKED, NOT ASSESSED
|
||||
- [ ] Contains "May I write" language before writing `tests/regression-suite.md`
|
||||
- [ ] Has a next-step handoff (`/test-helpers` for HIGH gaps, `/regression-suite audit` on drift, `/test-flakiness` for quarantined tests)
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
None. `/regression-suite` is a QA analysis utility. No director gates apply.
|
||||
None. `/regression-suite` is a QA utility. It spawns no agents (`Agent` is not
|
||||
in its allowed tools) and no director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Full Coverage — All ACs in sprint have corresponding tests
|
||||
### Case 1: Audit — Criteria Mapped, Formula Gap Raised to HIGH
|
||||
|
||||
**Fixture:**
|
||||
- `production/sprints/sprint-004.md` lists 3 stories with 2 ACs each (6 total)
|
||||
- `tests/unit/` and `tests/integration/` contain test files that match all 6 ACs
|
||||
(by system name and scenario description)
|
||||
- `qa.level: standard`; workflow tier `full`; `project.stage: Polish`
|
||||
- `design/gdd/systems-index.md` lists `combat`; `design/gdd/combat.md` has 4
|
||||
acceptance criteria: a damage formula, a hit-stun state transition, a
|
||||
knockback rule and a HUD damage-number display
|
||||
- `tests/unit/combat/` has tests for hit-stun (all cases) and knockback (happy
|
||||
path only); nothing tests the damage formula
|
||||
- `tests/regression-suite.md` exists
|
||||
|
||||
**Input:** `/regression-suite sprint-004`
|
||||
**Input:** `/regression-suite audit`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads all 6 ACs from sprint-004 stories
|
||||
2. Skill scans test files and matches each AC to at least one test assertion
|
||||
3. All 6 ACs have coverage
|
||||
4. Skill produces coverage report: "6/6 ACs covered"
|
||||
5. Skill asks "May I write to `production/qa/regression-sprint-004.md`?"
|
||||
6. File is written on approval; verdict is FULL COVERAGE
|
||||
1. Skill reads the existing manifest, globs the test inventory and reads the
|
||||
combat GDD's critical paths
|
||||
2. Coverage: hit-stun COVERED, knockback PARTIAL, damage formula MISSING, HUD
|
||||
display EXEMPT (UI, not automatable)
|
||||
3. The missing damage formula is raised to a HIGH PRIORITY gap with a suggested
|
||||
test file path
|
||||
4. The report shows the per-system table and a non-exempt coverage rate
|
||||
5. Skill asks "May I write/update `tests/regression-suite.md` with the current
|
||||
regression suite manifest?"; on approval audit rewrites the full manifest
|
||||
6. Verdict is COMPLETE; for the HIGH gap it suggests `/test-helpers`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All 6 ACs appear in the coverage report
|
||||
- [ ] Each AC is marked as covered with the matching test file referenced
|
||||
- [ ] Verdict is FULL COVERAGE
|
||||
- [ ] "May I write" is asked before writing the report
|
||||
- [ ] Each criterion gets one of COVERED / PARTIAL / MISSING / EXEMPT, matching the fixture
|
||||
- [ ] The HUD criterion is EXEMPT and excluded from the coverage rate
|
||||
- [ ] The damage-formula gap is HIGH PRIORITY with a `tests/unit/combat/` path
|
||||
- [ ] The manifest is written only after the "May I write/update" ask
|
||||
- [ ] Verdict is COMPLETE and `/test-helpers` is suggested
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Gaps Found — 3 ACs have no tests
|
||||
### Case 2: Update — Fixed Bug Without a Regression Test
|
||||
|
||||
**Fixture:**
|
||||
- Sprint has 5 stories with 8 total ACs
|
||||
- Tests exist for 5 of the 8 ACs; 3 ACs have no corresponding test file or assertion
|
||||
- `qa.level: standard`; `project.stage: Polish`
|
||||
- `production/sprints/sprint-007.md` is the current sprint plan; its one
|
||||
`Status: Complete` story is `production/epics/inventory/story-003.md`
|
||||
- `production/qa/bugs/BUG-0012.md` has `Status: Closed`, system `inventory`
|
||||
- No test under `tests/unit/inventory/` or `tests/integration/inventory/`
|
||||
references BUG-0012 or its failure scenario
|
||||
- `tests/regression-suite.md` has 6 registered tests
|
||||
|
||||
**Input:** `/regression-suite`
|
||||
**Input:** `/regression-suite update`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads all 8 ACs
|
||||
2. Skill scans tests — 5 matched, 3 unmatched
|
||||
3. Coverage report lists the 3 untested ACs by story and AC text
|
||||
4. Skill asks "May I write to `production/qa/regression-[sprint]-[date].md`?"
|
||||
5. Report is written; verdict is GAPS FOUND
|
||||
1. Skill reads the sprint plan's completed stories and the closed bugs
|
||||
2. BUG-0012 is MISSING REGRESSION TEST, with suggested path
|
||||
`tests/unit/inventory/[bug-slug]_regression_test.[ext]` and the note
|
||||
"Without this test, this bug can silently return in a future sprint."
|
||||
3. Report shows the Bug Regression Coverage table with BUG-0012 marked NO
|
||||
4. After the "May I write/update" ask, new entries are appended with targeted
|
||||
`Edit` insertions; the 6 existing entries remain
|
||||
5. Skill says the next sprint should include a story to write the missing test;
|
||||
verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The 3 untested ACs are listed by name in the report
|
||||
- [ ] Matched ACs are also shown (not only the gaps)
|
||||
- [ ] Verdict is GAPS FOUND (not FULL COVERAGE)
|
||||
- [ ] Report is written after "May I write" approval
|
||||
- [ ] BUG-0012 is reported as MISSING REGRESSION TEST
|
||||
- [ ] The suggested path follows `tests/unit/[system]/[bug-slug]_regression_test.[ext]`
|
||||
- [ ] No existing manifest entry is removed
|
||||
- [ ] The next-sprint story recommendation appears because bug regression gaps > 0
|
||||
- [ ] Verdict is COMPLETE after approval
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Critical AC Untested — CRITICAL GAPS verdict, flagged prominently
|
||||
### Case 3: qa.level minimal — Suite Not Generated
|
||||
|
||||
**Fixture:**
|
||||
- Sprint has 4 stories; one story is Priority: Critical with 2 ACs
|
||||
- One of the critical-priority ACs has no test
|
||||
- `qa.level: minimal`
|
||||
- GDDs and test files exist
|
||||
|
||||
**Input:** `/regression-suite`
|
||||
**Input:** `/regression-suite audit`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads all stories and ACs, noting which stories are critical priority
|
||||
2. Skill scans tests — the critical AC has no match
|
||||
3. Report prominently flags: "CRITICAL GAP: [AC text] — no test found (Critical priority story)"
|
||||
4. Skill recommends blocking story completion until test is added
|
||||
5. Verdict is CRITICAL GAPS
|
||||
1. The early `qa.level` guard fires before any scan
|
||||
2. Skill reports "Regression suite not generated at qa.level minimal" and stops
|
||||
3. It does not fall into the workflow-tier `minimal` branch (smoke-report scope)
|
||||
4. Verdict is NOT ASSESSED — the first rule of the verdict list
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Verdict is CRITICAL GAPS (not GAPS FOUND)
|
||||
- [ ] Critical priority AC is flagged more prominently than normal gaps
|
||||
- [ ] Recommendation to block story completion is included
|
||||
- [ ] Non-critical gaps (if any) are also listed
|
||||
- [ ] The "not generated at qa.level minimal" message is shown
|
||||
- [ ] No GDD, test or bug scan runs
|
||||
- [ ] The smoke-report branch is not entered on account of `qa.level`
|
||||
- [ ] Nothing is written and no "May I write" ask appears
|
||||
- [ ] Verdict is NOT ASSESSED — not COMPLETE, and not a stop without a verdict
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Orphan Tests — Test file has no matching AC
|
||||
### Case 4: Edge Case — No Test Files, Coverage NOT ASSESSED
|
||||
|
||||
**Fixture:**
|
||||
- `tests/unit/save_system_test.gd` exists with assertions for scenarios
|
||||
not present in any current story's AC list
|
||||
- Current sprint stories do not reference save system
|
||||
- `qa.level: standard`; workflow tier `full`; `project.stage: Polish`
|
||||
- `design/gdd/` holds GDDs with acceptance criteria
|
||||
- `tests/unit/`, `tests/integration/` and `tests/regression/` contain no files
|
||||
- The user approves the manifest write
|
||||
|
||||
**Input:** `/regression-suite`
|
||||
**Input:** `/regression-suite audit`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill scans tests and cross-references ACs
|
||||
2. `save_system_test.gd` assertions do not match any current AC
|
||||
3. Test file is flagged as ORPHAN TEST in the coverage report
|
||||
4. Report notes: "Orphan tests may belong to a past or future sprint, or AC was renamed"
|
||||
5. Verdict is FULL COVERAGE or GAPS FOUND depending on overall AC coverage
|
||||
(orphan tests do not affect verdict, they are advisory)
|
||||
1. The test globs return zero files
|
||||
2. Skill does not emit a coverage percentage
|
||||
3. It reports `Coverage: NOT ASSESSED — no test files found`, naming the empty
|
||||
side and `/test-setup` as the skill that produces the test scaffold
|
||||
4. The manifest is written only after the "May I write/update" ask
|
||||
5. Verdict is NOT ASSESSED — the first match, even though the write happened
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Orphan test is flagged in the report
|
||||
- [ ] Orphan flag includes the filename and suggestion (past sprint / renamed AC)
|
||||
- [ ] Orphan tests do not cause a GAPS FOUND verdict on their own
|
||||
- [ ] Overall verdict reflects AC coverage only
|
||||
- [ ] No `0%` (or any percentage) coverage figure is reported
|
||||
- [ ] Coverage reads `NOT ASSESSED — no test files found`
|
||||
- [ ] `/test-setup` is named as the producing skill
|
||||
- [ ] Any manifest write still requires the "May I write/update" ask
|
||||
- [ ] Verdict is NOT ASSESSED — never COMPLETE over a coverage figure that was not computed
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; regression-suite is a QA utility
|
||||
### Case 5: Director Gate Check — None; Report Mode Is Read-Only
|
||||
|
||||
**Fixture:**
|
||||
- Sprint with stories and test files
|
||||
- `qa.level: standard`; `project.stage: Polish`
|
||||
- `tests/regression-suite.md`, tests and closed bugs exist
|
||||
- Any review mode
|
||||
|
||||
**Input:** `/regression-suite`
|
||||
**Input:** `/regression-suite report`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill produces coverage report and writes it
|
||||
2. No director agents are spawned
|
||||
3. No gate IDs appear in output
|
||||
1. Skill produces the Regression Suite Status report in conversation
|
||||
2. No director or other agent is spawned; no gate IDs appear
|
||||
3. `report` mode writes nothing and asks no write question
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Verdict is FULL COVERAGE, GAPS FOUND, or CRITICAL GAPS — no gate verdict
|
||||
- [ ] No director gate is invoked and no gate skip message appears
|
||||
- [ ] No subagent is spawned
|
||||
- [ ] The status report is shown in conversation
|
||||
- [ ] No file is written and no "May I write" ask appears
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Reads story ACs from sprint files before scanning tests
|
||||
- [ ] Matches ACs to tests by system name and scenario (not file name alone)
|
||||
- [ ] Flags critical-priority untested ACs as CRITICAL GAPS
|
||||
- [ ] Flags orphan tests (exist in tests/ but no AC matches)
|
||||
- [ ] Asks "May I write" before persisting the coverage report
|
||||
- [ ] Verdict is FULL COVERAGE, GAPS FOUND, or CRITICAL GAPS
|
||||
- [ ] Stops before scanning at `qa.level: minimal`
|
||||
- [ ] Reports `NOT ASSESSED` instead of a percentage when GDDs or tests are absent
|
||||
- [ ] Asks "May I write/update `tests/regression-suite.md`" before any write
|
||||
- [ ] `update` only appends; existing manifest entries are never removed without approval
|
||||
- [ ] `report` writes nothing
|
||||
- [ ] Verdict, first match: NOT ASSESSED at the `qa.level: minimal` stop or when coverage was not computed (even after a write); else BLOCKED if the user declines the write; else COMPLETE
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- The heuristic for matching an AC to a test (by system name + scenario keywords)
|
||||
is approximate; exact matching logic is defined in the skill body.
|
||||
- Integration test coverage is mapped the same way as unit test coverage; no
|
||||
distinction in verdicts is made between the two.
|
||||
- This skill does not run the tests — it maps AC text to test assertions. Test
|
||||
execution is handled by the CI pipeline.
|
||||
- No-argument behaviour (runs `update` when a sprint is clearly active, else
|
||||
asks A/B/C) is not tested.
|
||||
- At workflow tier `minimal`, `audit` maps coverage against the latest
|
||||
`production/qa/smoke-*.md` report instead of GDD criteria, and stops if none
|
||||
exists; not separately tested.
|
||||
- Coverage-drift detection (new systems, stale manifest date) is not tested by a
|
||||
dedicated case.
|
||||
- The skill does not run tests; it maps criteria and bugs to test files by name.
|
||||
|
||||
@@ -2,176 +2,285 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/release-checklist` generates an internal release readiness checklist covering:
|
||||
sprint story completion, open bug severity, QA sign-off status, build stability,
|
||||
and changelog readiness. It is an internal gate — not a platform/store checklist
|
||||
(that is `/launch-checklist`). When a previous release checklist exists, it shows
|
||||
a delta of resolved and newly introduced issues.
|
||||
`/release-checklist [platform: pc|console|mobile|all]` builds a pre-release
|
||||
checklist scoped to the project. It resolves `modes.rigor`, `modes.workflow`,
|
||||
`project.stage` and `platform.cert_tier`, reads `CLAUDE.md` and the current
|
||||
milestone in `production/milestones/`, and reads open bugs from
|
||||
`production/qa/bugs/` (Severity and Status lines). The bug items use the release
|
||||
gate's thresholds for the resolved `workflow` — an open S1, S2 or S3 fails at
|
||||
`full`, only an open S1 at `standard`/`minimal` (an open S2/S3 is a listed risk)
|
||||
— and with no bug files they read `NOT ASSESSED — no bug records`, never zero.
|
||||
It scans the code root for TODO / FIXME / HACK,
|
||||
always stating the denominator (`scanned [N] source files: …`), or
|
||||
`NOT ASSESSED — code root unresolved` when no root resolves, or
|
||||
`NOT ASSESSED — no source files found to scan` when the root is absent or empty.
|
||||
Missing test output is reported as
|
||||
`Test results: NOT ASSESSED — no test output or CI logs found`.
|
||||
|
||||
The skill writes its checklist report to `production/releases/release-checklist-[date].md`
|
||||
after a "May I write" ask. No director gates apply — `/gate-check` handles
|
||||
formal phase gate logic. Verdicts: RELEASE READY, RELEASE BLOCKED, or CONCERNS.
|
||||
The checklist has Codebase Health, Build Verification, Quality Gates and Content
|
||||
Complete sections, device blocks chosen by the argument (default `all`), then a
|
||||
certification block chosen **only** by `platform.cert_tier`: `none` → a single
|
||||
"Certification: omitted — cert_tier is 'none' …" line; `itch` → itch.io rows;
|
||||
`steam` → Steamworks rows; `console` → TRC/XR/Lotcheck rows. Unset `cert_tier`
|
||||
means asking which platforms are in scope, or
|
||||
`### Certification: NOT ASSESSED — cert tier unknown`; unset is never treated as
|
||||
`none`. It ends with Store / Distribution, Launch Readiness and
|
||||
`### Go / No-Go: [READY / NOT ASSESSED / NOT READY]` — first match NOT READY (a
|
||||
blocking item fails), then NOT ASSESSED (any section or item NOT ASSESSED), then READY —
|
||||
then asks "May I write this to
|
||||
`production/releases/release-checklist-[version].md`?". No director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keywords: RELEASE READY, RELEASE BLOCKED, CONCERNS
|
||||
- [ ] Contains "May I write" collaborative protocol language before writing the report
|
||||
- [ ] Has a next-step handoff (e.g., `/launch-checklist` for external or `/gate-check` for phase)
|
||||
- [ ] Contains verdict keywords: READY, NOT READY, NOT ASSESSED
|
||||
- [ ] Contains "May I write" language before writing the checklist
|
||||
- [ ] Has a next-step handoff (`/gate-check` for the formal verdict, `/team-release` for sign-offs)
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
None. `/release-checklist` is an internal audit utility. Formal phase advancement
|
||||
is managed by `/gate-check`.
|
||||
None. `/release-checklist` spawns no agents (`Agent` is not in its allowed
|
||||
tools). The Sign-offs Required list (QA Lead, Technical Director, Producer,
|
||||
Creative Director) is a checklist for humans, not spawned gates. Formal phase
|
||||
advancement is `/gate-check`.
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — All Sprint Stories Complete, QA Passed, RELEASE READY
|
||||
### Case 1: Happy Path — PC Build on the Steam Tier
|
||||
|
||||
**Fixture:**
|
||||
- `production/sprints/sprint-008.md` — all stories are `Status: Done`
|
||||
- No open bugs with severity HIGH or CRITICAL in `production/bugs/`
|
||||
- `production/qa/qa-plan-sprint-008.md` has QA sign-off annotation
|
||||
- Changelog entry for this version exists
|
||||
- `production/stage.txt` contains `Polish`
|
||||
- `platform.cert_tier: steam`; `project.stage: Polish`; `engine.name: Godot`
|
||||
- `src/` has 40 source files containing 3 TODO, 1 FIXME and 0 HACK comments
|
||||
- `production/milestones/` holds the current milestone
|
||||
- `production/qa/bugs/` holds `BUG-0003.md` (S1, `**Status**: Closed`) and
|
||||
`BUG-0004.md` (S4, `**Status**: Open`); `modes.rigor` is unset (`workflow: minimal`)
|
||||
- CI test output exists
|
||||
|
||||
**Input:** `/release-checklist`
|
||||
**Input:** `/release-checklist pc`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads sprint-008: all stories Done
|
||||
2. Skill reads bugs: no HIGH or CRITICAL open bugs
|
||||
3. Skill confirms QA plan has sign-off
|
||||
4. Skill confirms changelog entry exists
|
||||
5. All checks pass; skill asks "May I write to
|
||||
`production/releases/release-checklist-2026-04-06.md`?"
|
||||
6. Report written; verdict is RELEASE READY
|
||||
1. Skill reads `CLAUDE.md` and the current milestone
|
||||
2. Codebase Health states the denominator: `scanned 40 source files: 3 TODO, 1
|
||||
FIXME, 0 HACK`, listing the FIXME
|
||||
3. The PC device block is emitted; no Console or Mobile device block
|
||||
4. Certification is the Steamworks block only
|
||||
5. Store / Distribution, Launch Readiness and the Go / No-Go line follow
|
||||
6. Skill presents total items and known blockers, then asks "May I write this
|
||||
to `production/releases/release-checklist-[version].md`?"; next steps name
|
||||
`/gate-check` and `/team-release`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All 4 check categories are evaluated (stories, bugs, QA, changelog)
|
||||
- [ ] All items appear with PASS markers
|
||||
- [ ] Verdict is RELEASE READY
|
||||
- [ ] "May I write" is asked before writing
|
||||
- [ ] TODO/FIXME/HACK counts are reported with the number of files scanned
|
||||
- [ ] Only the PC device block appears
|
||||
- [ ] Certification contains Steamworks rows and no itch.io or console certification rows
|
||||
- [ ] Go / No-Go is READY or NOT READY — never NOT ASSESSED, since every section in this fixture could be assessed
|
||||
- [ ] The Quality Gates bug line is counted from `production/qa/bugs/` (0 open S1, S2 or S3); the closed S1 and the open S4 fail nothing
|
||||
- [ ] Nothing is written before the "May I write" ask naming `production/releases/release-checklist-[version].md`
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Open HIGH Severity Bugs — RELEASE BLOCKED
|
||||
### Case 2: Device Argument and Certification Tier Are Separate Axes
|
||||
|
||||
**Fixture:**
|
||||
- All sprint stories are Done
|
||||
- `production/bugs/` contains 2 open bugs with severity HIGH
|
||||
- `platform.cert_tier: itch`
|
||||
- Source files exist in the code root
|
||||
|
||||
**Input:** `/release-checklist`
|
||||
**Input:** `/release-checklist console`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads sprint — stories complete
|
||||
2. Skill reads bugs — 2 HIGH severity bugs open
|
||||
3. Skill reports: "RELEASE BLOCKED — 2 open HIGH severity bugs must be resolved"
|
||||
4. Both bug filenames are listed in the report
|
||||
5. Verdict is RELEASE BLOCKED
|
||||
1. The `console` argument selects the Console device block (suspend/resume, user
|
||||
switching, controller prompts)
|
||||
2. Certification follows `cert_tier`, not the argument: only the itch.io block
|
||||
3. No Steamworks rows and no TRC/XR/Lotcheck rows appear
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Verdict is RELEASE BLOCKED (not CONCERNS)
|
||||
- [ ] Both bug filenames are listed explicitly
|
||||
- [ ] Skill makes clear HIGH severity bugs are blocking (not advisory)
|
||||
- [ ] The Console device block is present
|
||||
- [ ] The only certification block is itch.io
|
||||
- [ ] No console certification rows (TRC/Lotcheck, first-party submission) appear
|
||||
- [ ] No Steamworks rows appear
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Changelog Not Generated — CONCERNS
|
||||
### Case 3: cert_tier none vs Unset — Different Output
|
||||
|
||||
**Fixture:**
|
||||
- All stories Done, no HIGH/CRITICAL bugs
|
||||
- No changelog entry found for the current version/sprint
|
||||
- Run A: `platform.cert_tier: none`
|
||||
- Run B: `platform.cert_tier` absent (resolved block shows it as unset) and the
|
||||
user cannot say which platforms are in scope
|
||||
|
||||
**Input:** `/release-checklist`
|
||||
**Input:** `/release-checklist` (no argument → device blocks for `all`)
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill checks all items
|
||||
2. Changelog check fails: no changelog entry found
|
||||
3. Skill reports: "CONCERNS — Changelog not generated for this release"
|
||||
4. Skill suggests running `/changelog` to generate it
|
||||
5. Verdict is CONCERNS (advisory — not a hard block)
|
||||
1. Run A emits no certification block, only the line "Certification: omitted —
|
||||
cert_tier is 'none' (internal build, alpha or jam release)."
|
||||
2. Run B asks which platforms are in scope instead of emitting every track;
|
||||
with no answer it emits `### Certification: NOT ASSESSED — cert tier unknown`
|
||||
and says why
|
||||
3. In neither run does the `all` argument cause every certification track to
|
||||
be emitted
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Verdict is CONCERNS (not RELEASE BLOCKED — changelog is advisory)
|
||||
- [ ] `/changelog` is suggested as the remediation
|
||||
- [ ] Other passing checks are shown in the report
|
||||
- [ ] Missing changelog is described as advisory, not blocking
|
||||
- [ ] Run A shows the "omitted — cert_tier is 'none'" line and no certification rows
|
||||
- [ ] Run B asks which platforms are in scope
|
||||
- [ ] Run B's certification section is `NOT ASSESSED — cert tier unknown`, not the `none` line
|
||||
- [ ] Neither run emits itch.io, Steamworks and console certification together
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Previous Release Checklist Exists — Delta From Last Release
|
||||
### Case 4: Edge Case — Nothing to Scan, No Test Output
|
||||
|
||||
**Fixture:**
|
||||
- `production/releases/release-checklist-2026-03-20.md` exists
|
||||
- Previous: 1 story was incomplete, 1 HIGH bug open
|
||||
- Current: all stories Done, HIGH bug resolved, but now 1 MEDIUM bug appeared
|
||||
- `platform.cert_tier: steam`
|
||||
- No `engine.name`, no legacy engine value, and none of `src/`, `Assets/`,
|
||||
`Source/` exists (code root unresolved)
|
||||
- No test output directories or CI logs
|
||||
- `production/qa/bugs/` holds one bug, `BUG-0002.md` (S2, `**Status**: Closed`)
|
||||
|
||||
**Input:** `/release-checklist`
|
||||
**Input:** `/release-checklist pc`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill finds the previous checklist and loads it
|
||||
2. New checklist is generated and compared:
|
||||
- Newly resolved: "Story [X] — was open, now Done"
|
||||
- Newly resolved: "HIGH bug [filename] — was open, now closed"
|
||||
- New item: "1 MEDIUM bug appeared (advisory)"
|
||||
3. Delta section shows all changes prominently
|
||||
4. Verdict is CONCERNS (MEDIUM bug is advisory, not blocking)
|
||||
1. The code root is unresolved, so no scan runs
|
||||
2. Codebase Health reads `NOT ASSESSED — code root unresolved`
|
||||
(`.claude/docs/code-root-resolution.md`), not `0` TODO/FIXME/HACK
|
||||
3. The test line reads `Test results: NOT ASSESSED — no test output or CI logs
|
||||
found` rather than being omitted
|
||||
4. The rest of the checklist is still generated and offered for writing
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Delta section appears in the report with resolved and new items
|
||||
- [ ] Newly resolved items from the previous checklist are noted
|
||||
- [ ] New items not present in the previous checklist are highlighted
|
||||
- [ ] Verdict reflects current state (not previous state)
|
||||
- [ ] No bare `0` count is reported for TODO/FIXME/HACK
|
||||
- [ ] Codebase Health is `NOT ASSESSED — code root unresolved`
|
||||
- [ ] The test-results NOT ASSESSED line is present
|
||||
- [ ] Go / No-Go is never READY: it is NOT ASSESSED naming both unassessed sections,
|
||||
or NOT READY if a blocking item failed
|
||||
- [ ] The write still waits for the "May I write" ask
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; release-checklist is an internal audit
|
||||
### Case 5: Director Gate Check — None; Sign-Offs Are Listed, Not Spawned
|
||||
|
||||
**Fixture:**
|
||||
- Active sprint with stories and bug reports
|
||||
- `platform.cert_tier: console`; source files present
|
||||
- Any review mode
|
||||
|
||||
**Input:** `/release-checklist`
|
||||
**Input:** `/release-checklist all`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill runs the full checklist and writes the report
|
||||
2. No director agents are spawned
|
||||
3. No gate IDs appear in output
|
||||
1. Skill generates the checklist with all device blocks and the console
|
||||
certification block
|
||||
2. The Sign-offs Required list appears as unchecked items
|
||||
3. No director or other agent is spawned; no gate IDs appear
|
||||
4. Next steps point to `/gate-check` and `/team-release`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Verdict is RELEASE READY, RELEASE BLOCKED, or CONCERNS — no gate verdict
|
||||
- [ ] No director gate is invoked and no gate skip message appears
|
||||
- [ ] No subagent is spawned, although sign-offs are listed
|
||||
- [ ] The console certification block is the only certification block
|
||||
- [ ] Next steps name `/gate-check` and `/team-release`
|
||||
|
||||
---
|
||||
|
||||
### Case 6: A Known Blocker Outranks an Unassessed Section
|
||||
|
||||
**Fixture:**
|
||||
- `platform.cert_tier: steam`; `project.stage: Polish`
|
||||
- No `engine.name`, no legacy engine value, and none of `src/`, `Assets/`,
|
||||
`Source/` exists (code root unresolved)
|
||||
- CI test output exists
|
||||
- `production/qa/bugs/BUG-0007.md` is an S1 (Critical) bug with `**Status**: Open`
|
||||
|
||||
**Input:** `/release-checklist pc`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Codebase Health reads `NOT ASSESSED — code root unresolved`
|
||||
2. Quality Gates' "Zero open S1 (Critical) bugs" fails — a blocking item, at every tier
|
||||
3. Go / No-Go is NOT READY: first match wins, and NOT READY ranks above
|
||||
NOT ASSESSED
|
||||
4. The Rationale lists the S1 bug as a blocking item; the unassessed Codebase
|
||||
Health section still reads NOT ASSESSED rather than being dropped
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Go / No-Go is NOT READY — not NOT ASSESSED
|
||||
- [ ] The S1 bug is listed in the Rationale as a blocking item
|
||||
- [ ] Codebase Health still shows `NOT ASSESSED — code root unresolved`
|
||||
|
||||
---
|
||||
|
||||
### Case 7: Bug Thresholds Follow the Workflow Tier
|
||||
|
||||
**Fixture:**
|
||||
- `platform.cert_tier: steam`; source files present; CI test output exists
|
||||
- Run A: `modes.rigor: standard` (`workflow: standard`); `production/qa/bugs/BUG-0012.md`
|
||||
is an S2 (High) bug with `**Status**: Open`, and no other bug is open
|
||||
- Run B: `modes.rigor: full` (`workflow: full`); `production/qa/bugs/BUG-0013.md`
|
||||
is an S3 (Medium) bug with `**Status**: Open`, and no other bug is open
|
||||
|
||||
**Input:** `/release-checklist pc`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Both runs read the bugs from `production/qa/bugs/`, not from the milestone
|
||||
2. Run A: the open S2 fails no item — at `standard` only an open S1 blocks — and
|
||||
is listed in the Rationale as a risk; the S2/S3 row is not emitted
|
||||
3. Run B: the "Zero open S2 (High) and S3 (Medium) bugs" row fails on the open
|
||||
S3, which is named in the Rationale as a blocking item; Go / No-Go is NOT READY
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Run A's Go / No-Go is not NOT READY because of the S2, and the S2 appears in the Rationale as a risk
|
||||
- [ ] Run B's Go / No-Go is NOT READY, naming BUG-0013 as a blocking item
|
||||
- [ ] Neither run offers a "documented exception" for the S3 at `workflow: full`
|
||||
|
||||
---
|
||||
|
||||
### Case 8: No Bug Records — NOT ASSESSED, Not Zero
|
||||
|
||||
**Fixture:**
|
||||
- `platform.cert_tier: steam`; source files present; CI test output exists
|
||||
- `production/qa/bugs/` does not exist
|
||||
|
||||
**Input:** `/release-checklist pc`
|
||||
|
||||
**Expected behavior:**
|
||||
1. The Quality Gates bug line reads `NOT ASSESSED — no bug records`, not `0`
|
||||
2. The bug items are not ticked
|
||||
3. Go / No-Go is NOT ASSESSED (naming the bug items), unless a blocking item
|
||||
failed elsewhere — never READY
|
||||
4. The output says bugs are filed with `/bug-report`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No bare `0` bug count and no ticked "Zero open S1 (Critical) bugs" item
|
||||
- [ ] `NOT ASSESSED — no bug records` appears
|
||||
- [ ] Go / No-Go is not READY
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Checks sprint story completion status
|
||||
- [ ] Checks open bug severity (CRITICAL/HIGH = BLOCKED; MEDIUM/LOW = CONCERNS)
|
||||
- [ ] Checks QA plan sign-off status
|
||||
- [ ] Checks changelog existence
|
||||
- [ ] Compares against previous checklist when one exists
|
||||
- [ ] Asks "May I write" before writing the report
|
||||
- [ ] Verdict is RELEASE READY, RELEASE BLOCKED, or CONCERNS
|
||||
- [ ] Resolves `platform.cert_tier` and branches only on `none | itch | steam | console`
|
||||
- [ ] Treats unset `cert_tier` as a question (or NOT ASSESSED), never as `none`
|
||||
- [ ] Chooses device blocks from the argument and certification from `cert_tier`
|
||||
- [ ] States a denominator with every scan count, or reports NOT ASSESSED
|
||||
- [ ] Reads open bugs from `production/qa/bugs/` and applies the release gate's thresholds for the resolved `workflow` (S1–S3 at `full`, S1 at `standard`/`minimal`); no bug files → `NOT ASSESSED — no bug records`
|
||||
- [ ] Go / No-Go takes the first match: NOT READY, then NOT ASSESSED, then READY
|
||||
- [ ] Presents totals and blockers before asking "May I write"
|
||||
- [ ] Ends with `/gate-check` and `/team-release` as next steps
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- Build stability verification (no failed CI runs) is listed as a check category
|
||||
but relies on external CI system state; the skill notes this as a MANUAL CHECK
|
||||
if CI integration is not configured.
|
||||
- CRITICAL bugs always result in RELEASE BLOCKED regardless of other items;
|
||||
this is equivalent to the HIGH severity case in Case 2.
|
||||
- Stories with `Status: In Review` (not Done) are treated as incomplete
|
||||
and result in RELEASE BLOCKED; this edge case follows the same pattern
|
||||
as the HIGH bug case.
|
||||
- `project.stage` scoping (items for unreached phases marked out of scope) and
|
||||
`modes.rigor: minimal` trimming are not separately tested.
|
||||
- The mobile device block (App Store / Play Store rows kept in the device block
|
||||
by design) is not tested by a dedicated case.
|
||||
- How READY vs NOT READY is decided depends on the unchecked items and the
|
||||
user's inputs; a live run is needed to verify the rationale text.
|
||||
- A resolved code root that is absent or empty reads
|
||||
`NOT ASSESSED — no source files found to scan`; not separately tested.
|
||||
|
||||
@@ -2,28 +2,34 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/reverse-document` generates design or architecture documentation from existing
|
||||
source code. It reads the specified source file(s), infers design intent from
|
||||
class structure, method names, constants, and comments, and produces either a
|
||||
GDD skeleton (for gameplay systems) or an architecture overview (for technical
|
||||
systems). The output is a best-effort inference — magic numbers and undocumented
|
||||
logic may result in a PARTIAL verdict.
|
||||
`/reverse-document <type> <path>` works backwards from existing implementation to
|
||||
a document. `type` is `design` (a GDD, written to `design/gdd/[system-name].md`),
|
||||
`architecture` (an ADR, written to `docs/architecture/[decision-name].md`) or
|
||||
`concept` (a concept doc from a prototype). It reads the code, runs a
|
||||
sufficiency check (stopping when there is too little implementation to infer
|
||||
from), presents its findings with the questions code cannot answer, drafts the
|
||||
document, and asks "May I write this to [output path]?".
|
||||
|
||||
The skill asks "May I write to [inferred path]?" before creating the document.
|
||||
No director gates apply. Verdicts: COMPLETE (clean inference), PARTIAL (some
|
||||
fields are ambiguous and need human review).
|
||||
The `design` output scales with the resolved workflow tier: an 8-section GDD at
|
||||
`full`, a 5-section GDD at `standard`, a one-page brief at `minimal`.
|
||||
`architecture` and `concept` outputs are tier-independent. Intent the user did
|
||||
not confirm is carried into the document marked `INTENT UNKNOWN — inferred from
|
||||
implementation, not confirmed`, and every document carries a
|
||||
"Reverse-documented from implementation" provenance banner. No director gates
|
||||
apply. Verdicts: COMPLETE (document written after approval) or BLOCKED (user
|
||||
declined the write).
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keywords: COMPLETE, PARTIAL
|
||||
- [ ] Contains verdict keywords: COMPLETE, BLOCKED
|
||||
- [ ] Contains "May I write" collaborative protocol language before writing the doc
|
||||
- [ ] Has a next-step handoff (e.g., `/design-review` to validate the generated doc)
|
||||
- [ ] Has a next-step handoff (Phase 8 follow-up list, e.g. `/balance-check`)
|
||||
|
||||
---
|
||||
|
||||
@@ -35,146 +41,199 @@ None. `/reverse-document` is a documentation utility. No director gates apply.
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Well-Structured Source — Accurate design doc skeleton produced
|
||||
### Case 1: Well-Structured Source at `full` — 8-section GDD produced
|
||||
|
||||
**Fixture:**
|
||||
- `src/gameplay/health_system.gd` exists with:
|
||||
- `project.yaml` resolves `workflow: full`; no `system_overrides` row for health
|
||||
- `src/gameplay/health_system.gd` exists (~80 lines) with:
|
||||
- `@export var max_health: int = 100`
|
||||
- `func take_damage(amount: int)` with clamping logic
|
||||
- `func take_damage(amount: int)` containing
|
||||
`health = clamp(health - amount, 0, max_health)`
|
||||
- `signal health_changed(new_value: int)`
|
||||
- Docstrings on all public methods
|
||||
- The user answers every `UNCLEAR INTENT AREAS` question and approves the write
|
||||
|
||||
**Input:** `/reverse-document src/gameplay/health_system.gd`
|
||||
**Input:** `/reverse-document design src/gameplay/health_system.gd`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads the source file and identifies the health system
|
||||
2. Skill infers design intent: max health, take_damage behavior, health signal
|
||||
3. Skill produces GDD skeleton for health system with 8 required sections:
|
||||
Overview, Player Fantasy, Detailed Rules, Formulas, Edge Cases, Dependencies,
|
||||
Tuning Knobs, Acceptance Criteria
|
||||
4. Formulas section includes the inferred clamping formula
|
||||
5. Tuning Knobs notes `max_health = 100` as a configurable value
|
||||
6. Skill asks "May I write to `design/gdd/health-system.md`?"
|
||||
7. File written; verdict is COMPLETE
|
||||
1. Skill reads the source and passes the Phase 3b sufficiency check
|
||||
2. Skill presents findings (MECHANICS IMPLEMENTED, FORMULAS DISCOVERED,
|
||||
UNCLEAR INTENT AREAS) and waits for answers before drafting
|
||||
3. Skill drafts an 8-section GDD (Overview, Player Fantasy, Detailed Rules,
|
||||
Formulas, Edge Cases, Dependencies, Tuning Knobs, Acceptance Criteria)
|
||||
4. The Formulas section records the clamp expression; Tuning Knobs lists
|
||||
`max_health = 100`
|
||||
5. The provenance banner ("Reverse-documented from implementation … from
|
||||
`src/gameplay/health_system.gd`") sits directly under the title
|
||||
6. Skill asks "May I write this to `design/gdd/health-system.md`?"
|
||||
7. File written on approval; verdict is COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All 8 required GDD sections are present in the output
|
||||
- [ ] `max_health = 100` appears as a Tuning Knob
|
||||
- [ ] Clamping formula is captured in the Formulas section
|
||||
- [ ] "May I write" is asked with the inferred path
|
||||
- [ ] Findings are presented and answered before any draft is shown
|
||||
- [ ] All 8 GDD sections are present (tier resolved as `full`)
|
||||
- [ ] The clamp expression appears in Formulas; `max_health = 100` appears as a Tuning Knob
|
||||
- [ ] The provenance banner appears directly under the title
|
||||
- [ ] "May I write" names the path under `design/gdd/`
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Ambiguous Source — Magic Numbers, PARTIAL Verdict
|
||||
### Case 2: Unanswered Intent — INTENT UNKNOWN carried into the draft
|
||||
|
||||
**Fixture:**
|
||||
- `src/gameplay/enemy_ai.gd` exists with:
|
||||
- Inline magic numbers: `if distance < 150:`, `speed = 3.5`
|
||||
- `project.yaml` resolves `workflow: full`
|
||||
- `src/gameplay/enemy_ai.gd` exists (~120 lines) with:
|
||||
- A patrol/chase/attack state machine
|
||||
- Inline constants with use sites: `if distance < 150:`, `speed = 3.5`
|
||||
- No arithmetic expression that computes a gameplay value from inputs
|
||||
- No comments or docstrings
|
||||
- Complex state machine logic that is not self-explanatory
|
||||
- The user replies "just draft it" without answering the intent questions, then
|
||||
approves the write
|
||||
|
||||
**Input:** `/reverse-document src/gameplay/enemy_ai.gd`
|
||||
**Input:** `/reverse-document design src/gameplay/enemy_ai.gd`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads the file and detects magic numbers with no context
|
||||
2. Skill produces a GDD skeleton with notes: "AMBIGUOUS VALUE: 150 (unknown units —
|
||||
is this pixels, world units, or tiles?)"
|
||||
3. Skill marks the Formulas and Tuning Knobs sections as requiring human review
|
||||
4. Skill asks "May I write to `design/gdd/enemy-ai.md`?" with PARTIAL advisory
|
||||
5. File written with PARTIAL markers; verdict is PARTIAL
|
||||
1. Sufficiency check passes (mechanics and tuning values found)
|
||||
2. Findings list the state machine under MECHANICS IMPLEMENTED, the two
|
||||
constants as values, and `FORMULAS DISCOVERED: none found in the source`
|
||||
3. UNCLEAR INTENT AREAS asks what the constants represent (e.g. whether 150 is
|
||||
an aggro range and in what units)
|
||||
4. Because the questions go unanswered, the draft carries each one verbatim as
|
||||
an open question marked `INTENT UNKNOWN — inferred from implementation, not
|
||||
confirmed` — it does not present an inferred intent as settled
|
||||
5. No formula is invented to fill the Formulas section
|
||||
6. Skill asks "May I write this to `design/gdd/enemy-ai.md`?"; verdict COMPLETE
|
||||
on approval
|
||||
|
||||
**Assertions:**
|
||||
- [ ] AMBIGUOUS VALUE annotations appear for magic numbers
|
||||
- [ ] Sections needing human review are marked explicitly
|
||||
- [ ] Verdict is PARTIAL (not COMPLETE)
|
||||
- [ ] File is still written — PARTIAL is not a blocking failure
|
||||
- [ ] `FORMULAS DISCOVERED` reads `none found in the source` (heading kept, not omitted)
|
||||
- [ ] Each unanswered question appears in the document marked `INTENT UNKNOWN — inferred from implementation, not confirmed`
|
||||
- [ ] No formula or design intent absent from the code appears in the draft
|
||||
- [ ] "May I write" is asked before the file is written
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Multiple Interdependent Files — Cross-System Overview Produced
|
||||
### Case 3: Architecture Type on Interdependent Files — tier-independent ADR
|
||||
|
||||
**Fixture:**
|
||||
- User provides 2 source files: `combat_system.gd` and `damage_resolver.gd`
|
||||
- The files reference each other (combat calls damage_resolver)
|
||||
- `project.yaml` resolves `workflow: minimal`
|
||||
- `src/core/combat/` holds `combat_system.gd` and `damage_resolver.gd`;
|
||||
`combat_system.gd` calls `damage_resolver.gd` through an injected reference
|
||||
- The user answers the architecture questions and approves the write
|
||||
|
||||
**Input:** `/reverse-document src/gameplay/combat_system.gd src/gameplay/damage_resolver.gd`
|
||||
**Input:** `/reverse-document architecture src/core/combat/`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads both files and detects the dependency relationship
|
||||
2. Skill produces a cross-system architecture overview (not individual GDDs)
|
||||
3. Overview describes: Combat System → Damage Resolver interaction, shared
|
||||
interfaces, data flow between the two
|
||||
4. Skill asks "May I write to `docs/architecture/combat-damage-overview.md`?"
|
||||
5. Overview written after approval; verdict is COMPLETE (or PARTIAL if ambiguous)
|
||||
1. Skill reads every file under the directory path
|
||||
2. Phase 2 architecture analysis maps the dependency and coupling between the
|
||||
two files (combat_system → damage_resolver)
|
||||
3. Phase 3 asks architecture-intent questions (e.g. why the resolver is
|
||||
injected) rather than only describing the code
|
||||
4. Draft uses the `architecture-doc-from-code.md` template — the `minimal` tier
|
||||
does NOT turn it into a one-page brief
|
||||
5. Skill asks "May I write this to `docs/architecture/[decision-name].md`?";
|
||||
verdict COMPLETE on approval
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Both files are analyzed together (not as two separate docs)
|
||||
- [ ] Cross-system dependency is documented in the output
|
||||
- [ ] Output file is written to `docs/architecture/` (not `design/gdd/`)
|
||||
- [ ] Verdict is COMPLETE or PARTIAL
|
||||
- [ ] Both files are analyzed, and the combat_system → damage_resolver dependency is documented
|
||||
- [ ] Output is an ADR at `docs/architecture/`, not `design/gdd/` or a `-brief.md`
|
||||
- [ ] The `minimal` tier does not change the architecture output
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Source File Not Found — Error
|
||||
### Case 4: Thin Source — Sufficiency check stops the run
|
||||
|
||||
**Fixture:**
|
||||
- `src/gameplay/inventory_system.gd` does not exist
|
||||
- `src/gameplay/inventory_system.gd` exists with only
|
||||
`class_name InventorySystem`, `extends Node` and an empty `_ready()` —
|
||||
no mechanics, no formulas, no tuning values (under 20 non-boilerplate lines)
|
||||
|
||||
**Input:** `/reverse-document src/gameplay/inventory_system.gd`
|
||||
**Input:** `/reverse-document design src/gameplay/inventory_system.gd`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill attempts to read the specified file — not found
|
||||
2. Skill outputs: "Source file not found: src/gameplay/inventory_system.gd"
|
||||
3. Skill suggests checking the path or running `/map-systems` to identify
|
||||
the correct source file
|
||||
4. No document is created
|
||||
1. Skill reads the file; Phase 3b counts 0 mechanics, 0 formulas, 0 values
|
||||
2. Skill stops with: "`src/gameplay/inventory_system.gd` does not contain enough
|
||||
implementation to reverse-document. Found: 0 mechanics, 0 formulas, 0 tuning
|
||||
values. …"
|
||||
3. Skill points to `/design-system [name]` as the skill that captures a design
|
||||
that exists only in the user's head
|
||||
4. No findings are presented, no draft is produced, no file is written
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Error message names the missing file with the full path
|
||||
- [ ] Alternative suggestion (check path or `/map-systems`) is provided
|
||||
- [ ] The stop message names the path and the three counts (all 0)
|
||||
- [ ] `/design-system` is suggested — not a fabricated skeleton
|
||||
- [ ] No write tool is called
|
||||
- [ ] No verdict is issued (error state)
|
||||
- [ ] No verdict is issued (the run stops before Phase 4)
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; reverse-document is a utility
|
||||
### Case 5: Director Gate Check — No gate; declined write is BLOCKED
|
||||
|
||||
**Fixture:**
|
||||
- Well-structured source file exists
|
||||
- Same source as Case 1; the user declines at the "May I write" prompt
|
||||
|
||||
**Input:** `/reverse-document src/gameplay/health_system.gd`
|
||||
**Input:** `/reverse-document design src/gameplay/health_system.gd`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill generates and writes the design doc
|
||||
2. No director agents are spawned
|
||||
3. No gate IDs appear in output
|
||||
1. Skill analyzes, presents findings, and drafts the document
|
||||
2. No director agents are spawned; no gate IDs appear in output
|
||||
3. User declines the write; no file is created
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Verdict is COMPLETE or PARTIAL — no gate verdict involved
|
||||
- [ ] Verdict is BLOCKED (user declined write) — no gate verdict involved
|
||||
|
||||
---
|
||||
|
||||
### Case 6: Default Tier — a single system becomes a one-page brief
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` sets no `modes.rigor` and no `modes.workflow`, so the resolved
|
||||
tier is `minimal` — the default; no `system_overrides`
|
||||
- Same source as Case 1 (`src/gameplay/health_system.gd`)
|
||||
- The user answers every `UNCLEAR INTENT AREAS` question and approves the write
|
||||
|
||||
**Input:** `/reverse-document design src/gameplay/health_system.gd`
|
||||
|
||||
**Expected behavior:**
|
||||
1. The tier resolves to `minimal` from the config block, not assumed
|
||||
2. Findings are presented and answered before any draft
|
||||
3. The draft is a one-page game brief in the `.claude/docs/templates/game-brief.md`
|
||||
format — not a GDD — for this one system
|
||||
4. The provenance banner sits directly under the title
|
||||
5. Skill asks "May I write this to `design/health-system-brief.md`?" — not
|
||||
`design/gdd/` and not `design/game-brief.md`, which is the whole-game brief
|
||||
6. File written on approval; verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The output is a brief in the game-brief format, not an 8- or 5-section GDD
|
||||
- [ ] The path is `design/[system-name]-brief.md`
|
||||
- [ ] The provenance banner appears under the title
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Reads source file(s) before generating any content
|
||||
- [ ] Produces all 8 required GDD sections when target is a gameplay system
|
||||
- [ ] Annotates ambiguous values with AMBIGUOUS VALUE markers
|
||||
- [ ] Produces cross-system overview (not individual GDDs) for multiple files
|
||||
- [ ] Reads the source before generating any content
|
||||
- [ ] Stops at the Phase 3b sufficiency check when mechanics, formulas and values all count zero
|
||||
- [ ] Waits for answers to the intent questions; unanswered ones are carried as `INTENT UNKNOWN`
|
||||
- [ ] `design` output matches the resolved tier (8 sections at `full`, 5 at `standard`, brief at `minimal`)
|
||||
- [ ] Emits the provenance banner under the title of every document it writes
|
||||
- [ ] Asks "May I write" before creating any output file
|
||||
- [ ] Verdict is COMPLETE (clean inference) or PARTIAL (ambiguous fields)
|
||||
- [ ] Verdict is COMPLETE (written) or BLOCKED (write declined)
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- Architecture overview format (for technical/infrastructure systems) differs
|
||||
from GDD format; the inferred output type is determined by the nature of the
|
||||
source file (gameplay logic → GDD; engine/infra code → architecture doc).
|
||||
- The case where a source file is readable but contains only auto-generated
|
||||
boilerplate with no meaningful logic is not tested; skill would likely produce
|
||||
a near-empty skeleton with a PARTIAL verdict.
|
||||
- C# and Blueprint source files follow the same inference pattern as GDScript;
|
||||
language-specific differences are handled in the skill body.
|
||||
- The `standard` tier (5-section GDD without Player Fantasy and Tuning Knobs)
|
||||
follows Case 1's flow with a different section set; not separately
|
||||
fixture-tested. The `minimal` brief is Case 6.
|
||||
- The `concept` type (prototype → `prototypes/[name]/CONCEPT.md` or
|
||||
`design/concepts/[name].md`) is not separately tested.
|
||||
- `guided` and `autonomous` automation modes change the write prompt per
|
||||
`automation-modes.md`; only `collaborative` is tested here.
|
||||
|
||||
@@ -2,29 +2,38 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/setup-engine` configures the project's engine, language, rendering backend,
|
||||
physics engine, specialist agent assignments, and naming conventions by
|
||||
populating `technical-preferences.md`. It accepts an optional engine argument
|
||||
(e.g., `/setup-engine godot`) to skip the engine-selection step. For each
|
||||
section of `technical-preferences.md`, the skill presents a draft and asks
|
||||
"May I write to `technical-preferences.md`?" before updating.
|
||||
`/setup-engine` pins the project's engine, version and language. It accepts
|
||||
`[engine]`, `[engine] [version]`, `refresh`, `upgrade [old] [new]`, or no
|
||||
argument for guided selection. For Godot it asks which language (GDScript, C#
|
||||
or Both) before showing the Technology Stack. It then:
|
||||
|
||||
The skill also populates the specialist routing table (file extension → agent
|
||||
mappings) based on the chosen engine. It has no director gates — configuration
|
||||
is a technical utility task. The verdict is always COMPLETE when the file is
|
||||
fully written.
|
||||
- writes the Technology Stack and the `ENGINE-REFERENCE-IMPORT` line to
|
||||
`CLAUDE.md` after "May I write these engine settings to `CLAUDE.md`?"
|
||||
- fills `.claude/docs/technical-preferences.md` (the legacy mirror: naming
|
||||
conventions, specialist routing table, input/platform, performance budgets)
|
||||
after the user approves the presented defaults
|
||||
- dual-writes the `engine`, `specialists`, `naming` and `commands` blocks to
|
||||
`project.yaml` — the primary config store — after "May I write the `engine`,
|
||||
`specialists`, `naming`, and `commands` blocks to `project.yaml`?"
|
||||
- updates or creates `docs/engine-reference/<engine>/`, scaffolds
|
||||
`project.godot` for Godot (never for Unity or Unreal), and reads everything
|
||||
back with `project-coherence.sh`
|
||||
|
||||
On a re-run it edits existing `project.yaml` blocks in place rather than
|
||||
duplicating them. No director gates apply. The verdict is COMPLETE when the
|
||||
engine is configured.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keyword: COMPLETE
|
||||
- [ ] Contains "May I write" collaborative protocol language before updating technical-preferences.md
|
||||
- [ ] Has a next-step handoff (e.g., `/brainstorm` or `/start` depending on flow)
|
||||
- [ ] Contains "May I write" language before writing `CLAUDE.md` and before writing the `project.yaml` blocks
|
||||
- [ ] Has a next-step handoff (Section 12 prints one tier-matched Next-Steps list, e.g. `/brainstorm` at `minimal`)
|
||||
|
||||
---
|
||||
|
||||
@@ -39,30 +48,39 @@ None. `/setup-engine` is a technical configuration skill. No director gates appl
|
||||
### Case 1: Godot 4 + GDScript — Full engine configuration
|
||||
|
||||
**Fixture:**
|
||||
- `technical-preferences.md` contains only placeholders
|
||||
- Engine argument provided: `godot`
|
||||
- `project.yaml` has no `engine` block; `technical-preferences.md` holds only placeholders
|
||||
- No `design/gdd/game-concept.md` and no `design/game-brief.md`
|
||||
- `docs/engine-reference/godot/VERSION.md` already pins 4.6
|
||||
- No `project.godot` at the repo root; the user approves every write
|
||||
|
||||
**Input:** `/setup-engine godot`
|
||||
**Input:** `/setup-engine godot 4.6`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill skips engine-selection step (argument provided)
|
||||
2. Skill presents language options for Godot: GDScript or C#
|
||||
3. User selects GDScript
|
||||
4. Skill drafts all engine sections: engine/language/rendering/physics fields,
|
||||
naming conventions (snake_case for GDScript), specialist assignments
|
||||
(godot-specialist, gdscript-specialist, godot-shader-specialist, etc.)
|
||||
5. Skill populates the routing table: `.gd` → gdscript-specialist, `.gdshader` →
|
||||
godot-shader-specialist, `.tscn` → godot-specialist
|
||||
6. Skill asks "May I write to `technical-preferences.md`?"
|
||||
7. File is written after approval; verdict is COMPLETE
|
||||
1. Full-spec mode: Section 2's guided selection is skipped
|
||||
2. Skill asks the Godot language question (A GDScript / B C# / C Both); user picks GDScript
|
||||
3. Skill shows the Technology Stack and asks "May I write these engine settings to `CLAUDE.md`?"
|
||||
4. Technical preferences are presented with GDScript naming (snake_case files and
|
||||
variables) and the A3 routing table: `.gd` → godot-gdscript-specialist,
|
||||
`.gdshader` → godot-shader-specialist, `.tscn` → godot-specialist
|
||||
5. Because Section 2 never ran and no design artifact exists, the skill asks
|
||||
the project-shape question (2D/3D, web or not) instead of writing
|
||||
`Forward+`/`Jolt` from the engine name
|
||||
6. `commands.build` export preset is asked, never defaulted; `commands.test` is
|
||||
the gdUnit4 `GdUnitCmdTool.gd` command
|
||||
7. Skill asks "May I write the `engine`, `specialists`, `naming`, and `commands` blocks to `project.yaml`?"
|
||||
8. Section 6 reads its coverage table: 4.6 is beyond the Godot row (~4.3), so
|
||||
the knowledge risk is HIGH
|
||||
9. VERSION.md already pins 4.6 → nothing to regenerate; only its
|
||||
`Installed at pin time` row is written, from the Section 3 probe
|
||||
10. Skill proposes `project.godot` and asks before creating it; verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Engine field is set to Godot 4 (not a placeholder)
|
||||
- [ ] Language field is set to GDScript
|
||||
- [ ] Naming conventions are GDScript-appropriate (snake_case)
|
||||
- [ ] Routing table includes `.gd`, `.gdshader`, and `.tscn` entries
|
||||
- [ ] Specialists are assigned (not placeholders)
|
||||
- [ ] "May I write" is asked before writing
|
||||
- [ ] `engine.name` is Godot and `engine.language` is GDScript in `project.yaml`
|
||||
- [ ] `naming.variables` and `naming.files` are `snake_case`
|
||||
- [ ] Routing table includes `.gd`, `.gdshader`, and `.tscn` entries; `specialists.code` is `godot-gdscript-specialist`
|
||||
- [ ] The rendering/physics shape is asked, not inferred from the engine name
|
||||
- [ ] "May I write" is asked before `CLAUDE.md` and before `project.yaml` are written
|
||||
- [ ] Knowledge Risk is HIGH — the Section 6 table's cutoff is May 2025 and its Godot row ~4.3, as in the shipped VERSION.md
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
@@ -70,73 +88,89 @@ None. `/setup-engine` is a technical configuration skill. No director gates appl
|
||||
### Case 2: Unity + C# — Unity-specific configuration
|
||||
|
||||
**Fixture:**
|
||||
- `technical-preferences.md` contains only placeholders
|
||||
- Engine argument provided: `unity`
|
||||
- Placeholders only in `technical-preferences.md`; no `engine` block in `project.yaml`
|
||||
- The user supplies the version and the build target when asked
|
||||
|
||||
**Input:** `/setup-engine unity`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill sets engine to Unity, language to C#
|
||||
2. Naming conventions are C#-appropriate (PascalCase for classes, camelCase for fields)
|
||||
3. Specialist assignments reference unity-specialist, csharp-specialist
|
||||
4. Routing table: `.cs` → csharp-specialist, `.asmdef` → unity-specialist,
|
||||
`.unity` (scene) → unity-specialist
|
||||
5. Skill asks "May I write to `technical-preferences.md`?" and writes on approval
|
||||
1. Engine-only mode: version looked up and confirmed with the user
|
||||
2. No language question (Unity is C#); CLAUDE.md Language is `C#`
|
||||
3. Naming: PascalCase classes/methods, `_camelCase` private fields
|
||||
4. Routing: `.cs` → unity-specialist, `.shader`/`.shadergraph` → unity-shader-specialist,
|
||||
`.unity`/`.prefab` → unity-specialist, `.uxml`/`.uss` → unity-ui-specialist
|
||||
5. `rendering: URP`, `physics: PhysX` written directly — no shape question
|
||||
6. `commands.build` target is asked (no hardcoded `StandaloneLinux64`);
|
||||
`naming.constants` carries the `# UNVERIFIED` comment
|
||||
7. No Unity project is scaffolded; the user is told to create it in Unity Hub
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Engine field is set to Unity (not Godot or Unreal)
|
||||
- [ ] Language field is set to C#
|
||||
- [ ] Engine field is set to Unity and Language to C#
|
||||
- [ ] Naming conventions reflect C# conventions
|
||||
- [ ] Routing table includes `.cs` and `.unity` entries
|
||||
- [ ] No project-shape question is asked for Unity
|
||||
- [ ] No Unity project files are written
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Unreal + Blueprint — Unreal-specific configuration
|
||||
### Case 3: Unreal — C++ primary, Blueprint routed to its specialist
|
||||
|
||||
**Fixture:**
|
||||
- `technical-preferences.md` contains only placeholders
|
||||
- Engine argument provided: `unreal`
|
||||
- Placeholders only; the user never states that the project is Blueprint-primary
|
||||
- The session runs on Windows (`uname -s` is neither `Linux` nor `Darwin`), so the Windows `commands` block applies
|
||||
|
||||
**Input:** `/setup-engine unreal`
|
||||
**Input:** `/setup-engine unreal 5.7`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill sets engine to Unreal Engine 5, primary language to Blueprint (Visual Scripting)
|
||||
2. Specialist assignments reference unreal-specialist, blueprint-specialist
|
||||
3. Routing table: `.uasset` → blueprint-specialist or unreal-specialist,
|
||||
`.umap` → unreal-specialist
|
||||
4. Performance budgets are pre-set with Unreal defaults (e.g., higher draw call budget)
|
||||
5. Skill asks "May I write" and writes on approval; verdict is COMPLETE
|
||||
1. The primary-language question defaults to C++, so CLAUDE.md Language reads
|
||||
`C++ (primary), Blueprint (gameplay prototyping)` — the line follows the
|
||||
answer, and a Blueprint-primary answer would write
|
||||
`Blueprint (primary), C++ where needed`
|
||||
2. `project.yaml` gets `engine.language: "C++"` (Blueprint only when the project is Blueprint-primary)
|
||||
3. Routing: `.uasset` BP classes → ue-blueprint-specialist; `.umap`/`.uasset` → unreal-specialist;
|
||||
`specialists.additional` includes ue-blueprint-specialist
|
||||
4. `commands` are written as single-quoted scalars under a
|
||||
`# TODO: confirm these` comment; `engine.path` is recorded
|
||||
5. Performance budgets are asked via `AskUserQuestion` (set defaults / leave
|
||||
`[TO BE CONFIGURED]`), not pre-filled
|
||||
6. No `.uproject` is scaffolded; verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Engine field is set to Unreal Engine 5
|
||||
- [ ] Engine field is set to Unreal Engine 5.7 and `engine.language` is `C++`
|
||||
- [ ] Routing table includes `.uasset` and `.umap` entries
|
||||
- [ ] Blueprint specialist is assigned
|
||||
- [ ] ue-blueprint-specialist is assigned for Blueprint graphs
|
||||
- [ ] `commands` carry the `# TODO: confirm these` comment
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Engine Already Configured — Offers to reconfigure specific sections
|
||||
### Case 4: Re-run on a Configured Project — blocks edited in place
|
||||
|
||||
**Fixture:**
|
||||
- `technical-preferences.md` has engine set to Godot 4 with all fields populated
|
||||
- No engine argument provided
|
||||
- `project.yaml` already has `engine` (Godot 4.6), `specialists`, `naming`, a
|
||||
`modes` block, and a `commands` block missing its `smoke` key
|
||||
- `project.godot` exists at the repo root
|
||||
- `docs/engine-reference/godot/VERSION.md` pins 4.6
|
||||
|
||||
**Input:** `/setup-engine`
|
||||
**Input:** `/setup-engine godot 4.6`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads `technical-preferences.md` and detects fully configured engine (Godot 4)
|
||||
2. Skill reports: "Engine already configured as Godot 4 + GDScript"
|
||||
3. Skill presents options: reconfigure all, reconfigure specific section only
|
||||
(Engine/Language, Naming Conventions, Specialists, Performance Budgets)
|
||||
4. User selects "Reconfigure Performance Budgets only"
|
||||
5. Only the performance budget section is updated; all other fields unchanged
|
||||
6. Skill asks "May I write to `technical-preferences.md`?" and writes on approval
|
||||
1. Skill reads `project.yaml` before editing it
|
||||
2. Each existing block is edited in place; `commands.smoke` is added to the
|
||||
partial block; no second copy of any block is appended
|
||||
3. `modes` and every other non-engine block are preserved untouched
|
||||
4. The existing `project.godot` is not overwritten — the skill says so and skips
|
||||
5. VERSION.md pins the same version → nothing to regenerate: Engine Version,
|
||||
Project Pinned and Last Docs Verified are left as they are, and only the
|
||||
`Installed at pin time` row is written with the Section 3 probe result
|
||||
6. Verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Skill does NOT overwrite all fields when only a section update was requested
|
||||
- [ ] User is offered section-specific reconfiguration
|
||||
- [ ] Only the selected section is modified in the written file
|
||||
- [ ] No block is duplicated in `project.yaml`, and the missing `commands.smoke` key is added
|
||||
- [ ] `modes` (and all other non-engine content) is unchanged
|
||||
- [ ] `project.godot` is not overwritten
|
||||
- [ ] For a same-version pin, Engine Version, Project Pinned and Last Docs Verified are untouched and `Installed at pin time` is written
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
@@ -160,23 +194,83 @@ None. `/setup-engine` is a technical configuration skill. No director gates appl
|
||||
|
||||
---
|
||||
|
||||
### Case 6: Installed Editor Older Than the Chosen Version — ask, then record
|
||||
|
||||
**Fixture:**
|
||||
- As Case 1; `docs/engine-reference/godot/VERSION.md` pins 4.6
|
||||
- `godot --version` reports `4.5.1.stable`
|
||||
- The user picks "pin the newer one and upgrade later"
|
||||
|
||||
**Input:** `/setup-engine godot 4.6`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Section 3 probes the installed editor and finds 4.5.1 — a mismatch
|
||||
2. Skill states both versions and asks, offering three options: pin the
|
||||
installed version, pin the newer one and upgrade later, or pin the newer one
|
||||
deliberately
|
||||
3. The answer is recorded; the skill does not pick for the user
|
||||
4. VERSION.md already pins 4.6, so nothing is regenerated, but its
|
||||
`Installed at pin time` row is written with 4.5.1
|
||||
5. Verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Both versions are stated and all three options are offered
|
||||
- [ ] The skill waits for the user's choice rather than picking silently
|
||||
- [ ] `Installed at pin time` records 4.5.1 — the same-version branch does not skip it
|
||||
- [ ] Engine Version, Project Pinned and Last Docs Verified are not restamped
|
||||
|
||||
---
|
||||
|
||||
### Case 7: Godot not on PATH — the probed executable goes into `commands.*`
|
||||
|
||||
**Fixture:**
|
||||
- As Case 1, on Windows; `godot --version` fails — `godot` is not on `PATH`
|
||||
- The probe finds `C:/Program Files/Godot/Godot_v4.6.1-stable_win64.exe`, which reports `4.6.1.stable`
|
||||
- The user names the export preset `Windows Desktop`
|
||||
|
||||
**Input:** `/setup-engine godot 4.6`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Section 3 reads the version from the executable it found — it does not report "no engine installed"
|
||||
2. `build`, `test`, `run` and `smoke` each start with the quoted full path, as single-quoted scalars; `build` quotes its preset with double quotes:
|
||||
`build: '"C:/Program Files/Godot/Godot_v4.6.1-stable_win64.exe" --headless --export-debug "Windows Desktop"'`
|
||||
3. `test` keeps `--remote-debug tcp://127.0.0.1:0` and the gdUnit4 runner arguments after the path
|
||||
4. `engine.path` records the executable
|
||||
5. Verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No `commands.*` value names bare `godot`
|
||||
- [ ] Every Godot command value is a single-quoted scalar and none holds both quote types
|
||||
- [ ] `engine.path` records the editor executable
|
||||
- [ ] Variant — the probe finds nothing and the user does not know where Godot is: bare `godot` is kept under a `# TODO: godot is not on PATH here` comment, and the skill says the commands will not run until the path is set
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Presents draft configuration before asking to write
|
||||
- [ ] Asks "May I write to `technical-preferences.md`?" before writing
|
||||
- [ ] Respects engine argument when provided (skips selection step)
|
||||
- [ ] Detects existing config and offers partial reconfigure
|
||||
- [ ] Routing table is populated for all key file types for the chosen engine
|
||||
- [ ] Verdict is COMPLETE after file is written
|
||||
- [ ] Asks "May I write" before writing `CLAUDE.md` and the `project.yaml` blocks; technical preferences are written only after the user approves the presented defaults
|
||||
- [ ] Respects engine argument when provided (skips guided selection)
|
||||
- [ ] Dual-writes `project.yaml` and `technical-preferences.md`, then reads `project.yaml` back to confirm they agree
|
||||
- [ ] On re-run, edits existing `project.yaml` blocks in place without duplicating them
|
||||
- [ ] Writes the probed editor path into `commands.*` — for Unity always, for Godot whenever `godot` is not on `PATH` — never a bare executable that does not resolve
|
||||
- [ ] Never defaults the Godot export preset or the Unity build target — asks for it; Unreal's `commands` are best-effort values matched to the machine's OS (`-platform=Win64` on Windows, `Linux` on Linux) under a `# TODO` comment; on macOS only `build` is written, and `test`, `run` and `smoke` stay TODO lines
|
||||
- [ ] Writes the Section 3 probe into VERSION.md's `Installed at pin time` row whatever the outcome, after asking "May I record `Installed at pin time: …` in `docs/engine-reference/<engine>/VERSION.md`?"
|
||||
- [ ] Verdict is COMPLETE after configuration is written
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- Godot 4 + C# (instead of GDScript) follows the same flow as Case 1 with
|
||||
different naming conventions and the godot-csharp-specialist assignment.
|
||||
This variant is not separately tested.
|
||||
- The engine-version-specific guidance (e.g., Godot 4.6 knowledge gap warning
|
||||
from VERSION.md) is surfaced by the skill but not assertion-tested here.
|
||||
- Performance budget defaults per engine are noted as engine-specific but
|
||||
exact default values are not assertion-tested.
|
||||
- Godot C# and Both follow Case 1 with the A1/A2/A3 variants from
|
||||
`references/godot-language-config.md`; not separately tested.
|
||||
- The `refresh` and `upgrade` subcommands (Sections 10–11) are not tested here.
|
||||
- The Unreal Linux and macOS `commands` blocks (chosen by `uname -s`) are not
|
||||
fixture-tested; Case 3 pins Windows.
|
||||
- Section 6 is a lookup against the skill's own coverage table, asserted in
|
||||
Case 1. The Section 7 reference docs for a version with no reference set yet
|
||||
depend on WebSearch results; not assertion-tested.
|
||||
- A probe that cannot run reports `installed version NOT DETERMINED` — never
|
||||
"no engine installed"; not separately tested.
|
||||
- A Blueprint-primary Unreal answer (`engine.language: "Blueprint"`) follows
|
||||
Case 3 with the other Language line; not separately tested.
|
||||
|
||||
@@ -2,29 +2,35 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/skill-improve` runs an automated test-fix-retest improvement loop on a skill
|
||||
file. It invokes `/skill-test static` (and optionally `/skill-test category`) to
|
||||
establish a baseline score, diagnoses the failing checks, proposes targeted fixes
|
||||
to the SKILL.md file, asks "May I write the improvements to [skill path]?", applies
|
||||
the fixes, and re-runs the tests to confirm improvement.
|
||||
`/skill-improve [skill-name]` runs one test → fix → retest → keep-or-revert
|
||||
cycle on a single skill. It records a static baseline with
|
||||
`/skill-test static [name]` (FAIL and WARN counts, and which of Checks 1–7
|
||||
failed), then a category baseline with `/skill-test category [name]` when the
|
||||
skill has a `category:` in `CCGS Skill Testing Framework/catalog.yaml`. If both
|
||||
baselines are clean it stops with "No improvements needed." Otherwise it shows
|
||||
a diagnosis, proposes before/after fixes, asks "May I write this improved
|
||||
version to `.claude/skills/[name]/SKILL.md`?", writes, re-runs the same tests,
|
||||
and shows a before → after comparison per dimension plus one combined line
|
||||
(`Combined: 3 → 0 (improved)`).
|
||||
|
||||
If the proposed fix makes the skill worse (regression), the fix is reverted (with
|
||||
user confirmation) rather than applied. If the skill is already perfect (0 failures),
|
||||
the skill exits immediately without making changes. No director gates apply. Verdicts:
|
||||
IMPROVED (score went up), NO CHANGE (no improvements possible or user declined), or
|
||||
REVERTED (fix was applied but caused regression and was reverted).
|
||||
Phase 6 compares the combined count (static FAILs + category FAILs + static
|
||||
WARNs + category WARNs). Lower than baseline → "Score improved. Changes kept."
|
||||
Same or worse → "Combined score did not improve." followed by "May I restore
|
||||
`.claude/skills/[name]/SKILL.md` to the content it had before this run?" — the
|
||||
restore runs only on a yes, and writes back the copy recorded in Phase 5 (never
|
||||
`git checkout`, which would also drop earlier uncommitted edits). No director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keywords: IMPROVED, NO CHANGE, REVERTED
|
||||
- [ ] Phase 6 states both outcomes: "Score improved. Changes kept." and "Combined score did not improve."
|
||||
- [ ] Contains "May I write" collaborative protocol language before applying fixes
|
||||
- [ ] Has a next-step handoff (e.g., run `/skill-test spec` to validate behavioral compliance)
|
||||
- [ ] Has a next-step handoff (Phase 7: `/skill-test static all`, `/skill-improve [next-name]`, `/skill-test audit`)
|
||||
|
||||
---
|
||||
|
||||
@@ -36,119 +42,122 @@ None. `/skill-improve` is a meta-utility skill. No director gates apply.
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — Skill With 2 Static Failures, Both Fixed, IMPROVED
|
||||
### Case 1: Happy Path — Check 4 FAIL and Check 5 WARN fixed, changes kept
|
||||
|
||||
**Fixture:**
|
||||
- `.claude/skills/some-skill/SKILL.md` has 2 static failures:
|
||||
- Check 4: no "May I write" language despite having Write in allowed-tools
|
||||
- Check 5: no next-step handoff at the end
|
||||
- `.claude/skills/some-skill/SKILL.md` has `Write` in `allowed-tools` but no
|
||||
ask-before-write language (Check 4 FAIL) and no next-step section (Check 5 WARN)
|
||||
- catalog gives `some-skill` `category: utility`; its category baseline is
|
||||
1 FAIL (U1, because static checks fail)
|
||||
- The user approves the write; the re-test is clean in both dimensions
|
||||
|
||||
**Input:** `/skill-improve some-skill`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill runs `/skill-test static some-skill` — baseline: 5/7 checks pass
|
||||
2. Skill diagnoses the 2 failing checks (4 and 5)
|
||||
3. Skill proposes fixes:
|
||||
- Add "May I write" language to the appropriate phase
|
||||
- Add a next-step handoff section at the end
|
||||
4. Skill asks "May I write improvements to `.claude/skills/some-skill/SKILL.md`?"
|
||||
5. Fixes applied; `/skill-test static some-skill` re-run — now 7/7 checks pass
|
||||
6. Verdict is IMPROVED (5→7)
|
||||
1. Static baseline displayed: "Static baseline: 1 failures, 1 warnings" with
|
||||
"Failing: Check 4 (no ask-before-write), Check 5 (no handoff)"
|
||||
2. Category baseline displayed: "Category baseline: 1 failures, 0 warnings (utility rubric)"
|
||||
3. Full diagnosis shown before any change is proposed
|
||||
4. Before/after blocks proposed for only the failing parts; skill asks
|
||||
"May I write this improved version to `.claude/skills/some-skill/SKILL.md`?"
|
||||
5. File written; both tests re-run; comparison shows
|
||||
`Static: Before 1 failures, 1 warnings → After 0 failures, 0 warnings`, the
|
||||
category line, and one combined line, `Combined: 3 → 0 (improved)`
|
||||
6. "Score improved. Changes kept." with a per-dimension summary
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Baseline score is established before any changes (5/7)
|
||||
- [ ] Both failing checks are diagnosed and addressed in the proposed fix
|
||||
- [ ] "May I write" is asked before applying the fix
|
||||
- [ ] Re-test confirms improvement (7/7)
|
||||
- [ ] Verdict is IMPROVED with before/after score shown
|
||||
- [ ] Static and category baselines are displayed before any change is proposed
|
||||
- [ ] Both Check 4 and Check 5 are diagnosed and addressed in the proposed fix
|
||||
- [ ] "May I write" is asked before the file is written
|
||||
- [ ] The before → after comparison is shown for both dimensions, plus the single line `Combined: 3 → 0 (improved)`
|
||||
- [ ] Outcome is "Score improved. Changes kept."
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Fix Causes Regression — Score Comparison Shows Regression, REVERTED
|
||||
### Case 2: Fix Causes Regression — revert offered, runs only on yes
|
||||
|
||||
**Fixture:**
|
||||
- `.claude/skills/some-skill/SKILL.md` has 1 static failure (missing handoff)
|
||||
- Proposed fix inadvertently removes the verdict keywords section
|
||||
(introducing a new failure)
|
||||
- `.claude/skills/some-skill/SKILL.md` has a static baseline of 0 FAILs and
|
||||
1 WARN (Check 5, no handoff)
|
||||
- The applied fix adds a handoff but deletes the section holding the verdict
|
||||
keywords and the ask-before-write line: the static re-test shows 2 FAILs
|
||||
(Checks 3 and 4), and the category result is no better than its baseline —
|
||||
the combined count rises
|
||||
- The user approves the write, then answers yes to the revert
|
||||
|
||||
**Input:** `/skill-improve some-skill`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Baseline: 6/7 checks pass (1 failure: missing handoff)
|
||||
2. Skill proposes fix and asks "May I write improvements?"
|
||||
3. Fix is applied; re-test runs
|
||||
4. Re-test result: 5/7 (fixed the handoff but broke verdict keywords)
|
||||
5. Skill detects regression: score went DOWN
|
||||
6. Skill asks user: "Fix caused a regression (6→5). May I revert the changes?"
|
||||
7. User confirms; changes are reverted; verdict is REVERTED
|
||||
1. Baseline recorded; fix proposed; "May I write …" approved; file written
|
||||
2. The re-test comparison's combined line reads `Combined: [before] → [after] (worse)`,
|
||||
with the after count higher
|
||||
3. Skill reports "Combined score did not improve." and shows what changed and
|
||||
why it may not have helped
|
||||
4. Skill asks "May I restore `.claude/skills/some-skill/SKILL.md` to the content it had before this run?"
|
||||
5. On yes: writes the Phase 5 copy back with Write — it does not run `git checkout`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Re-test score is compared to baseline before finalizing
|
||||
- [ ] Regression is detected when score decreases
|
||||
- [ ] User is asked to confirm revert (not automatic)
|
||||
- [ ] File is reverted on user confirmation
|
||||
- [ ] Verdict is REVERTED
|
||||
- [ ] Re-test combined count is compared to the baseline before finalizing
|
||||
- [ ] A higher combined count takes the "did not improve" branch
|
||||
- [ ] The revert is asked for, not automatic
|
||||
- [ ] The file is restored only after the user says yes, from the recorded copy — never via `git checkout`
|
||||
- [ ] "Score improved. Changes kept." is NOT reported
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Skill With Category Assignment — Baseline Captures Both Scores
|
||||
### Case 3: Gate Skill — combined static + category baseline
|
||||
|
||||
**Fixture:**
|
||||
- `.claude/skills/gate-check/SKILL.md` is a gate skill with 1 static failure
|
||||
and 2 category (G-criteria) failures
|
||||
- `tests/skills/quality-rubric.md` has Gate Skills section
|
||||
- `.claude/skills/gate-check/SKILL.md` has 1 static FAIL and 2 category FAILs
|
||||
(G2 and G4 of the `### gate` section in `CCGS Skill Testing Framework/quality-rubric.md`)
|
||||
- catalog gives `gate-check` `category: gate`; the user approves the write, and
|
||||
the re-test is clean in both dimensions
|
||||
|
||||
**Input:** `/skill-improve gate-check`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill runs both static and category tests for the baseline:
|
||||
- Static: 6/7 checks pass
|
||||
- Category: 3/5 G-criteria pass
|
||||
2. Combined baseline: 9/12
|
||||
3. Skill diagnoses all 3 failures and proposes fixes
|
||||
4. "May I write improvements to `.claude/skills/gate-check/SKILL.md`?"
|
||||
5. Fixes applied; both test types re-run
|
||||
6. Re-test: static 7/7, category 5/5 = 12/12
|
||||
7. Verdict is IMPROVED (9→12)
|
||||
1. Static baseline and category baseline (`gate rubric`) both recorded
|
||||
2. Diagnosis names the static check and the two G-metrics with the exact text gap for each
|
||||
3. "May I write this improved version to `.claude/skills/gate-check/SKILL.md`?"
|
||||
4. Both `/skill-test static gate-check` and `/skill-test category gate-check` re-run
|
||||
5. The comparison ends `Combined: 3 → 0 (improved)`: "Score improved. Changes kept."
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Both static and category scores are captured in the baseline
|
||||
- [ ] Combined score is used for comparison (not just one type)
|
||||
- [ ] The combined count (FAILs + WARNs, both dimensions) is what Phase 6 compares
|
||||
- [ ] All 3 failures are addressed in the proposed fix
|
||||
- [ ] Re-test confirms improvement in both score types
|
||||
- [ ] Verdict is IMPROVED with combined before/after
|
||||
- [ ] Both test types are re-run after the write
|
||||
- [ ] Outcome is "Score improved. Changes kept." with the combined before/after
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Skill Already Perfect — No Improvements Needed
|
||||
### Case 4: Skill Already Clean — No improvements needed
|
||||
|
||||
**Fixture:**
|
||||
- `.claude/skills/brainstorm/SKILL.md` has no static failures
|
||||
- Category score is also 5/5 (if applicable)
|
||||
- `.claude/skills/brainstorm/SKILL.md` has 0 static FAILs and 0 WARNs
|
||||
- catalog gives `brainstorm` `category: utility`; U1 and U2 both pass
|
||||
|
||||
**Input:** `/skill-improve brainstorm`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill runs `/skill-test static brainstorm` — 7/7 checks pass
|
||||
2. If category applies: 5/5 criteria pass
|
||||
3. Skill outputs: "No improvements needed — brainstorm is fully compliant"
|
||||
4. Skill exits without proposing any changes
|
||||
5. No "May I write" is asked; no files are modified
|
||||
6. Verdict is NO CHANGE
|
||||
1. Static baseline 0/0 → proceeds to the category baseline (not an immediate stop)
|
||||
2. Category baseline 0/0 (utility rubric)
|
||||
3. Skill stops: "This skill already passes all static and category checks. No improvements needed."
|
||||
4. No diagnosis, no proposed change, no "May I write", no file modified
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Skill exits immediately after confirming 0 failures
|
||||
- [ ] The category baseline is run before stopping (a clean static result alone does not stop)
|
||||
- [ ] "No improvements needed" message is shown
|
||||
- [ ] No changes are proposed
|
||||
- [ ] No "May I write" is asked
|
||||
- [ ] Verdict is NO CHANGE
|
||||
- [ ] No "May I write" is asked and no file is modified
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; skill-improve is a meta utility
|
||||
|
||||
**Fixture:**
|
||||
- Skill with at least 1 static failure
|
||||
- `.claude/skills/some-skill/SKILL.md` has at least 1 static failure; the user
|
||||
approves the write
|
||||
|
||||
**Input:** `/skill-improve some-skill`
|
||||
|
||||
@@ -160,26 +169,55 @@ None. `/skill-improve` is a meta-utility skill. No director gates apply.
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Verdict is IMPROVED, NO CHANGE, or REVERTED — no gate verdict
|
||||
- [ ] The run ends with one of Phase 6's two outcome messages — no gate verdict
|
||||
|
||||
---
|
||||
|
||||
### Case 6: Equal Score — "no change" does not keep the fix as an improvement
|
||||
|
||||
**Fixture:**
|
||||
- `.claude/skills/some-skill/SKILL.md` has a static baseline of 0 FAILs and
|
||||
1 WARN (Check 7, argument hint does not match the documented modes)
|
||||
- catalog gives `some-skill` no `category:`, so category checks are skipped
|
||||
- The applied fix corrects the hint but drops the closing next-step section:
|
||||
the re-test shows 0 FAILs and 1 WARN (Check 5)
|
||||
- The user approves the write, then answers no to the restore
|
||||
|
||||
**Input:** `/skill-improve some-skill`
|
||||
|
||||
**Expected behavior:**
|
||||
1. "Category: not yet assigned — skipping category checks." is shown
|
||||
2. The comparison's combined line reads `Combined: 1 → 1 (no change)`
|
||||
3. An equal count takes the "did not improve" branch: "Combined score did not
|
||||
improve.", then "May I restore `.claude/skills/some-skill/SKILL.md` to the
|
||||
content it had before this run?"
|
||||
4. On no, the file is left as written and nothing is restored
|
||||
|
||||
**Assertions:**
|
||||
- [ ] An equal combined count is treated as "did not improve", not as improved
|
||||
- [ ] The combined line reads `Combined: 1 → 1 (no change)`
|
||||
- [ ] "Score improved. Changes kept." is NOT reported
|
||||
- [ ] The restore is asked for, and declining it leaves the written file in place
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Always establishes a baseline score before proposing any changes
|
||||
- [ ] Shows before/after score comparison in the output
|
||||
- [ ] Always establishes a baseline before proposing any changes
|
||||
- [ ] Shows the full diagnosis, then before/after blocks, before asking to write
|
||||
- [ ] Asks "May I write" before applying any fix
|
||||
- [ ] Detects regressions by comparing re-test score to baseline
|
||||
- [ ] Detects a non-improvement by comparing the re-test combined count to the baseline — an equal count is a non-improvement
|
||||
- [ ] Shows one combined before → after line, e.g. `Combined: 3 → 0 (improved)`, beside the per-dimension lines
|
||||
- [ ] Asks for user confirmation before reverting (not automatic)
|
||||
- [ ] Ends with IMPROVED, NO CHANGE, or REVERTED verdict
|
||||
- [ ] Ends with "Score improved. Changes kept." or "Combined score did not improve." — or stops earlier with "No improvements needed." / a declined write
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- The improvement loop is designed to run only one fix-retest cycle per
|
||||
invocation; running multiple iterations requires re-invoking `/skill-improve`.
|
||||
- Behavioral compliance (spec-mode test results) is not included in the
|
||||
improvement loop — only structural (static) and category scores are automated.
|
||||
- The case where the skill file cannot be read (permissions error or missing file)
|
||||
is not tested; this would result in an error before the baseline is established.
|
||||
- One fix-retest cycle per invocation; a further pass needs a new `/skill-improve` run.
|
||||
- Behavioral (spec-mode) results are not part of the loop — only static and
|
||||
category scores are.
|
||||
- The Phase 1 stops — usage text when no argument is given, "Skill '[name]' not
|
||||
found." when no `SKILL.md` exists — and the user declining the write in
|
||||
Phase 4 are not separately fixture-tested.
|
||||
|
||||
@@ -2,33 +2,40 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/skill-test` validates skill files for structural correctness, behavioral
|
||||
compliance, and category-rubric scoring. It operates in three modes:
|
||||
`/skill-test` validates skills (`.claude/skills/*/SKILL.md`) and agents
|
||||
(`.claude/agents/*.md`). It has four modes:
|
||||
|
||||
- **static**: Checks a single skill file for structural requirements
|
||||
(frontmatter fields, phase headings, verdict keywords, "May I write" language,
|
||||
next-step handoff) without needing a fixture. Produces a per-check PASS/FAIL
|
||||
table.
|
||||
- **spec**: Reads a test spec file from `tests/skills/` and evaluates the skill
|
||||
against each test case assertion, producing a case-by-case verdict.
|
||||
- **audit**: Produces a coverage table of all skills in `.claude/skills/` and
|
||||
all agents in `.claude/agents/`, showing which have spec files and which do not.
|
||||
- **static** `[name | all]`: skills only. Runs 7 structural checks (frontmatter,
|
||||
phases, verdict keywords, ask-before-write language, next-step handoff, fork
|
||||
complexity, argument hint) and reports each as PASS / WARN / FAIL. Per-skill
|
||||
results are COMPLIANT, WARNINGS, NON-COMPLIANT or NOT ASSESSED. Writes nothing.
|
||||
- **spec** `[skill-or-agent-name]`: resolves the name through
|
||||
`CCGS Skill Testing Framework/catalog.yaml`, reads the file and the spec at the
|
||||
entry's `spec:` path, plus `CLAUDE.md` and every file it imports with `@path`,
|
||||
and marks each assertion — the spec's Static Assertions included, for a skill
|
||||
as for an agent — PASS / PARTIAL / FAIL / NOT ASSESSED. Case and Overall
|
||||
Verdicts are the worst result beneath them (FAIL, PARTIAL, NOT ASSESSED,
|
||||
PASS). It then asks "May I write these results to
|
||||
`CCGS Skill Testing Framework/results/…` and update `catalog.yaml`?"
|
||||
- **category** `[name | all]`: scores the skill or agent against its section of
|
||||
`CCGS Skill Testing Framework/quality-rubric.md` (PASS / FAIL / WARN per
|
||||
metric) and asks "May I update `catalog.yaml` …?" before recording it.
|
||||
- **audit** (also the no-argument default): coverage table of every skill and
|
||||
every catalogued agent. Writes nothing and issues no verdict.
|
||||
|
||||
An additional **category** mode reads the quality rubric for a skill category
|
||||
(e.g., gate skills) and scores the skill against rubric criteria. The verdict
|
||||
system differs by mode.
|
||||
No director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdicts: COMPLIANT, NON-COMPLIANT, WARNINGS (static mode); PASS, FAIL, PARTIAL (spec mode); COMPLETE (audit mode)
|
||||
- [ ] Does NOT contain "May I write" language (skill is read-only in all modes)
|
||||
- [ ] Has a next-step handoff (e.g., `/skill-improve` to fix issues found)
|
||||
- [ ] Contains verdicts: COMPLIANT, WARNINGS, NON-COMPLIANT, NOT ASSESSED (static mode); PASS, PARTIAL, FAIL, NOT ASSESSED (spec mode)
|
||||
- [ ] Contains "May I write" before writing spec results, and "May I update" before recording a category check in `catalog.yaml`
|
||||
- [ ] Has a next-step handoff (Phase 3: contextual follow-up after each mode)
|
||||
|
||||
---
|
||||
|
||||
@@ -43,137 +50,273 @@ None. `/skill-test` is a meta-utility skill. No director gates apply.
|
||||
### Case 1: Static Mode — Well-formed skill, all 7 checks pass, COMPLIANT
|
||||
|
||||
**Fixture:**
|
||||
- `.claude/skills/brainstorm/SKILL.md` exists and is well-formed:
|
||||
- Has all required frontmatter fields
|
||||
- `.claude/skills/brainstorm/SKILL.md` exists and satisfies all 7 checks:
|
||||
- Has all 5 required frontmatter fields
|
||||
- Has ≥2 phase headings
|
||||
- Has verdict keywords
|
||||
- Has "May I write" language
|
||||
- Has a next-step handoff
|
||||
- Documents director gates
|
||||
- Documents gate mode behavior (lean/solo skips)
|
||||
- Ends with a next-step handoff
|
||||
- Does not set `context: fork`
|
||||
- `argument-hint` matches the modes its Parse Arguments phase documents
|
||||
|
||||
**Input:** `/skill-test static brainstorm`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads `.claude/skills/brainstorm/SKILL.md`
|
||||
1. Skill reads `.claude/skills/brainstorm/SKILL.md` fully
|
||||
2. Skill runs all 7 structural checks
|
||||
3. All 7 checks pass
|
||||
4. Skill outputs a PASS/FAIL table with all 7 checks marked PASS
|
||||
5. Verdict is COMPLIANT
|
||||
3. Output is the single-skill block `=== Skill Static Check: /brainstorm ===`
|
||||
listing Check 1 – Check 7, each PASS
|
||||
4. Verdict is COMPLIANT
|
||||
5. Phase 3 follow-up suggests `/skill-test spec brainstorm`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Exactly 7 structural checks are reported
|
||||
- [ ] All 7 are marked PASS
|
||||
- [ ] Verdict is COMPLIANT
|
||||
- [ ] No files are written
|
||||
- [ ] No files are written and no "May I write" is asked
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Static Mode — Skill Missing "May I Write" Despite Write Tool in allowed-tools
|
||||
### Case 2: Static Mode — Skill Missing Ask-Before-Write Despite Write Tool in allowed-tools
|
||||
|
||||
**Fixture:**
|
||||
- `.claude/skills/some-skill/SKILL.md` has `Write` in `allowed-tools` frontmatter
|
||||
- The skill body has no "May I write" or "May I update" language
|
||||
- The body has no ask-before-write language in any form Check 4 accepts:
|
||||
no "May I write", no "before writing"/"approval" near a write instruction,
|
||||
no "ask" and "write" in the same section
|
||||
|
||||
**Input:** `/skill-test static some-skill`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads `some-skill/SKILL.md`
|
||||
2. Check 4 (collaborative write protocol) fails: `Write` in allowed-tools but no
|
||||
"May I write" language found
|
||||
3. All other checks may pass
|
||||
4. Verdict is NON-COMPLIANT with Check 4 as the failing assertion
|
||||
5. Output lists Check 4 as FAIL with explanation
|
||||
2. Check 4 (collaborative protocol) is FAIL — `Write` is in `allowed-tools` and
|
||||
no ask-before-write language is found
|
||||
3. The other six checks are listed with their own results
|
||||
4. Verdict is NON-COMPLIANT, with a Recommended line naming the fix
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Check 4 is marked FAIL
|
||||
- [ ] Explanation identifies the specific mismatch (Write tool without "May I write" language)
|
||||
- [ ] Check 4 is marked FAIL (not WARN — Write is in allowed-tools)
|
||||
- [ ] The Check 4 line or the Recommended line names the mismatch (Write tool without ask-before-write language)
|
||||
- [ ] Verdict is NON-COMPLIANT
|
||||
- [ ] Other passing checks are shown (not only the failure)
|
||||
- [ ] Other checks are shown (not only the failure)
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Spec Mode — gate-check Skill Evaluated Against Spec
|
||||
### Case 3: Spec Mode — gate-check Skill Evaluated Against Its Spec
|
||||
|
||||
**Fixture:**
|
||||
- `tests/skills/gate-check.md` exists with 5 test cases
|
||||
- `.claude/skills/gate-check/SKILL.md` exists
|
||||
- `catalog.yaml` lists `gate-check` under `skills:` with
|
||||
`spec: CCGS Skill Testing Framework/skills/gate/gate-check.md`
|
||||
- That spec has 5 test cases and a Static Assertions list;
|
||||
`.claude/skills/gate-check/SKILL.md` exists
|
||||
- Suppose one of the spec's assertions reads "Balance values link to their
|
||||
source formula or rationale" — met only by `.claude/docs/coding-standards.md`
|
||||
(imported by `CLAUDE.md`), not by the skill or its gate files themselves
|
||||
- The user declines the results write
|
||||
|
||||
**Input:** `/skill-test spec gate-check`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads both the skill file and the spec file
|
||||
2. Skill evaluates each of the 5 test case assertions against the skill's behavior
|
||||
3. For each case: PASS if skill behavior matches spec assertions, FAIL if not
|
||||
4. Skill produces a case-by-case result table
|
||||
5. Overall verdict: PASS (all 5), PARTIAL (some), or FAIL (majority failing)
|
||||
1. Skill resolves `gate-check` as a skill via the catalog and takes the spec
|
||||
path from its `spec:` field
|
||||
2. Skill reads the skill, the spec, `CLAUDE.md` and its `@`-imported files
|
||||
3. The spec's Static Assertions are checked against the SKILL.md in a
|
||||
`Static Assertions:` block before Case 1
|
||||
4. Each assertion is marked PASS / PARTIAL / FAIL / NOT ASSESSED, with a Reason
|
||||
for each non-PASS; the balance-values assertion is PASS, citing
|
||||
`coding-standards.md`
|
||||
5. Each case gets a Case Verdict; the report ends with an Overall Verdict, the
|
||||
worst result across the static assertions, the cases and Protocol Compliance
|
||||
6. Skill asks "May I write these results to
|
||||
`CCGS Skill Testing Framework/results/skill-test-spec-gate-check-[date].md`
|
||||
and update `CCGS Skill Testing Framework/catalog.yaml`?"; on decline nothing is written
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All 5 test cases from the spec are evaluated
|
||||
- [ ] Each case has an individual PASS/FAIL result
|
||||
- [ ] Overall verdict is PASS, PARTIAL, or FAIL based on case results
|
||||
- [ ] No files are written
|
||||
- [ ] The spec path comes from the catalog `spec:` field
|
||||
- [ ] The spec's Static Assertions are evaluated against the SKILL.md, not skipped as "static mode's job"
|
||||
- [ ] All 5 test cases are evaluated, each with its own Case Verdict
|
||||
- [ ] The balance-values assertion is PASS, and the report cites `coding-standards.md` as the file it came from
|
||||
- [ ] Overall verdict is the worst result — FAIL, PARTIAL, NOT ASSESSED or PASS; an assertion-level NOT ASSESSED keeps it from PASS
|
||||
- [ ] No file is written without approval (declined here, so nothing is written)
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Audit Mode — Coverage Table of All Skills and Agents
|
||||
|
||||
**Fixture:**
|
||||
- `.claude/skills/` contains 72+ skill directories
|
||||
- `.claude/agents/` contains 49+ agent files
|
||||
- `tests/skills/` contains spec files for a subset of skills
|
||||
- `.claude/skills/` contains 74 skill directories; 73 have a spec at the path
|
||||
their catalog entry names
|
||||
- `catalog.yaml` has an `agents:` section with 49 entries, all with specs
|
||||
|
||||
**Input:** `/skill-test audit`
|
||||
**Input:** `/skill-test audit` (and, separately, `/skill-test` with no argument)
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill enumerates all skills in `.claude/skills/` and all agents in `.claude/agents/`
|
||||
2. Skill checks `tests/skills/` for a corresponding spec file for each
|
||||
3. Skill produces a coverage table:
|
||||
- Each skill/agent listed
|
||||
- "Has Spec" column: YES or NO
|
||||
- Summary: "X of Y skills have specs; A of B agents have specs"
|
||||
4. Verdict is COMPLETE
|
||||
1. Both inputs run audit mode — no argument is the audit default, not a usage
|
||||
message
|
||||
2. Skill globs `.claude/skills/*/SKILL.md` for skills and reads the catalog's
|
||||
`agents:` section for agents
|
||||
3. For each skill: Has Spec (from the catalog `spec:` path, or a glob under
|
||||
`CCGS Skill Testing Framework/skills/`), last static/category dates and
|
||||
results, category, priority
|
||||
4. Separate SKILLS and AGENTS tables, a Top 5 Priority Gaps list, and the lines
|
||||
`Skill coverage: 73/74 specs (99%)` and `Agent coverage: 49/49 specs (100%)`
|
||||
5. No verdict line; the output ends with the offer to run `static all`,
|
||||
`category all` or `spec [name]`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] `/skill-test` with no argument produces this audit, not usage text
|
||||
- [ ] All skill directories are enumerated (not just a sample)
|
||||
- [ ] "Has Spec" column is accurate for each entry
|
||||
- [ ] Summary counts are correct
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] Summary coverage counts are correct for skills and for agents
|
||||
- [ ] No files are written and no verdict is issued in audit mode
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Category Mode — Gate Skill Evaluated Against Quality Rubric
|
||||
|
||||
**Fixture:**
|
||||
- `tests/skills/quality-rubric.md` exists with a "Gate Skills" section defining
|
||||
criteria G1-G5 (e.g., G1: has mode guard, G2: has verdict table, etc.)
|
||||
- `.claude/skills/gate-check/SKILL.md` is a gate skill
|
||||
- `CCGS Skill Testing Framework/quality-rubric.md` has a `### gate` section with
|
||||
metrics G1–G5
|
||||
- `catalog.yaml` gives `gate-check` `category: gate`
|
||||
- `.claude/skills/gate-check/SKILL.md` meets every metric except G2 — Director
|
||||
panel width (only CD-PHASE-GATE is spawned at `workflow: full`)
|
||||
- The user approves the catalog update
|
||||
|
||||
**Input:** `/skill-test category gate-check`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads `quality-rubric.md` and identifies the Gate Skills section
|
||||
2. Skill evaluates `gate-check/SKILL.md` against criteria G1-G5
|
||||
3. Each criterion is scored: PASS, PARTIAL, or FAIL
|
||||
4. Overall category score is computed (e.g., 4/5 criteria pass)
|
||||
5. Verdict is COMPLIANT (all pass), WARNINGS (some partial), or NON-COMPLIANT (failures)
|
||||
1. Skill reads the `category:` from the catalog, then the `### gate` section of the rubric
|
||||
2. Skill reads the skill and the `CLAUDE.md` context, and evaluates G1–G5
|
||||
3. Each metric is marked PASS, FAIL or WARN; G2 is FAIL with a quoted Gap line
|
||||
4. Report ends `Verdict: FAIL (1 failure, 0 warnings)` and a Fix line
|
||||
5. Skill asks "May I update `CCGS Skill Testing Framework/catalog.yaml` to record
|
||||
this category check (`last_category`, `last_category_result`) for gate-check?"
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All gate criteria (G1-G5) from quality-rubric.md are evaluated
|
||||
- [ ] Each criterion has an individual score
|
||||
- [ ] Overall verdict reflects the score distribution
|
||||
- [ ] No files are written
|
||||
- [ ] All gate metrics (G1–G5) from quality-rubric.md are evaluated
|
||||
- [ ] Each metric has an individual PASS / FAIL / WARN result, with a Gap line for G2
|
||||
- [ ] Overall verdict reflects the metric results
|
||||
- [ ] The catalog is updated only after the "May I update" approval
|
||||
|
||||
---
|
||||
|
||||
### Case 6: Spec Mode — Agent Name Resolves to the Agent
|
||||
|
||||
**Fixture:**
|
||||
- `catalog.yaml` lists `creative-director` under `agents:` with
|
||||
`spec: CCGS Skill Testing Framework/agents/directors/creative-director.md`
|
||||
- `.claude/agents/creative-director.md` declares `model: opus`; there is no
|
||||
`.claude/skills/creative-director/`
|
||||
- The spec asserts "Model tier is `opus`"
|
||||
- Suppose one spec case is a redirect: the agent is asked to pick the project's
|
||||
engine. The agent's "What This Agent Must NOT Do" list says "Make engine or
|
||||
architecture choices (delegate to technical-director)"; it never restates the
|
||||
redirect in its own words
|
||||
|
||||
**Input:** `/skill-test spec creative-director`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill resolves the name through the catalog's `agents:` section — it does not
|
||||
report "Skill 'creative-director' not found"
|
||||
2. Skill reads the agent file and the agent spec
|
||||
3. Static assertions are compared exactly with the frontmatter: "Model tier is
|
||||
`opus`" PASSES against `model: opus`, and would FAIL (not PARTIAL) against
|
||||
`model: inherit`
|
||||
4. Test cases are evaluated against the agent's instructions; Protocol
|
||||
Compliance uses the spec's own list
|
||||
5. The redirect case is met: the loaded `coordination-rules.md` rule 2 covers
|
||||
it and the agent's must-not list names technical-director — the report cites
|
||||
both
|
||||
6. Report header is `=== Agent Spec Test: creative-director ===`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The agent is resolved and evaluated, not refused as a missing skill
|
||||
- [ ] A literal-value assertion is an exact comparison (near match = FAIL)
|
||||
- [ ] Every case and the spec's Protocol Compliance list are evaluated
|
||||
- [ ] The redirect case passes on `coordination-rules.md` plus the agent's own must-not list, both cited — the agent is not required to restate it
|
||||
- [ ] No files are written without approval
|
||||
|
||||
---
|
||||
|
||||
### Case 7: Static All — an unreadable skill is NOT ASSESSED and counted
|
||||
|
||||
**Fixture:**
|
||||
- `.claude/skills/` holds 74 skill directories; one `SKILL.md` cannot be
|
||||
read (the Read call fails — permission denied), so none of its checks can
|
||||
start. (A file that is read but whose frontmatter does not parse is a
|
||||
different case: Check 1 fails and it is NON-COMPLIANT.)
|
||||
|
||||
**Input:** `/skill-test static all`
|
||||
|
||||
**Expected behavior:**
|
||||
1. The 73 readable skills are checked and listed with their Result
|
||||
2. The unreadable skill appears in the table as `NOT ASSESSED`, with the
|
||||
reason — not omitted, not COMPLIANT
|
||||
3. The header states the denominator, `73 of 74 skills checked`, and the
|
||||
Summary line counts the NOT ASSESSED skill
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The unreadable skill's Result is NOT ASSESSED, with a reason
|
||||
- [ ] The header or summary reads `73 of 74 skills checked`
|
||||
- [ ] The Summary counts sum to the skills in the table
|
||||
|
||||
---
|
||||
|
||||
### Case 8: Spec Mode — an unevaluable assertion makes the run NOT ASSESSED
|
||||
|
||||
**Fixture:**
|
||||
- `.claude/skills/brainstorm/SKILL.md` and its spec exist
|
||||
- Every assertion in the spec passes except one, which names a fixture file
|
||||
the spec never defines
|
||||
- The user approves the results write
|
||||
|
||||
**Input:** `/skill-test spec brainstorm`
|
||||
|
||||
**Expected behavior:**
|
||||
1. That assertion is NOT ASSESSED, with a Reason — never resolved to PASS
|
||||
because the skill "probably" handles it
|
||||
2. Its Case Verdict is NOT ASSESSED; with no PARTIAL or FAIL anywhere, the
|
||||
Overall Verdict is NOT ASSESSED
|
||||
3. On approval, `catalog.yaml` records `last_spec_result: NOT ASSESSED`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The unevaluable assertion is NOT ASSESSED, not PASS
|
||||
- [ ] Overall Verdict is NOT ASSESSED, not PASS
|
||||
- [ ] The catalog records `NOT ASSESSED` as the result
|
||||
|
||||
---
|
||||
|
||||
### Case 9: Category Mode — no rubric section for the category
|
||||
|
||||
**Fixture:**
|
||||
- `catalog.yaml` gives `some-skill` `category: tooling` — a category added to
|
||||
the catalog before its rubric section was written
|
||||
- `CCGS Skill Testing Framework/quality-rubric.md` has no `tooling` section
|
||||
|
||||
**Input:** `/skill-test category some-skill`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads the category, then looks for its rubric section and finds none
|
||||
2. The result is NOT ASSESSED, naming the missing `tooling` section
|
||||
3. No other category's metrics are borrowed to fill the gap — not even
|
||||
`utility`'s
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The result is NOT ASSESSED and names the missing rubric section
|
||||
- [ ] No metric from another category is evaluated in its place
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Static mode checks exactly 7 structural assertions
|
||||
- [ ] Spec mode evaluates each test case from the spec file individually
|
||||
- [ ] Spec mode evaluates each test case from the spec file individually, and a skill spec's Static Assertions against the SKILL.md
|
||||
- [ ] Each mode can report NOT ASSESSED — per skill in static mode, per assertion and overall in spec mode, and for a missing rubric section in category mode
|
||||
- [ ] Audit mode covers all skills AND agents (not just one category)
|
||||
- [ ] Spec and category modes accept an agent name as well as a skill name
|
||||
- [ ] Category mode reads quality-rubric.md to get criteria (not hardcoded)
|
||||
- [ ] Does not write any files in any mode
|
||||
- [ ] Suggests `/skill-improve` as the next step when issues are found
|
||||
- [ ] Static and audit modes write nothing; spec and category modes write only after "May I write" / "May I update" approval
|
||||
- [ ] After a spec FAIL, recommends reviewing the failing assertions and updating the skill or agent, or the spec
|
||||
|
||||
---
|
||||
|
||||
@@ -183,6 +326,10 @@ None. `/skill-test` is a meta-utility skill. No director gates apply.
|
||||
mode case for skill-test's own SKILL.md is not separately fixture-tested to
|
||||
avoid infinite recursion in test design.
|
||||
- The specific 7 structural checks are defined in the skill body; only Check 4
|
||||
(May I write) is individually tested here because it has the most nuanced logic.
|
||||
(ask-before-write) is individually tested here because it has the most
|
||||
nuanced logic. The single-skill NOT ASSESSED verdict follows Case 7's
|
||||
per-skill rule; not separately tested.
|
||||
- Audit mode counts are approximate — the exact number of skills and agents will
|
||||
change as the system grows; assertions use "all" rather than fixed counts.
|
||||
change as the system grows; the fixture's numbers are illustrative.
|
||||
- A spec case that names a director gate ID has the agent evaluated against
|
||||
`.claude/docs/director-gates/[gate-id].md`; not separately fixture-tested.
|
||||
|
||||
@@ -2,16 +2,35 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/smoke-check` is the gate between implementation and QA hand-off. It detects the
|
||||
test environment, runs the automated test suite (via Bash), scans test coverage
|
||||
against sprint stories, and uses `AskUserQuestion` to batch-verify manual smoke
|
||||
checks with the developer. It writes a report to `production/qa/smoke-[date].md`
|
||||
after explicit user approval.
|
||||
`/smoke-check` is the gate between implementation and QA hand-off. Phase 1 runs
|
||||
`project-coherence.sh`, verifies that **game test files** exist under the
|
||||
engine's test root or `tests/smoke/` (not merely that `tests/` exists), and
|
||||
reads the engine from `project.yaml` (falling back to `technical-preferences.md`),
|
||||
then resolves the editor executable — the one `commands.test` names, else
|
||||
`engine.path`, else the name on `PATH` — before it calls the runner unavailable.
|
||||
Phase 2 runs `commands.test` (or the per-engine default) under a timeout and
|
||||
reads the result by engine — gdUnit4 exit codes for Godot; on Unity, Play Mode
|
||||
tests under `Assets/Tests/PlayMode/` get a second run with their own results
|
||||
file. At `qa.level: minimal` with no game tests, the automated row is WAIVED
|
||||
and Phase 2 runs a build check instead — `commands.smoke`, else
|
||||
`commands.build`, when set. Phase 3 scans test coverage against the QA plan or
|
||||
sprint stories; Phase 4 batch-verifies manual smoke checks with
|
||||
`AskUserQuestion`, and Batch 1 first asks whether the build was launched this
|
||||
session — "not launched" makes the manual checks NOT ASSESSED. The report is written
|
||||
to `production/qa/smoke-[date].md` after explicit approval.
|
||||
|
||||
Verdicts: PASS (tests pass, all smoke checks pass, no missing test evidence),
|
||||
PASS WITH WARNINGS (tests pass or NOT RUN, all critical checks pass, but advisory
|
||||
gaps exist such as missing test coverage), or FAIL (any automated test failure or
|
||||
any Batch 1/Batch 2 smoke check returns FAIL).
|
||||
Verdicts, first matching rule wins: FAIL (a test failure, a suite that did not
|
||||
complete — timeout, compile/parse/build error, no results file — or a Batch
|
||||
1/Batch 2 check FAILED, or a failed build check), NOT ASSESSED (suite unconfirmed NOT RUN, a runner that
|
||||
could not run or executed zero tests, unconfirmed `PlayMode: NOT RUN`, a check
|
||||
that could not execute — including a build not launched this session — an
|
||||
`UNKNOWN` coverage row, Batch 3 skipped — or no game tests at `qa.level`
|
||||
`standard`/`full`; at `minimal` the automated row is WAIVED and the batches carry
|
||||
the verdict), PASS WITH WARNINGS (MISSING test evidence, a runner warning such
|
||||
as gdUnit4 101's leaked orphans, or a NOT RUN suite the developer confirmed),
|
||||
PASS. NOT ASSESSED ranks above both pass values and below FAIL.
|
||||
Whether a FAIL blocks hand-off depends on `qa.level` (advisory at `minimal`)
|
||||
and `testing.strict.config` (unset → blocking).
|
||||
|
||||
No director gates apply. The skill does NOT invoke any director agents.
|
||||
|
||||
@@ -19,13 +38,13 @@ No director gates apply. The skill does NOT invoke any director agents.
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keywords: PASS, PASS WITH WARNINGS, FAIL
|
||||
- [ ] Contains verdict keywords: PASS, PASS WITH WARNINGS, NOT ASSESSED, FAIL
|
||||
- [ ] Contains "May I write" collaborative protocol language before writing the report
|
||||
- [ ] Has a next-step handoff (e.g., `/bug-report` on FAIL, QA hand-off guidance on PASS)
|
||||
- [ ] Has a next-step handoff (re-run `/smoke-check` after a FAIL; QA hand-off to qa-tester on PASS)
|
||||
|
||||
---
|
||||
|
||||
@@ -40,29 +59,30 @@ None. `/smoke-check` is a pre-QA utility skill. No director gates apply.
|
||||
### Case 1: Happy Path — Automated tests pass, manual items confirmed, PASS
|
||||
|
||||
**Fixture:**
|
||||
- `tests/` directory exists with a GDUnit4 runner script
|
||||
- Engine detected as Godot from `technical-preferences.md`
|
||||
- `project.yaml` has `engine.name: Godot` and `commands.test` set to the gdUnit4 command
|
||||
- `addons/gdUnit4/bin/GdUnitCmdTool.gd` exists; game tests exist under
|
||||
`tests/unit/` and `tests/integration/`
|
||||
- `production/qa/qa-plan-sprint-005.md` exists
|
||||
- Automated test runner reports 12 tests, 12 passing, 0 failing
|
||||
- Developer confirms all Batch 1 and Batch 2 smoke checks as PASS
|
||||
- All sprint stories have matching test files (no MISSING coverage)
|
||||
- The runner exits 0 with 12 tests, 12 passing
|
||||
- The developer says the build was launched this session and selects no FAILED item in Batch 1, Batch 2 or Batch 3
|
||||
- Every sprint story is COVERED or EXPECTED (no MISSING, no UNKNOWN)
|
||||
|
||||
**Input:** `/smoke-check`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill detects test directory and engine, notes QA plan found
|
||||
2. Runs `godot --headless --script tests/gdunit4_runner.gd` via Bash
|
||||
3. Parses output: 12/12 passing
|
||||
4. Scans test coverage — all stories COVERED or EXPECTED
|
||||
5. Uses `AskUserQuestion` for Batch 1 (core stability) and Batch 2 (sprint mechanics)
|
||||
6. Developer selects PASS for all items
|
||||
7. Report assembled: automated tests PASS, all smoke checks PASS, no MISSING coverage
|
||||
8. Asks "May I write this smoke check report to `production/qa/smoke-[date].md`?"
|
||||
9. Writes report after approval
|
||||
10. Delivers verdict: PASS
|
||||
1. Phase 1 reports environment: Godot, tests found, QA plan path
|
||||
2. Runs `godot --headless --path . --import`, then `commands.test`
|
||||
(`godot --headless -s -d --remote-debug tcp://127.0.0.1:0 res://addons/gdUnit4/bin/GdUnitCmdTool.gd -a res://tests --ignoreHeadlessMode`)
|
||||
under `timeout 900`; the report says `commands.test` was the command run
|
||||
3. Exit 0 with tests found → automated tests PASS (12/12)
|
||||
4. Coverage scan: all stories COVERED or EXPECTED
|
||||
5. `AskUserQuestion` for Batch 1, Batch 2 and Batch 3 — nothing selected
|
||||
6. Report names its checklist source (the QA plan's Smoke Test Scope)
|
||||
7. Asks "May I write this smoke check report to `production/qa/smoke-[date].md`?"
|
||||
8. Writes after approval; delivers "Smoke check passed cleanly. The build is ready for manual QA."
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Automated test runner is invoked via Bash
|
||||
- [ ] Automated test runner is invoked via Bash, after the Godot import
|
||||
- [ ] `AskUserQuestion` is used for manual smoke check batches
|
||||
- [ ] "May I write" is asked before writing the report file
|
||||
- [ ] Report is written to `production/qa/smoke-[date].md`
|
||||
@@ -70,83 +90,115 @@ None. `/smoke-check` is a pre-QA utility skill. No director gates apply.
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Failure Path — Automated test fails, FAIL verdict
|
||||
### Case 2: Failure Path — Automated test fails, blocking FAIL
|
||||
|
||||
**Fixture:**
|
||||
- `tests/` directory exists, engine is Godot
|
||||
- Automated test runner reports 10 tests run: 8 passing, 2 failing
|
||||
- Failing tests: `test_health_clamp_at_zero`, `test_damage_calculation_negative`
|
||||
- QA plan exists
|
||||
- As Case 1, but `project.yaml` sets `modes.rigor: standard` (so `qa.level`
|
||||
resolves to `standard`) and leaves `testing.strict` unset
|
||||
- The runner exits 100: 10 tests run, 8 passing, 2 failing —
|
||||
`test_health_clamp_at_zero`, `test_damage_calculation_negative`
|
||||
|
||||
**Input:** `/smoke-check`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill runs automated tests via Bash
|
||||
2. Parses output — 2 failures detected
|
||||
3. Records failing test names
|
||||
4. Proceeds through manual smoke check batches
|
||||
5. Report shows automated tests as FAIL with failing test names listed
|
||||
6. Asks to write report; writes after approval
|
||||
7. Delivers FAIL verdict with message: "The smoke check failed. Do not hand off to
|
||||
QA until these failures are resolved." Lists failing tests and suggests fixing
|
||||
then re-running `/smoke-check`
|
||||
1. Skill runs automated tests via Bash; exit 100 → FAIL
|
||||
2. Records both failing test names
|
||||
3. Proceeds through the manual smoke check batches
|
||||
4. Report shows automated tests as FAIL with the failing test names listed
|
||||
5. Asks to write the report; writes after approval
|
||||
6. Gate is blocking (`qa.level: standard`, `testing.strict.config` unset → blocking)
|
||||
7. Delivers: "The smoke check failed. Do not hand off to QA until these failures
|
||||
are resolved: …" listing each failure, then "Fix the failures and run
|
||||
`/smoke-check` again to re-gate before QA hand-off."
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Failing test names are listed in the report
|
||||
- [ ] Verdict is FAIL
|
||||
- [ ] Post-verdict message directs developer to fix failures before QA hand-off
|
||||
- [ ] The blocking message directs the developer to fix failures before QA hand-off
|
||||
- [ ] `/smoke-check` re-run is suggested after fixing
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Manual Confirmation — AskUserQuestion used, PASS WITH WARNINGS
|
||||
### Case 3: Manual Confirmation — MISSING coverage, PASS WITH WARNINGS
|
||||
|
||||
**Fixture:**
|
||||
- `tests/` directory exists, engine is Godot
|
||||
- Automated test runner reports all tests passing (8/8)
|
||||
- One Logic story has no matching test file (MISSING coverage)
|
||||
- Developer confirms all Batch 1 and Batch 2 smoke checks as PASS
|
||||
- As Case 1: runner exits 0 with 8/8 passing
|
||||
- One Logic story has no matching test file (MISSING coverage); no UNKNOWN rows
|
||||
- The developer selects no FAILED item in Batch 1, Batch 2 or Batch 3
|
||||
|
||||
**Input:** `/smoke-check`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Automated tests PASS
|
||||
2. Coverage scan finds 1 MISSING entry for a Logic story
|
||||
3. `AskUserQuestion` is used for Batch 1 and Batch 2 — developer confirms all PASS
|
||||
4. Report shows: automated tests PASS, manual checks all PASS, 1 MISSING coverage entry
|
||||
5. Verdict is PASS WITH WARNINGS — build ready for QA, but MISSING entry must be
|
||||
resolved before `/story-done` closes the affected story
|
||||
6. Asks to write report; writes after approval
|
||||
3. `AskUserQuestion` is used for Batches 1–3 — nothing selected
|
||||
4. Report shows automated tests PASS, manual checks PASS, and the MISSING entry
|
||||
under "Missing Test Evidence" with its expected test path
|
||||
5. Verdict is PASS WITH WARNINGS; the message says the build is ready for manual
|
||||
QA and lists the MISSING entry as an advisory item to resolve before
|
||||
`/story-done` on the affected story
|
||||
6. Asks to write the report; writes after approval
|
||||
|
||||
**Assertions:**
|
||||
- [ ] `AskUserQuestion` is used for manual smoke check batches (not inline text prompts)
|
||||
- [ ] MISSING test coverage entry appears in the report
|
||||
- [ ] Verdict is PASS WITH WARNINGS (not PASS, not FAIL)
|
||||
- [ ] Advisory note explains MISSING entry must be resolved before `/story-done`
|
||||
- [ ] Advisory note says the MISSING entry must be resolved before `/story-done`
|
||||
- [ ] Report file is written to `production/qa/smoke-[date].md`
|
||||
|
||||
---
|
||||
|
||||
### Case 4: No Test Directory — Skill stops with guidance
|
||||
### Case 4: Scaffold Without Game Tests — NOT ASSESSED, then stop
|
||||
|
||||
**Fixture:**
|
||||
- `tests/` directory does not exist
|
||||
- Engine is configured as Godot
|
||||
- `project.yaml` sets `modes.rigor: standard` (so `qa.level` resolves to `standard`)
|
||||
- Engine is Godot; `tests/unit/` and `tests/integration/` exist but hold only
|
||||
the `.gdignore_placeholder` files `/test-setup` creates
|
||||
- No `tests/smoke/` directory
|
||||
|
||||
**Input:** `/smoke-check`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Phase 1 checks for `tests/` directory — not found
|
||||
2. Skill outputs: "No test directory found at `tests/`. Run `/test-setup` to
|
||||
scaffold the testing infrastructure, or create the directory manually if
|
||||
tests live elsewhere."
|
||||
3. Skill stops — no automated tests run, no manual smoke checks, no report written
|
||||
1. Phase 1 counts game test files, not directories — none are found
|
||||
2. Skill delivers: "Smoke check: **NOT ASSESSED — no game tests found** under
|
||||
[the test root] or `tests/smoke/`. Run `/test-setup` to scaffold the testing
|
||||
infrastructure, or point me at where tests live."
|
||||
3. Skill stops — no automated run, no manual batches, no report written
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Error message references the missing `tests/` directory
|
||||
- [ ] The presence of `tests/` and its subdirectories is not treated as tests existing
|
||||
- [ ] Verdict is NOT ASSESSED — the skill does not stop without a verdict
|
||||
- [ ] `/test-setup` is suggested as the remediation step
|
||||
- [ ] Skill stops after this message (no further phases run)
|
||||
- [ ] No report file is written
|
||||
- [ ] No further phases run and no report file is written
|
||||
|
||||
---
|
||||
|
||||
### Case 4b: No Game Tests at `qa.level: minimal` — WAIVED, the build carries the verdict
|
||||
|
||||
**Fixture:**
|
||||
- No `modes.rigor` set (the default, `minimal`, so `qa.level` resolves to `minimal`)
|
||||
- Engine is Godot; no game test files under `tests/unit/`, `tests/integration/` or `tests/smoke/`
|
||||
- `commands.smoke` is `godot --headless --quit-after 5`; it exits 0 with no `SCRIPT ERROR` line
|
||||
- The user says the build was launched this session and confirms every Batch 1, Batch 2 and Batch 3 check
|
||||
|
||||
**Input:** `/smoke-check`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Phase 1 finds no game test files and, at `qa.level: minimal`, records the automated row as `WAIVED — qa.level: minimal, no game tests`
|
||||
2. Phase 2's test run and Phase 3 are skipped, each with a one-line note saying why
|
||||
3. Phase 2's build check runs `commands.smoke` (after the Godot import) under a timeout: `Build check: PASS`
|
||||
4. Phase 4 runs Batches 1–3; every check passes
|
||||
5. Verdict: PASS — the report's Automated Tests status reads WAIVED
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The missing tests are WAIVED, not NOT ASSESSED, because `qa.level` is `minimal`
|
||||
- [ ] The skipped test run and Phase 3 are announced, not silent
|
||||
- [ ] The build check runs `commands.smoke` and its result is in the report
|
||||
- [ ] Batch 1 and Batch 2 still run — PASS is never given without them
|
||||
- [ ] Verdict is PASS and the report's Automated Tests row reads WAIVED
|
||||
- [ ] A Batch 1 or Batch 2 check that could not be executed (no build) makes the verdict NOT ASSESSED, as at any level
|
||||
- [ ] Variant — the user answers that the build was not launched this session: Batches 2 and 3 are not asked (the skip is said), the Batch 1 checks read `NOT RUN — build not launched this session`, and the verdict is NOT ASSESSED — never PASS from an empty failure list
|
||||
- [ ] Variant — `commands.smoke` prints `SCRIPT ERROR` (Godot's boot check exits 0 on it): the build check is FAIL and so is the verdict
|
||||
|
||||
---
|
||||
|
||||
@@ -158,7 +210,7 @@ None. `/smoke-check` is a pre-QA utility skill. No director gates apply.
|
||||
**Input:** `/smoke-check`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill runs all phases and produces a PASS or PASS WITH WARNINGS verdict
|
||||
1. Skill runs all phases and produces a verdict
|
||||
2. No director agents are spawned at any point
|
||||
3. No gate IDs (CD-*, TD-*, AD-*, PR-*) appear in output
|
||||
4. No `/gate-check` is invoked
|
||||
@@ -166,28 +218,134 @@ None. `/smoke-check` is a pre-QA utility skill. No director gates apply.
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Verdict is PASS, PASS WITH WARNINGS, or FAIL — no gate verdict involved
|
||||
- [ ] Verdict is PASS, PASS WITH WARNINGS, NOT ASSESSED, or FAIL — no gate verdict involved
|
||||
|
||||
---
|
||||
|
||||
### Case 6: gdUnit4 Lookalikes — exit 0 over zero tests, and exit 101
|
||||
|
||||
**Fixture:**
|
||||
- As Case 1, with every story COVERED or EXPECTED and nothing selected in
|
||||
Batch 1, Batch 2 or Batch 3
|
||||
- Run A: the suites under `tests/unit/` declare no test functions; the runner
|
||||
prints `No test cases found` and exits 0
|
||||
- Run B: the runner exits 101 — 12 tests, 12 passing, 2 orphan nodes leaked
|
||||
|
||||
**Input:** `/smoke-check`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Run A: exit 0 is not read as a pass — the automated tests are
|
||||
`NOT ASSESSED — no tests found`, and the verdict is NOT ASSESSED even though
|
||||
every manual check passed
|
||||
2. Run B: 101 is PASS WITH WARNINGS, naming the orphan count; with no MISSING
|
||||
entry, the verdict is PASS WITH WARNINGS and the advisory list names the
|
||||
orphans
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Run A's verdict is NOT ASSESSED — not PASS
|
||||
- [ ] Run B's Status line is PASS WITH WARNINGS with the orphan count
|
||||
- [ ] Run B's verdict is PASS WITH WARNINGS — not PASS, and not FAIL
|
||||
|
||||
---
|
||||
|
||||
### Case 7: Timeout — the suite never completed, FAIL
|
||||
|
||||
**Fixture:**
|
||||
- As Case 2 (`qa.level: standard`, `testing.strict` unset)
|
||||
- The runner hangs; `timeout 900` ends it with exit 124 and no summary printed
|
||||
|
||||
**Input:** `/smoke-check`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Exit 124 is a gate FAILURE, never a pass — the run did not complete and
|
||||
nothing was verified
|
||||
2. The Status line reads FAIL, naming the timeout
|
||||
3. The verdict is FAIL and the gate is blocking
|
||||
|
||||
**Assertions:**
|
||||
- [ ] A timeout is FAIL — not NOT ASSESSED and not NOT RUN
|
||||
- [ ] The report names the timeout as the reason
|
||||
- [ ] The blocking message is delivered
|
||||
|
||||
---
|
||||
|
||||
### Case 8: Batch 3 Skipped — NOT ASSESSED, not PASS
|
||||
|
||||
**Fixture:**
|
||||
- As Case 1: runner exits 0 with 12/12 passing, no MISSING or UNKNOWN rows
|
||||
- In Batch 3 the developer selects only "Performance not checked this session"
|
||||
|
||||
**Input:** `/smoke-check`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Batches 1 and 2 pass
|
||||
2. The skipped Batch 3 item is recorded as not checked — it is not a FAIL
|
||||
3. The verdict is NOT ASSESSED, and the Phase 6 message names the performance
|
||||
check that did not run
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Verdict is NOT ASSESSED — not PASS, not FAIL
|
||||
- [ ] The unchecked performance item is named in the report
|
||||
|
||||
---
|
||||
|
||||
### Case 9: Unity — Play Mode is a second run
|
||||
|
||||
**Fixture:**
|
||||
- `engine.name: Unity`; `com.unity.test-framework` is in `Packages/manifest.json`
|
||||
- `commands.test` is the Edit Mode command `/setup-engine` writes
|
||||
- `Assets/Tests/EditMode/` and `Assets/Tests/PlayMode/` both hold test files
|
||||
- Edit Mode exits 0 with 8/8 passing; Play Mode exits 2 with 1 of 3 failing
|
||||
|
||||
**Input:** `/smoke-check`
|
||||
|
||||
**Expected behavior:**
|
||||
1. The Edit Mode run deletes `test-results/editmode.xml` first, then runs
|
||||
2. Because `commands.test` does not pass `-testPlatform PlayMode` and
|
||||
`Assets/Tests/PlayMode/` holds tests, the skill runs Play Mode too, deleting
|
||||
and then writing `test-results/playmode.xml`
|
||||
3. The report has one Status line per platform: `EditMode: PASS (8/8)` and
|
||||
`PlayMode: FAIL (1 failure)`
|
||||
4. The verdict is FAIL
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Play Mode is run as a second run with its own results file
|
||||
- [ ] Each platform is reported on its own line
|
||||
- [ ] The Play Mode failure makes the verdict FAIL — the Edit Mode pass does not cover it
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Uses `AskUserQuestion` for all manual smoke check batches (Batch 1, Batch 2, Batch 3)
|
||||
- [ ] Uses `AskUserQuestion` for all manual smoke check batches (Batch 1, Batch 2, Batch 3), and Batch 1 asks whether the build was launched this session; "not launched" yields NOT ASSESSED
|
||||
- [ ] Resolves the editor executable from `commands.test`, then `engine.path`, then `PATH` before reporting the runner unavailable; a bare `godot` that does not resolve is never read as "no engine installed"
|
||||
- [ ] Never edits `project.yaml`: a `commands.test` missing flags or using a relative path is adjusted for the run and reported to the user
|
||||
- [ ] Runs automated tests via Bash before asking any manual questions
|
||||
- [ ] Asks "May I write" before creating the report file — never writes without approval
|
||||
- [ ] Verdict vocabulary is strictly PASS / PASS WITH WARNINGS / FAIL — no other verdicts
|
||||
- [ ] FAIL is triggered by automated test failures or Batch 1/Batch 2 FAIL responses
|
||||
- [ ] PASS WITH WARNINGS is triggered when MISSING test coverage exists but no critical failures
|
||||
- [ ] NOT RUN (engine binary unavailable) is recorded as a warning, not a FAIL
|
||||
- [ ] Verdict vocabulary is PASS / PASS WITH WARNINGS / NOT ASSESSED / FAIL — no other verdicts
|
||||
- [ ] FAIL is triggered by automated test failures, a suite that did not complete (timeout, compile/parse/build error, no results file), or Batch 1/Batch 2 FAIL responses
|
||||
- [ ] PASS WITH WARNINGS is triggered by MISSING test coverage, a runner warning on a passing suite (gdUnit4 101), or a confirmed NOT RUN — with no critical failures
|
||||
- [ ] Unconfirmed NOT RUN (runner not available) yields NOT ASSESSED — never FAIL, never a pass verdict
|
||||
- [ ] A run over zero tests, a runner that could not run, and an unconfirmed `PlayMode: NOT RUN` yield NOT ASSESSED
|
||||
- [ ] Does not invoke director gates at any point
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- The `quick` argument (skips Phase 3 coverage scan and Batch 3) is not separately
|
||||
fixture-tested; it follows the same pattern as Case 1 with a coverage-skip note in output.
|
||||
- The `--platform` argument adds platform-specific AskUserQuestion batches and a
|
||||
per-platform verdict table; not separately tested here.
|
||||
- The case where the engine binary is not on PATH (NOT RUN) follows the PASS WITH
|
||||
WARNINGS pattern and is covered by the protocol compliance assertions above.
|
||||
- The `quick` argument (skips the Phase 3 coverage scan and Batch 3) is not
|
||||
separately fixture-tested.
|
||||
- The `--platform` argument adds platform-specific `AskUserQuestion` batches and
|
||||
a per-platform verdict table; not separately tested here.
|
||||
- The runner-not-available path (NOT RUN, then NOT ASSESSED until the developer
|
||||
confirms a result from their IDE or CI) is covered by the protocol compliance
|
||||
assertions above.
|
||||
- At `qa.level: minimal` a FAIL is advisory and never blocks hand-off; Case 2
|
||||
pins `standard` to test the blocking path. Unity's `testcasecount="0"` and
|
||||
Unreal's `No automation tests matched` (→ NOT ASSESSED) follow Case 6 Run A;
|
||||
not separately fixture-tested.
|
||||
- The Linux and macOS Unreal commands (picked by `uname -s`) are not
|
||||
fixture-tested. On macOS, Unreal tests are NOT ASSESSED until `commands.test`
|
||||
is set — asserted in the skill, not given a case here.
|
||||
- A Play Mode run that cannot be made (`PlayMode: NOT RUN`, unconfirmed →
|
||||
NOT ASSESSED) is covered by protocol compliance, not by its own case.
|
||||
|
||||
@@ -2,29 +2,35 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/soak-test` generates a structured soak test protocol — an extended runtime
|
||||
test plan designed to surface memory leaks, performance drift, and stability
|
||||
issues that only appear under sustained gameplay. The skill produces a document
|
||||
specifying the test duration, system under test, monitoring checkpoints (e.g.,
|
||||
memory sample every 30 minutes), pass/fail thresholds, and conditions for early
|
||||
termination.
|
||||
`/soak-test [duration] [focus]` generates an observation protocol for an
|
||||
extended play session — the human plays, the skill writes the plan. Duration is
|
||||
`30m`, `1h`, `2h` or `4h` (default `1h`); focus is `memory`, `stability`,
|
||||
`balance` or `all` (default `all`). It reads `engine.name` and `performance.*`
|
||||
from `project.yaml` (falling back to `technical-preferences.md`), the game
|
||||
concept or brief, the latest playtest and QA plan, then builds timed
|
||||
checkpoints for the duration and engine-specific memory guidance whose alert
|
||||
thresholds are relative to the session's own T+0 baseline, in units recorded as
|
||||
displayed. The engine tool and counter names it gives are not covered by
|
||||
`docs/engine-reference/`, so they are marked NOT SOURCEABLE — pointers the
|
||||
tester confirms at T+0, recording the tool actually used.
|
||||
|
||||
The skill asks "May I write to `production/qa/soak-[slug]-[date].md`?" before
|
||||
persisting. If a previous soak test for the same system exists, the skill offers
|
||||
to extend the duration or add new conditions. No director gates apply. The verdict
|
||||
is COMPLETE when the soak test protocol is written.
|
||||
The protocol document carries a Verdict section with PASS / PASS WITH CONCERNS /
|
||||
NOT ASSESSED / FAIL for the tester to fill in after the session; a short soak
|
||||
cannot return PASS. The skill asks "May I write this soak test protocol to
|
||||
`production/qa/soak-test-[date]-[duration].md`?" before writing, and never runs
|
||||
the soak itself. No director gates apply.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keyword: COMPLETE
|
||||
- [ ] Contains verdict keywords: PASS, PASS WITH CONCERNS, NOT ASSESSED, FAIL
|
||||
- [ ] Contains "May I write" collaborative protocol language before writing the protocol
|
||||
- [ ] Has a next-step handoff (e.g., `/regression-suite` or `/release-checklist`)
|
||||
- [ ] Has a next-step handoff (`/bug-triage sprint` after the session; `/smoke-check` after fixing a FAIL)
|
||||
|
||||
---
|
||||
|
||||
@@ -36,113 +42,120 @@ None. `/soak-test` is a QA planning utility. No director gates apply.
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — Online gameplay feature, 2-hour soak protocol
|
||||
### Case 1: Happy Path — 2-hour, all-focus soak on Godot
|
||||
|
||||
**Fixture:**
|
||||
- User specifies: system = "online multiplayer lobby", duration = "2 hours"
|
||||
- `technical-preferences.md` has engine configured
|
||||
- `project.yaml` has `engine.name: Godot` and `performance.target_framerate: 60`
|
||||
- `design/gdd/game-concept.md` states the intended session length
|
||||
- The user approves the write
|
||||
|
||||
**Input:** `/soak-test online-lobby 2h`
|
||||
**Input:** `/soak-test 2h all`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill generates a 2-hour soak test protocol for the online lobby system
|
||||
2. Protocol includes: monitoring checkpoints every 30 minutes, metrics to track
|
||||
(memory usage, connection count, packet loss), pass thresholds, early termination
|
||||
conditions (crash or >20% memory growth)
|
||||
3. Networking-specific checks are included (session drop rate, reconnect handling)
|
||||
4. Skill asks "May I write to `production/qa/soak-online-lobby-2026-04-06.md`?"
|
||||
5. File is written on approval; verdict is COMPLETE
|
||||
1. Skill loads context: engine, performance budgets, concept, latest playtest and QA plan
|
||||
2. Checkpoints for `2h`: T+0, T+20, T+40, T+60, T+80, T+100, T+120
|
||||
3. Godot memory items: the Debugger's memory monitors, Static Memory (unit as
|
||||
displayed), Object Count, Orphan Nodes — names marked NOT SOURCEABLE, since
|
||||
`docs/engine-reference/godot/` does not cover them, for the tester to confirm
|
||||
at T+0; alert when memory grows > 20% from T+0 after the first 15 minutes;
|
||||
Orphan Nodes must return to its T+0 value after a scene unload
|
||||
4. Stability items (60 fps target) and balance/fatigue items are both included
|
||||
5. Protocol document has Pre-Session Setup, Baseline (T+0), a Checkpoint Log per
|
||||
checkpoint, Post-Session Analysis, and a Verdict section left for the tester
|
||||
6. Skill asks "May I write this soak test protocol to
|
||||
`production/qa/soak-test-[date]-2h.md`?"
|
||||
7. After writing, prints the run steps ending with `/bug-triage sprint`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Protocol duration matches the requested 2 hours
|
||||
- [ ] Monitoring checkpoints are at reasonable intervals (e.g., every 30 minutes)
|
||||
- [ ] Network-specific checks are included (not just generic memory checks)
|
||||
- [ ] "May I write" is asked with the correct file path
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] Checkpoints are exactly T+0, T+20, T+40, T+60, T+80, T+100, T+120
|
||||
- [ ] Memory thresholds are relative to T+0, and the unit is recorded as displayed (no unit assumed)
|
||||
- [ ] The Godot tool and counter names are marked NOT SOURCEABLE, not presented as verified, and Pre-Session Setup asks for the tool actually used
|
||||
- [ ] Memory, stability and balance observation items are all present (focus `all`)
|
||||
- [ ] "May I write" is asked with `production/qa/soak-test-[date]-2h.md`
|
||||
- [ ] The skill issues no verdict itself — the protocol's Verdict section is left for the tester
|
||||
|
||||
---
|
||||
|
||||
### Case 2: No Target Defined — Prompts for system, duration, and conditions
|
||||
### Case 2: No Arguments — Defaults applied
|
||||
|
||||
**Fixture:**
|
||||
- No arguments provided
|
||||
- No soak test config in session state
|
||||
- No arguments provided; engine configured
|
||||
|
||||
**Input:** `/soak-test`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill detects no target system or duration specified
|
||||
2. Skill asks: "What system or feature should be soak-tested?"
|
||||
3. After user responds with system: Skill asks: "What duration? (e.g., 1h, 4h, 8h)"
|
||||
4. After user responds with duration: Skill asks for specific conditions or
|
||||
uses defaults (normal gameplay loop, default player count)
|
||||
5. Skill generates protocol from collected inputs and asks "May I write"
|
||||
1. Duration defaults to `1h`, focus to `all` — no questions are needed to start
|
||||
2. Checkpoints: T+0, T+15, T+30, T+45, T+60
|
||||
3. Protocol header records `Duration: 1h` and `Focus: all`
|
||||
4. Skill asks "May I write this soak test protocol to `production/qa/soak-test-[date]-1h.md`?"
|
||||
|
||||
**Assertions:**
|
||||
- [ ] At minimum 2 follow-up questions are asked (system + duration)
|
||||
- [ ] Default conditions are applied when user doesn't specify custom ones
|
||||
- [ ] Protocol is not generated until system and duration are known
|
||||
- [ ] Verdict is COMPLETE after file is written
|
||||
- [ ] Duration `1h` and focus `all` are used when no argument is given
|
||||
- [ ] Checkpoints are exactly T+0, T+15, T+30, T+45, T+60
|
||||
- [ ] The output path ends `-1h.md`
|
||||
- [ ] Nothing is written before the "May I write" approval
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Previous Soak Test Exists — Offers to extend or add conditions
|
||||
### Case 3: Narrow Focus on Unreal — 4-hour memory soak
|
||||
|
||||
**Fixture:**
|
||||
- `production/qa/soak-online-lobby-2026-03-15.md` exists with a 1-hour protocol
|
||||
- User wants to extend to 4 hours with new memory threshold conditions
|
||||
- `project.yaml` has `engine.name: Unreal`
|
||||
|
||||
**Input:** `/soak-test online-lobby 4h`
|
||||
**Input:** `/soak-test 4h memory`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill finds existing soak test for online-lobby
|
||||
2. Skill reports: "Previous soak test found: soak-online-lobby-2026-03-15.md (1h)"
|
||||
3. Skill presents options: create new protocol (4h standalone), or extend the
|
||||
existing protocol to 4h and add new conditions
|
||||
4. User selects extend; existing checkpoints are preserved, new ones added
|
||||
5. Skill asks "May I write to `production/qa/soak-online-lobby-2026-04-06.md`?"
|
||||
(new file, not overwriting old one)
|
||||
1. Checkpoints for `4h`: T+0, T+30, T+60, T+90, T+120, T+180, T+240
|
||||
2. Unreal memory items: a memory readout at each checkpoint — `stat memory`,
|
||||
marked NOT SOURCEABLE since `docs/engine-reference/unreal/` does not cover
|
||||
it; record Physical Memory Used and Available (units as displayed)
|
||||
3. Alert threshold: Physical Memory Used growth > 20% over the full soak,
|
||||
against this run's T+0 — no absolute MB threshold
|
||||
4. Balance/fatigue observation items are not generated (focus is `memory`)
|
||||
5. Output path `production/qa/soak-test-[date]-4h.md`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Existing soak test is surfaced and referenced
|
||||
- [ ] User is offered extend vs. new options
|
||||
- [ ] New file is created (old file is not overwritten)
|
||||
- [ ] Extended protocol includes both old and new checkpoints
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] Checkpoints are exactly T+0, T+30, T+60, T+90, T+120, T+180, T+240
|
||||
- [ ] The Unreal threshold is a relative > 20% growth, not an absolute size
|
||||
- [ ] `stat memory` is marked NOT SOURCEABLE, a pointer to confirm rather than a verified command
|
||||
- [ ] Balance/fatigue items are omitted for focus `memory`
|
||||
- [ ] "May I write" names the `-4h.md` path
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Mobile Target Platform — Memory-specific checkpoints added
|
||||
### Case 4: No Performance Budgets — "not set", never invented
|
||||
|
||||
**Fixture:**
|
||||
- `technical-preferences.md` specifies target platform: Mobile
|
||||
- User requests soak test for "gameplay session" at 30 minutes
|
||||
- `project.yaml` has `engine.name: Unity` and no `performance` block
|
||||
- `technical-preferences.md` Performance Budgets are `[TO BE CONFIGURED]`
|
||||
|
||||
**Input:** `/soak-test gameplay 30m`
|
||||
**Input:** `/soak-test 30m memory`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads `technical-preferences.md` and detects mobile target platform
|
||||
2. Soak test protocol includes mobile-specific memory checkpoints:
|
||||
- Check heap memory growth vs. device baseline
|
||||
- Check texture memory at checkpoint intervals
|
||||
- Add warning threshold at 300MB (mobile ceiling)
|
||||
3. Protocol also includes thermal/battery drain advisory notes
|
||||
4. Skill asks "May I write?" and writes on approval; verdict is COMPLETE
|
||||
1. Skill falls back from `project.yaml` to `technical-preferences.md` for budgets
|
||||
2. Budget notes read `Memory ceiling: not set`, `Target FPS: not set`,
|
||||
`Frame budget: not set` — no number is supplied in their place
|
||||
3. Checkpoints: T+0, T+10, T+20, T+30
|
||||
4. Unity memory items: a memory profiler, marked NOT SOURCEABLE (not covered by
|
||||
`docs/engine-reference/unity/`); record Total Reserved Memory, GC Allocated,
|
||||
Object Count (units as displayed); alert when GC Allocated grows
|
||||
monotonically across 3+ checkpoints
|
||||
5. Skill asks "May I write" before writing `production/qa/soak-test-[date]-30m.md`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Mobile platform is detected from technical-preferences.md
|
||||
- [ ] Memory checkpoints include mobile-appropriate thresholds (not desktop)
|
||||
- [ ] Thermal/battery notes are present in the protocol
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] Missing budgets are recorded as "not set", not filled with defaults
|
||||
- [ ] Checkpoints are exactly T+0, T+10, T+20, T+30
|
||||
- [ ] The Unity threshold is the unit-free monotonic-growth check
|
||||
- [ ] "May I write" is asked before the file is written
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; soak-test is a planning utility
|
||||
|
||||
**Fixture:**
|
||||
- Valid system and duration provided
|
||||
- Engine configured; valid duration and focus provided
|
||||
|
||||
**Input:** `/soak-test combat 1h`
|
||||
**Input:** `/soak-test 1h stability`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill generates and writes the soak test protocol
|
||||
@@ -152,27 +165,27 @@ None. `/soak-test` is a QA planning utility. No director gates apply.
|
||||
**Assertions:**
|
||||
- [ ] No director gate is invoked
|
||||
- [ ] No gate skip messages appear
|
||||
- [ ] Skill reaches COMPLETE without any gate check
|
||||
- [ ] The run ends at "Protocol written." with the run steps — no gate verdict, and no soak verdict issued by the skill
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Collects system, duration, and conditions before generating protocol
|
||||
- [ ] Includes monitoring checkpoints at regular intervals
|
||||
- [ ] Includes pass/fail thresholds and early termination conditions
|
||||
- [ ] Adapts checkpoints to target platform (mobile vs. desktop)
|
||||
- [ ] Asks "May I write" before creating the protocol file
|
||||
- [ ] Verdict is COMPLETE when file is written
|
||||
- [ ] Parses `[duration] [focus]`, defaulting to `1h` and `all`
|
||||
- [ ] Uses the Phase 3 checkpoint table for the chosen duration
|
||||
- [ ] Memory alert thresholds are ratios or deltas against T+0; units are recorded as displayed
|
||||
- [ ] Engine tool names not covered by `docs/engine-reference/` are marked NOT SOURCEABLE; a memory tool the tester cannot find leaves the verdict NOT ASSESSED
|
||||
- [ ] The protocol's Verdict section offers PASS / PASS WITH CONCERNS / NOT ASSESSED / FAIL and states a short soak cannot return PASS
|
||||
- [ ] Asks "May I write" before creating `production/qa/soak-test-[date]-[duration].md`
|
||||
- [ ] Never attempts to run the soak itself
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- Soak tests for specific engine subsystems (rendering pipeline, physics
|
||||
simulation) follow the same protocol structure and are not separately tested.
|
||||
- The case where the user provides a duration shorter than the minimum useful
|
||||
soak period (e.g., 5 minutes) is not tested; the skill would note this is
|
||||
too short for meaningful results.
|
||||
- Automated execution of the soak test protocol is outside this skill's scope —
|
||||
this skill generates the plan, not the runner.
|
||||
- The `stability` and `balance` focus values follow Case 3's pattern with the
|
||||
other observation groups; not separately tested.
|
||||
- At `rigor: minimal` the skill reads `design/game-brief.md` instead of the
|
||||
game concept; not separately tested.
|
||||
- Executing the protocol is outside this skill's scope — it generates the plan,
|
||||
a human runs it.
|
||||
|
||||
@@ -2,29 +2,39 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/start` is the first-time onboarding skill for new projects. It guides the
|
||||
user through naming the project, choosing a game engine, and setting up the
|
||||
initial directory structure. It creates stub configuration files (CLAUDE.md,
|
||||
technical-preferences.md) and then routes to `/setup-engine` with the chosen
|
||||
engine as an argument. Each file or directory created is gated behind a
|
||||
"May I write" ask, following the collaborative protocol.
|
||||
`/start` is the first-time onboarding skill. Phase 1 silently detects project
|
||||
state (engine from `project.yaml` or `technical-preferences.md`, concept or
|
||||
brief, source code under the engine's code root, prototypes, GDDs, production
|
||||
artifacts, and whether `.git/config`'s `origin` still points at the template
|
||||
repo). Phase 2 asks where the user is with four `AskUserQuestion` options
|
||||
(A No idea yet, B Vague idea, C Clear concept, D Existing work); engine choice
|
||||
is not asked here — it is deferred to `/setup-engine`.
|
||||
|
||||
The skill detects whether a project is already configured and whether a
|
||||
partial setup exists, offering to resume or restart as appropriate. It has
|
||||
no director gates — it is a utility setup skill that runs before any agent
|
||||
hierarchy exists.
|
||||
Phase 3 routes by answer and names only the immediate next step. Phase 3c asks
|
||||
"May I write `project.stage: [stage]` to `project.yaml` (and the legacy mirror
|
||||
`production/stage.txt`)?" — one approval that also covers the Phase 3d and 3e
|
||||
answers — then writes both and announces them. Phase 3d asks the rigor question and Phase
|
||||
3e the automation question, each skipped when the key is already set; each
|
||||
answer is written to `project.yaml` straight from the selection. `/start` never
|
||||
writes any of the six knobs `modes.rigor` fronts (`modes.review_mode`,
|
||||
`modes.workflow`, `docs.density`, `qa.level`, `modes.story_granularity`,
|
||||
`team.size`) or `production/review-mode.txt`. Phase 4 prints the path for the
|
||||
chosen rigor — Path D2's retrofit path included — and asks which step to take
|
||||
first; Phase 5 prints a single "Type `[skill command]` to begin." line. A
|
||||
returning user (engine configured and a concept or brief present) skips
|
||||
onboarding. No director gates apply. Verdict: COMPLETE.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keywords: COMPLETE, BLOCKED
|
||||
- [ ] Contains "May I write" collaborative protocol language for each config file
|
||||
- [ ] Has a next-step handoff at the end (routes to `/setup-engine`)
|
||||
- [ ] Contains verdict keyword: COMPLETE
|
||||
- [ ] States the write rule for every write it makes: "May I write" before the `project.yaml` write, the same ask naming the `production/stage.txt` legacy mirror; `modes.rigor` and `modes.automation` written as a direct consequence of the user's selection, under that approval
|
||||
- [ ] Has a next-step handoff at the end (Phase 5: "Type `[skill command]` to begin.")
|
||||
|
||||
---
|
||||
|
||||
@@ -37,111 +47,143 @@ point this skill runs.
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — Fresh repo, no engine, full onboarding flow
|
||||
### Case 1: Happy Path — Fresh repo, Path A, minimal rigor
|
||||
|
||||
**Fixture:**
|
||||
- Empty repository: no CLAUDE.md overrides, no `production/stage.txt`, no
|
||||
`technical-preferences.md` content beyond placeholders
|
||||
- No existing design docs or source code
|
||||
- No `project.yaml`, no `production/stage.txt`; `technical-preferences.md`
|
||||
holds only placeholders
|
||||
- No design docs, prototypes or source code; `origin` is the user's own repo
|
||||
- The user picks `A) No idea yet`, approves the `project.yaml` write, then picks
|
||||
`Jam / prototype / first game`, then `Guided`
|
||||
|
||||
**Input:** `/start`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill detects no existing configuration and begins fresh onboarding
|
||||
2. Skill asks for project name
|
||||
3. Skill presents 3 engine options: Godot 4, Unity, Unreal Engine 5
|
||||
4. User selects an engine
|
||||
5. Skill asks "May I write the initial directory structure?"
|
||||
6. Skill creates all directories defined in `directory-structure.md`
|
||||
7. Skill asks "May I write CLAUDE.md stub?" and writes it on approval
|
||||
8. Skill routes to `/setup-engine [chosen-engine]` to complete technical config
|
||||
1. Phase 1 detection runs without printing its findings
|
||||
2. Phase 2 `AskUserQuestion` offers exactly A–D; no engine question is asked
|
||||
3. Path A: explains `/brainstorm` tier-neutrally and names only `/brainstorm open`
|
||||
as the next step ("I'll lay out the full path once I know how much process you want")
|
||||
4. Phase 3c asks "May I write `project.stage: Concept` to `project.yaml` (and the
|
||||
legacy mirror `production/stage.txt`)?",
|
||||
saying the file will be created; on approval it creates `project.yaml` from
|
||||
the v1.1 minimal template with `project: stage: Concept` (no `modes.rigor`,
|
||||
no `modes.review_mode`), writes `Concept` to `production/stage.txt` under
|
||||
the same approval, and announces both
|
||||
5. Phase 3d offers `Jam / prototype / first game (Recommended)` first (no
|
||||
concept → minimal); `modes.rigor: minimal` is written on selection
|
||||
6. Phase 3e writes `modes.automation: guided` on selection
|
||||
7. Phase 4 prints the minimal 4-step path (`/setup-engine`, `/brainstorm`,
|
||||
`/create-stories`, `/dev-story`) and asks which step to take first
|
||||
8. Phase 5 prints only "Type `[skill command]` to begin."; verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Project name is captured before any file is written
|
||||
- [ ] Exactly 3 engine options are presented
|
||||
- [ ] "May I write" is asked for each config file individually
|
||||
- [ ] No file is written without explicit user approval
|
||||
- [ ] Handoff to `/setup-engine` occurs at the end with the chosen engine argument
|
||||
- [ ] Verdict is COMPLETE after all files are written and handoff is issued
|
||||
- [ ] Phase 2 offers exactly the four A–D options and no engine choice
|
||||
- [ ] `project.yaml` and `production/stage.txt` are written only after the one "May I write" ask that names both
|
||||
- [ ] `project.stage` and `production/stage.txt` are both `Concept`
|
||||
- [ ] `project.yaml` ends up holding only `project.stage`, `modes.rigor` and `modes.automation` from `/start` — none of the six rigor-fronted knobs — and `production/review-mode.txt` is not written
|
||||
- [ ] The recommended rigor option is listed first with ` (Recommended)`
|
||||
- [ ] Phase 4 prints the minimal 4-step path
|
||||
- [ ] The next skill is not auto-run; Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Already Configured — Detects existing config, offers to skip or reconfigure
|
||||
### Case 2: Returning User — Onboarding skipped
|
||||
|
||||
**Fixture:**
|
||||
- `technical-preferences.md` has engine already set (not placeholder)
|
||||
- `production/stage.txt` exists with `Concept`
|
||||
- `project.yaml` has `engine.name: Godot` and `modes.rigor: minimal`; no explicit `modes.review_mode`
|
||||
- `design/game-brief.md` exists
|
||||
|
||||
**Input:** `/start`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads `technical-preferences.md` and detects configured engine
|
||||
2. Skill reports: "This project is already configured with [engine]"
|
||||
3. Skill presents options: skip (exit), reconfigure engine, or reconfigure specific sections
|
||||
4. If user selects skip: skill exits cleanly with a summary of current config
|
||||
5. If user selects reconfigure: skill proceeds to the engine-selection step
|
||||
1. Phase 1 finds the engine configured and the brief present
|
||||
2. Skill skips onboarding: "It looks like you're already set up! Your engine is
|
||||
Godot and you have … a game brief at `design/game-brief.md`."
|
||||
3. Review mode is reported as `solo` (resolved from `modes.rigor: minimal`)
|
||||
4. Skill points to `/help` or asks what the user would like to work on
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Skill does NOT overwrite existing config without user choosing reconfigure
|
||||
- [ ] Detected engine name is shown to the user in the status message
|
||||
- [ ] User is offered at least 2 options (skip or reconfigure)
|
||||
- [ ] Verdict is COMPLETE whether user skips or reconfigures
|
||||
- [ ] The A–D starting-point question is not asked
|
||||
- [ ] The message names the configured engine (Godot) and the brief path
|
||||
- [ ] Review mode is reported as `solo`, resolved from rigor
|
||||
- [ ] No write is made to `project.yaml` or `production/stage.txt`
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Engine Choice — User picks Godot 4, routes to /setup-engine godot
|
||||
### Case 3: Existing Work on Unity — Path D2, stage and rigor already set
|
||||
|
||||
**Fixture:**
|
||||
- Fresh repo — no existing configuration
|
||||
- `project.yaml` has `engine.name: Unity`, `modes.rigor: standard` and
|
||||
`modes.automation: collaborative`
|
||||
- C# scripts exist under `Assets/Scripts/`; `design/gdd/` holds 3 system GDDs;
|
||||
no `design/gdd/game-concept.md`, no `docs/architecture/`
|
||||
- The user picks `D) Existing work` and approves the `project.yaml` write
|
||||
|
||||
**Input:** `/start`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill presents engine options and user selects Godot 4
|
||||
2. Skill writes initial stubs (directory structure, CLAUDE.md) after approval
|
||||
3. Skill explicitly routes to `/setup-engine godot` as the next step
|
||||
4. Handoff message clearly names the engine and the next skill invocation
|
||||
1. Phase 1 resolves the code root as `Assets/` for Unity and finds source files
|
||||
there (it does not scan `src/` and conclude there is no code)
|
||||
2. Phase 3 D shares what it found (source files, 3 design docs, engine Unity)
|
||||
3. Sub-case D2: recommends `/project-stage-detect` then `/adopt`, naming
|
||||
`/project-stage-detect` as the immediate step; the tiered path waits for
|
||||
Phase 4
|
||||
4. Phase 3c asks "May I write `project.stage: Systems Design` to
|
||||
`project.yaml` (and the legacy mirror `production/stage.txt`)?", then writes `Systems Design` (GDDs, no architecture) to the
|
||||
existing `project.yaml` (read first, then edited) and to `production/stage.txt`
|
||||
5. Phase 3d prints "Rigor is set to `standard`." and Phase 3e "Automation is set
|
||||
to `collaborative`." — neither question is asked
|
||||
6. Phase 4 prints the D2 retrofit path at `standard`: `/project-stage-detect`,
|
||||
`/adopt`, `/design-system retrofit`, `/architecture-decision retrofit`,
|
||||
`/architecture-review`, `/gate-check`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Handoff command is `/setup-engine godot` (not generic `/setup-engine`)
|
||||
- [ ] Handoff is issued after all initial stubs are written, not before
|
||||
- [ ] Engine choice is echoed back to user before writing begins
|
||||
- [ ] Source files are found under `Assets/`, not reported as absent
|
||||
- [ ] `/project-stage-detect` and `/adopt` are recommended for D2
|
||||
- [ ] Stage `Systems Design` is written to both `project.yaml` and `production/stage.txt`, after the "May I write" ask
|
||||
- [ ] The rigor and automation questions are skipped because both keys are set
|
||||
- [ ] The D2 retrofit path is printed in Phase 4, including the GDD, ADR and registry retrofit steps
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Interrupted Setup — Partial config detected, offers resume or restart
|
||||
### Case 4: Clone Still Wired to the Template Repo — remote note, never run
|
||||
|
||||
**Fixture:**
|
||||
- Directory structure exists (was created) but `technical-preferences.md` is
|
||||
still all placeholders (engine was never chosen — setup was interrupted)
|
||||
- No `production/stage.txt`
|
||||
- Fresh project as in Case 1
|
||||
- `.git/config` has `[remote "origin"]` with url
|
||||
`https://github.com/Donchitos/Claude-Code-Game-Studios.git`
|
||||
- The user picks `B) Vague idea` with the hint "a small weekend puzzler"
|
||||
|
||||
**Input:** `/start`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill detects partial state: directories exist but engine is unconfigured
|
||||
2. Skill reports: "A partial setup was detected — directories exist but engine is not configured"
|
||||
3. Skill offers: resume from engine selection, or restart from scratch
|
||||
4. If resume: skill skips directory creation, proceeds to engine choice
|
||||
5. If restart: skill asks "May I overwrite existing structure?" before proceeding
|
||||
1. Phase 1 reads `.git/config` as a file and matches the template URL case-insensitively
|
||||
2. Path B recommends `/brainstorm a small weekend puzzler` as the immediate step
|
||||
3. Phase 3d recommends `Jam / prototype / first game` for the described scope
|
||||
4. After the Phase 4 path, the skill adds once: the clone's `origin` still points
|
||||
at the Claude Code Game Studios repo, and suggests
|
||||
`git remote rename origin template` then `git remote add origin <your repo URL>`
|
||||
5. The commands are suggested only — no git command is run
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Partial state is correctly identified (directories present, engine absent)
|
||||
- [ ] User is offered resume vs. restart choice — not forced into one path
|
||||
- [ ] Resume path skips re-creating directories (no redundant "May I write" for structure)
|
||||
- [ ] Restart path asks for permission to overwrite before touching any files
|
||||
- [ ] The template-origin note appears exactly once, after the path
|
||||
- [ ] Both suggested commands are given as written above
|
||||
- [ ] No git command is executed (read of `.git/config` only)
|
||||
- [ ] The minimal rigor option is the recommended one for the described scope
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; start is a utility setup skill
|
||||
|
||||
**Fixture:**
|
||||
- Any fixture
|
||||
- Fresh project and answers as in Case 1 (`A) No idea yet`, the write approved,
|
||||
`Jam / prototype / first game`, `Guided`)
|
||||
|
||||
**Input:** `/start`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill completes full onboarding flow
|
||||
1. Skill completes the onboarding flow through Phase 5
|
||||
2. No director agents are spawned at any point
|
||||
3. No gate IDs (CD-*, TD-*, AD-*, PR-*) appear in the output
|
||||
|
||||
@@ -152,22 +194,56 @@ point this skill runs.
|
||||
|
||||
---
|
||||
|
||||
### Case 6: Path D2 With Rigor Unset — the retrofit path waits for the answer
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` has `engine.name: Godot` and no `modes` block
|
||||
- `src/` holds GDScript files; `design/gdd/` holds 4 system GDDs; no
|
||||
`docs/architecture/`
|
||||
- The user picks `D) Existing work`, approves the `project.yaml` write, then
|
||||
picks `Several systems that affect each other` and `Collaborative`
|
||||
|
||||
**Input:** `/start`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Phase 3 D2 names `/project-stage-detect` as the immediate step and prints no
|
||||
tiered path yet
|
||||
2. Phase 3c asks, then writes `Systems Design`
|
||||
3. Phase 3d asks the rigor question, since `modes.rigor` is unset;
|
||||
`modes.rigor: standard` is written on selection
|
||||
4. Phase 4 prints the D2 retrofit path at `standard`, including
|
||||
`/design-system retrofit`, `/architecture-decision retrofit` and
|
||||
`/architecture-review` — not the minimal stop after `/adopt` and
|
||||
`/setup-engine`
|
||||
|
||||
**Assertions:**
|
||||
- [ ] No tiered D2 path is printed before Phase 3d has the rigor
|
||||
- [ ] The rigor question is asked because the key is unset
|
||||
- [ ] Phase 4's D2 path matches the chosen `standard`, with the GDD, ADR and registry retrofit steps
|
||||
- [ ] None of the six rigor-fronted knobs is written — only `modes.rigor` and `modes.automation` are added
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Asks for project name before any file is written
|
||||
- [ ] Presents engine options as a structured choice (not free text)
|
||||
- [ ] Asks "May I write" separately for directory structure and for CLAUDE.md stub
|
||||
- [ ] Ends with a handoff to `/setup-engine` with the engine name as argument
|
||||
- [ ] Verdict is clearly stated (COMPLETE or BLOCKED) at end of output
|
||||
- [ ] Detects project state silently before the first question
|
||||
- [ ] Asks the starting-point question via `AskUserQuestion` with options A–D; defers engine choice to `/setup-engine`
|
||||
- [ ] Never writes any of the six rigor-fronted knobs (`modes.review_mode`, `modes.workflow`, `docs.density`, `qa.level`, `modes.story_granularity`, `team.size`) or `production/review-mode.txt`
|
||||
- [ ] Skips the rigor and automation questions when those keys are already set
|
||||
- [ ] Asks "May I write" before the `project.yaml` write, naming `production/stage.txt` in the same ask; announces the stage it wrote; writes rigor/automation only from the user's selection
|
||||
- [ ] Prints the Path D2 retrofit path only in Phase 4, after the rigor is known
|
||||
- [ ] Never auto-runs the next skill; a completed onboarding run (not the returning-user skip, which ends at its `/help` pointer) ends with "Type `[skill command]` to begin." and Verdict COMPLETE
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- The case where the user rejects all engine options and provides a custom
|
||||
engine name is not tested — the skill is designed for the three supported
|
||||
engines only.
|
||||
- Git initialization (if any) is not tested here; that is an infrastructure
|
||||
concern outside the skill boundary.
|
||||
- Solo vs. lean mode behavior is not applicable — this skill has no gates and
|
||||
mode selection is irrelevant.
|
||||
- Path C (clear concept, `Formalize it first` vs `Jump straight in`) is not
|
||||
separately tested.
|
||||
- The `standard` and `full` Phase 4 paths follow Case 1 with the longer
|
||||
pipeline; not separately tested.
|
||||
- The mismatch note (a rigor that contradicts the described project) and the
|
||||
edge cases "picks D but project is empty" / "picks A but code exists" are not
|
||||
separately tested.
|
||||
- A declined Phase 3c write (neither file written, the stage and the two
|
||||
answers kept for the run and reported as not saved) is not separately tested.
|
||||
|
||||
@@ -2,27 +2,37 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/test-helpers` generates engine-specific test helper utilities for the project's
|
||||
test suite. Helpers include factory functions (for creating test entities with
|
||||
known state), fixture loaders, assertion helpers, and mock stubs for external
|
||||
dependencies. Generated helpers follow the naming and structure conventions in
|
||||
`coding-standards.md` and are written to `tests/helpers/`.
|
||||
`/test-helpers [system-name | all | scaffold]` generates test helper files in
|
||||
the engine's **helper root** — the only place its test runner compiles them:
|
||||
`tests/helpers/` on Godot, `Assets/Tests/EditMode/Helpers/` on Unity (inside the
|
||||
`EditModeTests` assembly), `Source/<Module>/Private/Tests/Helpers/` on Unreal. It
|
||||
writes base assertion and factory helpers (`scaffold`), and a
|
||||
`[helper root]/[system]_factory.[ext]` per system built from that system's GDD
|
||||
(`[system-name]` or `all`). No argument runs `scaffold` when no helpers exist,
|
||||
else `all`. It reads `engine.name`, `engine.language` and `testing.framework`
|
||||
from `project.yaml` (falling back to `technical-preferences.md`), samples up to
|
||||
5 existing test files for house style, and section-greps the GDD (Formulas,
|
||||
Edge Cases, Detailed Rules/Design) rather than reading it whole.
|
||||
|
||||
Each helper file is gated behind a "May I write" ask. If a helper file already
|
||||
exists, the skill offers to extend it rather than replace. No director gates
|
||||
apply. The verdict is COMPLETE when helper files are written.
|
||||
Helpers must assert through the configured framework's API — never bare
|
||||
`assert()` — and must not extend the framework's test-suite base class. If the
|
||||
framework's assertion API cannot be confirmed, the skill says so and generates
|
||||
no helper. It never overwrites an existing helper: it skips it with a message.
|
||||
After presenting the file list it asks "May I write these helper files to
|
||||
`[helper root]`?". No director gates apply. Verdict: COMPLETE when helper files
|
||||
are created.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keyword: COMPLETE
|
||||
- [ ] Contains "May I write" collaborative protocol language before writing helpers
|
||||
- [ ] Has a next-step handoff (e.g., write a test using the generated helper)
|
||||
- [ ] Has a next-step handoff (Next Steps: `/test-setup`, `/dev-story`, `/skill-test`)
|
||||
|
||||
---
|
||||
|
||||
@@ -34,113 +44,119 @@ None. `/test-helpers` is a scaffolding utility. No director gates apply.
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — Player factory helper generated for Godot/GDScript
|
||||
### Case 1: Happy Path — Player helper generated for Godot/GDScript
|
||||
|
||||
**Fixture:**
|
||||
- `technical-preferences.md` has engine Godot 4, language GDScript
|
||||
- `tests/` directory exists (test-setup has been run)
|
||||
- `design/gdd/player.md` exists with defined player properties
|
||||
- No existing helpers in `tests/helpers/`
|
||||
- `project.yaml` has `engine.name: Godot`, `engine.language: GDScript`,
|
||||
`testing.framework: gdUnit4`; `addons/gdUnit4/` is installed, so its
|
||||
assertion API can be confirmed from the project
|
||||
- `tests/unit/player/player_movement_test.gd` exists
|
||||
- `design/gdd/player.md` has a Formulas section bounding health to 0–100
|
||||
- No existing files in `tests/helpers/`; the user approves the write
|
||||
|
||||
**Input:** `/test-helpers player-factory`
|
||||
**Input:** `/test-helpers player`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads engine (Godot 4 / GDScript) and player GDD for property context
|
||||
2. Skill generates a deterministic `PlayerFactory` helper in GDScript:
|
||||
- `create_player(health: int = 100, speed: float = 200.0)` function
|
||||
- Returns a player node pre-configured to a known state
|
||||
- Uses dependency injection (no singletons)
|
||||
3. Skill asks "May I write to `tests/helpers/player_factory.gd`?"
|
||||
4. File is written on approval; verdict is COMPLETE
|
||||
1. Skill reads engine, language and framework; samples the existing test file
|
||||
2. Skill greps `design/gdd/player.md` for the Formulas / Edge Cases / Detailed
|
||||
Rules headings instead of reading it whole
|
||||
3. Skill drafts `tests/helpers/player_factory.gd`: a `class_name` helper that
|
||||
`extends RefCounted`, static `make_*` factories with default parameters,
|
||||
bound constants commented as coming from the GDD, and a header line
|
||||
"Based on: design/gdd/player.md"
|
||||
4. Every assertion goes through gdUnit4's confirmed API — no bare `assert()`,
|
||||
no `FAIL_IF` placeholder
|
||||
5. Skill presents the file list and asks "May I write these helper files to `tests/helpers/`?"
|
||||
6. Files written on approval; verdict COMPLETE, followed by the per-engine usage note
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Generated helper is in GDScript (not C# or Blueprint)
|
||||
- [ ] Factory function parameters use defaults matching GDD values
|
||||
- [ ] Helper uses dependency injection (no Autoload/singleton references)
|
||||
- [ ] Filename follows snake_case convention for GDScript
|
||||
- [ ] Generated helper is GDScript at `tests/helpers/player_factory.gd`
|
||||
- [ ] Bound constants trace to the GDD's Formulas section (not invented values)
|
||||
- [ ] Helper extends `RefCounted`, not the test-suite base class, and uses no Autoload/singleton
|
||||
- [ ] No bare `assert()` and no unresolved `FAIL_IF` reaches the file
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 2: No Test Setup Exists — Redirects to /test-setup
|
||||
### Case 2: Engine Not Configured — Stops with /setup-engine
|
||||
|
||||
**Fixture:**
|
||||
- `tests/` directory does not exist
|
||||
- `project.yaml` has no `engine` block; `technical-preferences.md` shows
|
||||
`[TO BE CONFIGURED]` for Engine
|
||||
|
||||
**Input:** `/test-helpers player-factory`
|
||||
**Input:** `/test-helpers player`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill checks for `tests/` directory — not found
|
||||
2. Skill reports: "Test directory not found — test framework must be set up first"
|
||||
3. Skill suggests running `/test-setup` before generating helpers
|
||||
4. No helper file is created
|
||||
1. Skill checks `project.yaml`, then `technical-preferences.md` — no engine in either
|
||||
2. Skill outputs: "Engine not configured. Run `/setup-engine` first."
|
||||
3. Skill stops — no test scan, no draft, no write
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Error message identifies the missing tests/ directory
|
||||
- [ ] `/test-setup` is suggested as the prerequisite step
|
||||
- [ ] The message states the engine is not configured
|
||||
- [ ] `/setup-engine` is named as the prerequisite
|
||||
- [ ] No write tool is called
|
||||
- [ ] Verdict is not COMPLETE (blocked state)
|
||||
- [ ] Verdict is not COMPLETE (nothing was created)
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Helper Already Exists — Offers to extend rather than replace
|
||||
### Case 3: Helper Already Exists — Skipped, never overwritten
|
||||
|
||||
**Fixture:**
|
||||
- `tests/helpers/player_factory.gd` already exists with a `create_player()` function
|
||||
- User requests a new `create_enemy()` function be added to the factory
|
||||
- Godot/GDScript/gdUnit4 configured as in Case 1
|
||||
- `tests/helpers/game_assertions.gd` already exists with hand-written additions
|
||||
- `tests/helpers/game_factory.gd` and `tests/helpers/scene_runner_helper.gd` do
|
||||
not exist; the user approves the write
|
||||
|
||||
**Input:** `/test-helpers enemy-factory`
|
||||
**Input:** `/test-helpers scaffold`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill finds an existing `player_factory.gd` and checks if it's the right file
|
||||
to extend (or if a separate `enemy_factory.gd` should be created)
|
||||
2. Skill presents options: add `create_enemy()` to existing factory or create
|
||||
`tests/helpers/enemy_factory.gd`
|
||||
3. User selects extend; skill drafts the `create_enemy()` function
|
||||
4. Skill asks "May I extend `tests/helpers/player_factory.gd`?"
|
||||
5. Function is added on approval; verdict is COMPLETE
|
||||
1. Skill drafts the Godot base helpers only — `game_assertions.gd`,
|
||||
`game_factory.gd` and `scene_runner_helper.gd`; no system-specific helpers in
|
||||
`scaffold` mode
|
||||
2. Skill asks "May I write these helper files to `tests/helpers/`?"
|
||||
3. The two missing files, `tests/helpers/game_factory.gd` and
|
||||
`tests/helpers/scene_runner_helper.gd`, are created
|
||||
4. For the existing file, skill reports: "Skipping `tests/helpers/game_assertions.gd`
|
||||
— already exists. Remove the file manually if you want it regenerated."
|
||||
5. Verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Existing helper is detected and surfaced
|
||||
- [ ] User is given extend vs. new file choice
|
||||
- [ ] "May I extend" language is used (not "May I write" for replacement)
|
||||
- [ ] Existing `create_player()` is preserved in the extended file
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] `scaffold` mode generates no `[system]_factory` helper
|
||||
- [ ] The existing `game_assertions.gd` is left byte-for-byte unchanged
|
||||
- [ ] The skip message names the existing file and how to regenerate it
|
||||
- [ ] Only the two missing files are written; Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 4: System Has No GDD — Notes missing design context in helper
|
||||
### Case 4: Framework Assertion API Unconfirmed — No helper generated
|
||||
|
||||
**Fixture:**
|
||||
- `technical-preferences.md` has Godot 4 / GDScript
|
||||
- `tests/` exists
|
||||
- User requests a helper for the "inventory system" but no `design/gdd/inventory.md` exists
|
||||
- `project.yaml` has `engine.name: Godot`, `engine.language: GDScript`;
|
||||
`testing.framework` is absent and `technical-preferences.md` shows no Framework
|
||||
- No `addons/gdUnit4/` and no existing test files to learn the API from
|
||||
|
||||
**Input:** `/test-helpers inventory-factory`
|
||||
**Input:** `/test-helpers scaffold`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill looks for `design/gdd/inventory.md` — not found
|
||||
2. Skill notes: "No GDD found for inventory — generating helper with placeholder defaults"
|
||||
3. Skill generates an `inventory_factory.gd` with generic placeholder values
|
||||
(item_count = 0, max_capacity = 20) and a comment: "# TODO: align defaults
|
||||
with inventory GDD when written"
|
||||
4. Skill asks "May I write to `tests/helpers/inventory_factory.gd`?"
|
||||
5. File is written; verdict is COMPLETE with advisory note
|
||||
1. Skill cannot confirm which assertion call registers a failure with the runner
|
||||
2. Skill says so, and generates no helper rather than guessing a form such as
|
||||
`assert_that`, `assert_eq`, `assert_true` or a bare `assert()`
|
||||
3. No "May I write" is asked and nothing is written
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Skill proceeds without GDD (does not block)
|
||||
- [ ] Generated helper has placeholder defaults with TODO comment
|
||||
- [ ] Missing GDD is noted in the output (advisory warning)
|
||||
- [ ] Verdict is COMPLETE
|
||||
- [ ] The skill states it cannot confirm the framework's assertion API
|
||||
- [ ] No helper file is written
|
||||
- [ ] No guessed assertion form (and no `FAIL_IF` placeholder) is presented as ready to write
|
||||
- [ ] Verdict is not COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Director Gate Check — No gate; test-helpers is a scaffolding utility
|
||||
|
||||
**Fixture:**
|
||||
- Engine configured, tests/ exists
|
||||
- Engine and framework configured as in Case 1, no existing helpers
|
||||
|
||||
**Input:** `/test-helpers player-factory`
|
||||
**Input:** `/test-helpers player`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill generates and writes the helper file
|
||||
@@ -154,22 +170,55 @@ None. `/test-helpers` is a scaffolding utility. No director gates apply.
|
||||
|
||||
---
|
||||
|
||||
### Case 6: Unity — Helpers under the engine's test root
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml` has `engine.name: Unity`, `engine.language: C#`,
|
||||
`testing.framework: NUnit`; `com.unity.test-framework` is in
|
||||
`Packages/manifest.json`, so NUnit's assertion API can be confirmed from the
|
||||
installed package
|
||||
- `/test-setup` has created `Assets/Tests/EditMode/EditModeTests.asmdef`
|
||||
- No helpers exist yet; the user approves the write
|
||||
|
||||
**Input:** `/test-helpers scaffold`
|
||||
|
||||
**Expected behavior:**
|
||||
1. The helper root is `Assets/Tests/EditMode/Helpers/` — inside the
|
||||
`EditModeTests` assembly, where Unity compiles it
|
||||
2. Skill drafts `GameAssertions.cs` and `GameFactory.cs` for that folder,
|
||||
asserting through NUnit's `Assert`
|
||||
3. Skill asks "May I write these helper files to `Assets/Tests/EditMode/Helpers/`?"
|
||||
4. Files written on approval; the usage note says to reference the test assembly;
|
||||
verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Helpers are written under `Assets/Tests/EditMode/Helpers/`
|
||||
- [ ] Nothing is written under `tests/helpers/` — outside `Assets/`, Unity never compiles it
|
||||
- [ ] The "May I write" ask names the Unity helper root
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Reads engine before generating any helper (helpers are engine-specific)
|
||||
- [ ] Reads GDD for default values when available
|
||||
- [ ] Notes missing GDD context rather than blocking
|
||||
- [ ] Detects existing helper files and offers extend rather than replace
|
||||
- [ ] Asks "May I write" (or "May I extend") before any file operation
|
||||
- [ ] Verdict is COMPLETE when helper is written
|
||||
- [ ] Reads engine, language and test framework before generating any helper
|
||||
- [ ] Writes helpers only to the engine's helper root (`tests/helpers/`, `Assets/Tests/EditMode/Helpers/` or `Source/<Module>/Private/Tests/Helpers/`)
|
||||
- [ ] Section-greps the GDD for system helpers (numbered headings such as `## 4. Formulas` included); constants trace to its Formulas section
|
||||
- [ ] Godot signal asserts keep their flag in a Dictionary the lambda mutates and connect a variadic lambda (`func(...args)`) — a captured `bool` never changes in the caller, and a one-argument lambda fails on any other signal
|
||||
- [ ] Asserts only through the configured framework — never bare `assert()` — and generates nothing when that API cannot be confirmed
|
||||
- [ ] Helpers never extend the framework's test-suite base class
|
||||
- [ ] Never overwrites an existing helper; reports the skip instead
|
||||
- [ ] Asks "May I write" before any file is created
|
||||
- [ ] Verdict is COMPLETE when helper files are written
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- Mock/stub helper generation (for dependencies like save systems or audio buses)
|
||||
follows the same pattern as factory helpers and is not separately tested.
|
||||
- Unity C# helper generation (using NSubstitute or custom mocks) follows the
|
||||
same logic as Case 1 with language-appropriate output.
|
||||
- The case where the requested helper type is not recognized is not tested;
|
||||
the skill would ask the user to clarify the helper type.
|
||||
- `all` mode (one factory per system with test files) follows Case 1 per system;
|
||||
not separately tested.
|
||||
- Unreal helper generation (`Source/<Module>/Private/Tests/Helpers/`) follows
|
||||
Case 6's pattern for its own helper root; not separately tested. Nor is a
|
||||
Unity project whose PlayMode tests need the helpers too (the skill asks before
|
||||
creating a shared `TestHelpers.asmdef`).
|
||||
- A requested system with no GDD is not tested; the skill does not define that path.
|
||||
|
||||
@@ -2,29 +2,34 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/test-setup` scaffolds the test framework for the project based on the
|
||||
configured engine. It creates the `tests/` directory structure defined in
|
||||
`coding-standards.md` (unit/, integration/, performance/, playtest/) and
|
||||
generates the appropriate test runner configuration for the detected engine:
|
||||
GdUnit4 config for Godot, Unity Test Runner asmdef for Unity, or Unreal headless
|
||||
runner for Unreal Engine.
|
||||
`/test-setup [force]` scaffolds the test framework for the configured engine.
|
||||
It reads `engine.name` from `project.yaml` (falling back to
|
||||
`technical-preferences.md`) and stops with a `/setup-engine` redirect when
|
||||
neither names an engine. It checks for existing infrastructure, presents a plan,
|
||||
and asks "May I create these files? I will not overwrite any test files that
|
||||
already exist at these paths." before writing.
|
||||
|
||||
Each file or directory created is gated behind a "May I write" ask. If the test
|
||||
framework already exists, the skill verifies the configuration rather than
|
||||
reinitializing. No director gates apply. The verdict is COMPLETE when the
|
||||
scaffold is in place.
|
||||
It creates `tests/README.md`, `tests/smoke/critical-paths.md`,
|
||||
`production/qa/evidence/`, `.github/workflows/tests.yml`, and the engine's test
|
||||
root: for Godot `tests/unit/` and `tests/integration/` (placeholder files, and
|
||||
no runner script — gdUnit4's `addons/gdUnit4/bin/GdUnitCmdTool.gd` is the
|
||||
runner); for Unity `Assets/Tests/EditMode/` and `Assets/Tests/PlayMode/` with
|
||||
their `.asmdef` files; for Unreal `Source/<Module>/Private/Tests/`. When the
|
||||
infrastructure already exists it exits early unless `force` is passed, and
|
||||
`force` only creates missing files — it never overwrites. No director gates
|
||||
apply. The verdict is COMPLETE when the scaffold is in place.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keyword: COMPLETE
|
||||
- [ ] Contains "May I write" collaborative protocol language before creating files
|
||||
- [ ] Has a next-step handoff (e.g., `/test-helpers` to generate helper utilities)
|
||||
- [ ] Contains ask-before-write language before creating files ("May I create these files?")
|
||||
- [ ] Has a next-step handoff (Phase 6 Next steps: `/qa-plan sprint`, `/smoke-check`)
|
||||
|
||||
---
|
||||
|
||||
@@ -39,26 +44,34 @@ None. `/test-setup` is a scaffolding utility. No director gates apply.
|
||||
### Case 1: Happy Path — Godot project, scaffolds GdUnit4 test structure
|
||||
|
||||
**Fixture:**
|
||||
- `technical-preferences.md` has engine set to Godot 4, language GDScript
|
||||
- `tests/` directory does not exist yet
|
||||
- `project.yaml` has `engine.name: Godot`
|
||||
- `tests/` does not exist; `addons/gdUnit4/` is not installed; no CI workflow
|
||||
- The user approves the plan
|
||||
|
||||
**Input:** `/test-setup`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads engine from `technical-preferences.md` → Godot 4 + GDScript
|
||||
2. Skill drafts the test directory structure: tests/unit/, tests/integration/,
|
||||
tests/performance/, tests/playtest/, and a GdUnit4 runner config file
|
||||
3. Skill asks "May I write the tests/ directory structure?"
|
||||
4. Directories and GdUnit4 runner script created on approval
|
||||
5. Skill confirms the runner script matches the CI command in coding-standards.md:
|
||||
`godot --headless --script tests/gdunit4_runner.gd`
|
||||
6. Verdict is COMPLETE
|
||||
1. Phase 1 reports "Engine: Godot. Test directory: not found. CI workflow: not found."
|
||||
2. Phase 2 presents the plan: `tests/unit/`, `tests/integration/`, `tests/smoke/`,
|
||||
`tests/README.md`, `production/qa/evidence/`, `.github/workflows/tests.yml` —
|
||||
and no runner script
|
||||
3. Skill asks "May I create these files? I will not overwrite any test files that
|
||||
already exist at these paths."
|
||||
4. On approval: `tests/unit/.gdignore_placeholder`,
|
||||
`tests/integration/.gdignore_placeholder`, `tests/README.md` (with the
|
||||
Installing GdUnit4 steps and the capital-U `addons/gdUnit4/` path),
|
||||
`tests/smoke/critical-paths.md`, and a file under `production/qa/evidence/`
|
||||
5. `.github/workflows/tests.yml` uses `godot-gdunit-labs/gdUnit4-action@v1`
|
||||
with `version: 'installed'` and `checks: write`
|
||||
6. The documented test command matches `coding-standards.md`'s CI command:
|
||||
`godot --headless -s -d --remote-debug tcp://127.0.0.1:0 res://addons/gdUnit4/bin/GdUnitCmdTool.gd -a res://tests --ignoreHeadlessMode`
|
||||
7. Phase 6 summary; verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] All 4 subdirectories (unit/, integration/, performance/, playtest/) are created
|
||||
- [ ] GdUnit4 runner config is generated
|
||||
- [ ] Runner script path matches coding-standards.md CI command
|
||||
- [ ] "May I write" is asked before creating any files
|
||||
- [ ] `tests/unit/`, `tests/integration/`, `tests/smoke/` and `production/qa/evidence/` all exist afterwards (each holds a written file)
|
||||
- [ ] No custom runner script is written (the old hand-written runner loaded a file gdUnit4 never shipped and failed every run)
|
||||
- [ ] The test command matches the coding-standards.md CI command
|
||||
- [ ] Approval is asked before any file is created
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
@@ -66,63 +79,69 @@ None. `/test-setup` is a scaffolding utility. No director gates apply.
|
||||
### Case 2: Unity Project — Scaffolds Unity Test Runner with asmdef
|
||||
|
||||
**Fixture:**
|
||||
- `technical-preferences.md` has engine set to Unity, language C#
|
||||
- `tests/` directory does not exist
|
||||
- `project.yaml` has `engine.name: Unity`
|
||||
- `Packages/manifest.json` lists `com.unity.test-framework`
|
||||
- No `Assets/Tests/`; the user approves the plan
|
||||
|
||||
**Input:** `/test-setup`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads engine → Unity + C#
|
||||
2. Skill creates `Tests/` directory with Unity conventions (capitalized)
|
||||
3. Skill generates `Tests/Tests.asmdef` and `Tests/Editor/EditorTests.asmdef`
|
||||
4. EditMode and PlayMode test runner modes are configured
|
||||
5. Skill asks "May I write the Tests/ directory structure?"
|
||||
6. Verdict is COMPLETE
|
||||
1. Skill reads engine → Unity; the Test Framework package check passes
|
||||
2. The plan places unit and integration tests under `Assets/Tests/` — Unity
|
||||
compiles only `Assets/` and `Packages/` — not under `tests/unit/`
|
||||
3. Skill generates `Assets/Tests/EditMode/EditModeTests.asmdef` (Editor-only)
|
||||
and `Assets/Tests/PlayMode/PlayModeTests.asmdef`
|
||||
4. `.github/workflows/tests.yml` runs `game-ci/unity-test-runner@v4` for
|
||||
`editmode` and `playmode`, and the `UNITY_LICENSE` secret is noted
|
||||
5. The README's Running Unity tests note gives two runs — Edit Mode into
|
||||
`test-results/editmode.xml`, Play Mode into `test-results/playmode.xml`
|
||||
6. The Phase 6 gate note names Unity's test root (`Assets/Tests/EditMode/` and
|
||||
`Assets/Tests/PlayMode/`), not `tests/unit/` and `tests/integration/`
|
||||
7. Approval is asked before writing; verdict COMPLETE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Unity-specific `Tests/` structure is created (not the Godot structure)
|
||||
- [ ] Tests are under `Assets/Tests/`, never `tests/` or a top-level `Tests/` (neither is compiled: 0 tests, "Passed")
|
||||
- [ ] `.asmdef` files are generated
|
||||
- [ ] EditMode and PlayMode runner config is present
|
||||
- [ ] EditMode and PlayMode runner config is present, in CI and as two local runs with separate results files
|
||||
- [ ] The gate note names the Unity test root
|
||||
- [ ] Verdict is COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Test Framework Already Exists — Verifies config, not re-initialized
|
||||
### Case 3: Infrastructure Already Exists — Early exit, nothing re-initialized
|
||||
|
||||
**Fixture:**
|
||||
- `tests/unit/`, `tests/integration/` exist
|
||||
- GdUnit4 runner script exists (Godot project)
|
||||
- Godot project; `tests/unit/`, `tests/integration/`, `.github/workflows/tests.yml`
|
||||
and `addons/gdUnit4/bin/GdUnitCmdTool.gd` all exist
|
||||
- No `force` argument
|
||||
|
||||
**Input:** `/test-setup`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill detects existing tests/ structure
|
||||
2. Skill reports: "Test framework already exists — verifying configuration"
|
||||
3. Skill checks: runner script path, directory completeness, CI command alignment
|
||||
4. If all checks pass: reports "Configuration verified — no changes needed"
|
||||
5. If checks fail (e.g., missing tests/performance/): reports specific gap and
|
||||
asks "May I add the missing directories?"
|
||||
1. Phase 1 reports "Engine: Godot. Test directory: found. CI workflow: found."
|
||||
2. Skill says: "Test infrastructure appears to be in place. Re-run with
|
||||
`/test-setup force` to regenerate. Proceeding will not overwrite existing test files."
|
||||
3. No plan approval is requested and no file is written
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Skill does NOT reinitialize when framework exists
|
||||
- [ ] Verification checks are performed on existing structure
|
||||
- [ ] Only missing parts trigger a "May I write" ask
|
||||
- [ ] Verdict is COMPLETE whether everything was OK or gaps were fixed
|
||||
- [ ] Skill does NOT re-initialize when the infrastructure exists
|
||||
- [ ] The message points to `/test-setup force`
|
||||
- [ ] No existing file is modified and no new file is written
|
||||
|
||||
---
|
||||
|
||||
### Case 4: No Engine Configured — Redirects to /setup-engine
|
||||
|
||||
**Fixture:**
|
||||
- `technical-preferences.md` contains only placeholders (engine not set)
|
||||
- `project.yaml` has no `engine.name`; `technical-preferences.md` shows
|
||||
`[TO BE CONFIGURED]` for Engine
|
||||
|
||||
**Input:** `/test-setup`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads `technical-preferences.md` and finds engine placeholder
|
||||
2. Skill reports: "Engine not configured — cannot scaffold engine-specific test framework"
|
||||
3. Skill suggests running `/setup-engine` first
|
||||
4. No directories or files are created
|
||||
1. Skill checks `project.yaml`, then `technical-preferences.md` — no engine
|
||||
2. Skill stops: "Engine not configured. Run `/setup-engine` first, then re-run `/test-setup`."
|
||||
3. No directories or files are created
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Error message explicitly states engine is not configured
|
||||
@@ -153,21 +172,23 @@ None. `/test-setup` is a scaffolding utility. No director gates apply.
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Reads engine from `technical-preferences.md` before generating any scaffold
|
||||
- [ ] Generates engine-appropriate test runner config (not generic)
|
||||
- [ ] Creates all 4 subdirectories from coding-standards.md
|
||||
- [ ] Asks "May I write" before creating files
|
||||
- [ ] Detects existing framework and offers verification (not reinitialization)
|
||||
- [ ] Reads the engine (`project.yaml`, then `technical-preferences.md`) before generating any scaffold
|
||||
- [ ] Generates engine-appropriate test runner config and CI workflow (not generic)
|
||||
- [ ] Puts unit and integration tests in the engine's test root (`tests/` for Godot, `Assets/Tests/` for Unity, `Source/<Module>/Private/Tests/` for Unreal)
|
||||
- [ ] Creates `production/qa/evidence/` — the path the Phase 6 summary reports
|
||||
- [ ] Asks approval before creating files
|
||||
- [ ] Exits early when infrastructure exists; `force` creates only missing files and never overwrites
|
||||
- [ ] Verdict is COMPLETE when scaffold is in place
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- Unreal Engine test scaffolding (headless runner with `-nullrhi`) follows the
|
||||
same pattern as Cases 1 and 2 and is not separately fixture-tested.
|
||||
- CI integration file generation (e.g., `.github/workflows/test.yml`) is
|
||||
referenced but not assertion-tested here — it may be a separate skill concern.
|
||||
- Unreal Engine test scaffolding (`Source/<Module>/Private/Tests/README.md`, a
|
||||
self-hosted Windows CI runner) is not separately fixture-tested.
|
||||
- The `force` path on a partial scaffold (create only the missing files) is
|
||||
covered by protocol compliance, not by its own case.
|
||||
- The case where tests/ exists but is from a different engine (e.g., Unity tests
|
||||
in a now-Godot project) is not tested; the skill would detect the mismatch
|
||||
and offer to reconcile.
|
||||
in a now-Godot project) is not tested.
|
||||
- A Unity project without `com.unity.test-framework` (the skill asks to add the
|
||||
version the installed editor bundles, never a copied number) is not tested.
|
||||
|
||||
Reference in New Issue
Block a user