Release 1.1.2 — the default path works end to end

Fix release for 1.1.1: the minimal path end to end, engine commands that
run on Godot, Unity and Unreal, hook warnings you can see, and verdicts
that say when they could not check. Details in CHANGELOG.md; upgrade
steps in UPGRADING.md (v1.1.1 → v1.1.2).

Fixes #67
Fixes #131

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WuJn2FdynEybfL5KC697nJ
This commit is contained in:
Donchitos
2026-09-29 21:32:19 +10:00
co-authored by Claude Opus 5.5
parent 7ed2c3e9c4
commit b21fa0f7f2
383 changed files with 22888 additions and 10618 deletions
@@ -2,171 +2,209 @@
## Skill Summary
`/regression-suite` maps test coverage to GDD requirements: it reads the
acceptance criteria from story files in the current sprint (or a specified epic),
then scans `tests/` for corresponding test files and checks whether each AC has
a matching assertion. It produces a coverage report identifying which ACs are
fully covered, partially covered, or untested, and which test files have no
matching AC (orphan tests).
`/regression-suite [update | audit | report]` maintains `tests/regression-suite.md`,
a curated manifest of existing tests that guard critical paths and fixed bugs.
`audit` maps each GDD acceptance criterion (scoped by workflow tier) to tests in
`tests/unit/[system]/` and `tests/integration/[system]/` as COVERED / PARTIAL /
MISSING / EXEMPT, raising MISSING formula or state-machine criteria to HIGH
PRIORITY. `update` checks bugs in `production/qa/bugs/` with `Status: Closed` or
`Status: Fixed` for a regression test (HAS / MISSING REGRESSION TEST) and appends
new manifest entries without removing any. `report` is read-only. All modes
check for coverage drift.
The skill may write a coverage report to `production/qa/` after a "May I write"
ask. No director gates apply. Verdicts: FULL COVERAGE (all ACs have tests),
GAPS FOUND (some ACs are untested), or CRITICAL GAPS (a critical-priority AC
has no test).
At `qa.level: minimal` the skill stops before any scan with "Regression suite
not generated at qa.level minimal". With zero critical paths or zero test files
it reports `Coverage: NOT ASSESSED — [no GDDs found | no test files found]`
instead of a percentage. Before writing it asks "May I write/update
`tests/regression-suite.md` with the current regression suite manifest?".
Verdict, first match: **NOT ASSESSED** at the `qa.level: minimal` stop or when
coverage was not computed (never COMPLETE over a missing figure, even after a
write), **BLOCKED** if the user declines the write, **COMPLETE** after the
write — or, in read-only `report` mode, once the report is shown. `qa.level`
also sets timing — the suite is due at Polish-stage entry at `standard` and at
Production-stage entry at `full` — so each fixture pins `project.stage`. No
director gates apply.
---
## Static Assertions (Structural)
Verified automatically by `/skill-test static` — no fixture needed.
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
- [ ] Has ≥2 phase headings
- [ ] Contains verdict keywords: FULL COVERAGE, GAPS FOUND, CRITICAL GAPS
- [ ] Contains "May I write" language (skill may write coverage report)
- [ ] Has a next-step handoff (e.g., `/test-setup` if framework missing, `/qa-plan` if plan missing)
- [ ] Contains verdict keywords: COMPLETE, BLOCKED, NOT ASSESSED
- [ ] Contains "May I write" language before writing `tests/regression-suite.md`
- [ ] Has a next-step handoff (`/test-helpers` for HIGH gaps, `/regression-suite audit` on drift, `/test-flakiness` for quarantined tests)
---
## Director Gate Checks
None. `/regression-suite` is a QA analysis utility. No director gates apply.
None. `/regression-suite` is a QA utility. It spawns no agents (`Agent` is not
in its allowed tools) and no director gates apply.
---
## Test Cases
### Case 1: Full Coverage — All ACs in sprint have corresponding tests
### Case 1: Audit — Criteria Mapped, Formula Gap Raised to HIGH
**Fixture:**
- `production/sprints/sprint-004.md` lists 3 stories with 2 ACs each (6 total)
- `tests/unit/` and `tests/integration/` contain test files that match all 6 ACs
(by system name and scenario description)
- `qa.level: standard`; workflow tier `full`; `project.stage: Polish`
- `design/gdd/systems-index.md` lists `combat`; `design/gdd/combat.md` has 4
acceptance criteria: a damage formula, a hit-stun state transition, a
knockback rule and a HUD damage-number display
- `tests/unit/combat/` has tests for hit-stun (all cases) and knockback (happy
path only); nothing tests the damage formula
- `tests/regression-suite.md` exists
**Input:** `/regression-suite sprint-004`
**Input:** `/regression-suite audit`
**Expected behavior:**
1. Skill reads all 6 ACs from sprint-004 stories
2. Skill scans test files and matches each AC to at least one test assertion
3. All 6 ACs have coverage
4. Skill produces coverage report: "6/6 ACs covered"
5. Skill asks "May I write to `production/qa/regression-sprint-004.md`?"
6. File is written on approval; verdict is FULL COVERAGE
1. Skill reads the existing manifest, globs the test inventory and reads the
combat GDD's critical paths
2. Coverage: hit-stun COVERED, knockback PARTIAL, damage formula MISSING, HUD
display EXEMPT (UI, not automatable)
3. The missing damage formula is raised to a HIGH PRIORITY gap with a suggested
test file path
4. The report shows the per-system table and a non-exempt coverage rate
5. Skill asks "May I write/update `tests/regression-suite.md` with the current
regression suite manifest?"; on approval audit rewrites the full manifest
6. Verdict is COMPLETE; for the HIGH gap it suggests `/test-helpers`
**Assertions:**
- [ ] All 6 ACs appear in the coverage report
- [ ] Each AC is marked as covered with the matching test file referenced
- [ ] Verdict is FULL COVERAGE
- [ ] "May I write" is asked before writing the report
- [ ] Each criterion gets one of COVERED / PARTIAL / MISSING / EXEMPT, matching the fixture
- [ ] The HUD criterion is EXEMPT and excluded from the coverage rate
- [ ] The damage-formula gap is HIGH PRIORITY with a `tests/unit/combat/` path
- [ ] The manifest is written only after the "May I write/update" ask
- [ ] Verdict is COMPLETE and `/test-helpers` is suggested
---
### Case 2: Gaps Found — 3 ACs have no tests
### Case 2: Update — Fixed Bug Without a Regression Test
**Fixture:**
- Sprint has 5 stories with 8 total ACs
- Tests exist for 5 of the 8 ACs; 3 ACs have no corresponding test file or assertion
- `qa.level: standard`; `project.stage: Polish`
- `production/sprints/sprint-007.md` is the current sprint plan; its one
`Status: Complete` story is `production/epics/inventory/story-003.md`
- `production/qa/bugs/BUG-0012.md` has `Status: Closed`, system `inventory`
- No test under `tests/unit/inventory/` or `tests/integration/inventory/`
references BUG-0012 or its failure scenario
- `tests/regression-suite.md` has 6 registered tests
**Input:** `/regression-suite`
**Input:** `/regression-suite update`
**Expected behavior:**
1. Skill reads all 8 ACs
2. Skill scans tests — 5 matched, 3 unmatched
3. Coverage report lists the 3 untested ACs by story and AC text
4. Skill asks "May I write to `production/qa/regression-[sprint]-[date].md`?"
5. Report is written; verdict is GAPS FOUND
1. Skill reads the sprint plan's completed stories and the closed bugs
2. BUG-0012 is MISSING REGRESSION TEST, with suggested path
`tests/unit/inventory/[bug-slug]_regression_test.[ext]` and the note
"Without this test, this bug can silently return in a future sprint."
3. Report shows the Bug Regression Coverage table with BUG-0012 marked NO
4. After the "May I write/update" ask, new entries are appended with targeted
`Edit` insertions; the 6 existing entries remain
5. Skill says the next sprint should include a story to write the missing test;
verdict COMPLETE
**Assertions:**
- [ ] The 3 untested ACs are listed by name in the report
- [ ] Matched ACs are also shown (not only the gaps)
- [ ] Verdict is GAPS FOUND (not FULL COVERAGE)
- [ ] Report is written after "May I write" approval
- [ ] BUG-0012 is reported as MISSING REGRESSION TEST
- [ ] The suggested path follows `tests/unit/[system]/[bug-slug]_regression_test.[ext]`
- [ ] No existing manifest entry is removed
- [ ] The next-sprint story recommendation appears because bug regression gaps > 0
- [ ] Verdict is COMPLETE after approval
---
### Case 3: Critical AC Untested — CRITICAL GAPS verdict, flagged prominently
### Case 3: qa.level minimal — Suite Not Generated
**Fixture:**
- Sprint has 4 stories; one story is Priority: Critical with 2 ACs
- One of the critical-priority ACs has no test
- `qa.level: minimal`
- GDDs and test files exist
**Input:** `/regression-suite`
**Input:** `/regression-suite audit`
**Expected behavior:**
1. Skill reads all stories and ACs, noting which stories are critical priority
2. Skill scans tests — the critical AC has no match
3. Report prominently flags: "CRITICAL GAP: [AC text] — no test found (Critical priority story)"
4. Skill recommends blocking story completion until test is added
5. Verdict is CRITICAL GAPS
1. The early `qa.level` guard fires before any scan
2. Skill reports "Regression suite not generated at qa.level minimal" and stops
3. It does not fall into the workflow-tier `minimal` branch (smoke-report scope)
4. Verdict is NOT ASSESSED — the first rule of the verdict list
**Assertions:**
- [ ] Verdict is CRITICAL GAPS (not GAPS FOUND)
- [ ] Critical priority AC is flagged more prominently than normal gaps
- [ ] Recommendation to block story completion is included
- [ ] Non-critical gaps (if any) are also listed
- [ ] The "not generated at qa.level minimal" message is shown
- [ ] No GDD, test or bug scan runs
- [ ] The smoke-report branch is not entered on account of `qa.level`
- [ ] Nothing is written and no "May I write" ask appears
- [ ] Verdict is NOT ASSESSED — not COMPLETE, and not a stop without a verdict
---
### Case 4: Orphan Tests — Test file has no matching AC
### Case 4: Edge Case — No Test Files, Coverage NOT ASSESSED
**Fixture:**
- `tests/unit/save_system_test.gd` exists with assertions for scenarios
not present in any current story's AC list
- Current sprint stories do not reference save system
- `qa.level: standard`; workflow tier `full`; `project.stage: Polish`
- `design/gdd/` holds GDDs with acceptance criteria
- `tests/unit/`, `tests/integration/` and `tests/regression/` contain no files
- The user approves the manifest write
**Input:** `/regression-suite`
**Input:** `/regression-suite audit`
**Expected behavior:**
1. Skill scans tests and cross-references ACs
2. `save_system_test.gd` assertions do not match any current AC
3. Test file is flagged as ORPHAN TEST in the coverage report
4. Report notes: "Orphan tests may belong to a past or future sprint, or AC was renamed"
5. Verdict is FULL COVERAGE or GAPS FOUND depending on overall AC coverage
(orphan tests do not affect verdict, they are advisory)
1. The test globs return zero files
2. Skill does not emit a coverage percentage
3. It reports `Coverage: NOT ASSESSED — no test files found`, naming the empty
side and `/test-setup` as the skill that produces the test scaffold
4. The manifest is written only after the "May I write/update" ask
5. Verdict is NOT ASSESSED — the first match, even though the write happened
**Assertions:**
- [ ] Orphan test is flagged in the report
- [ ] Orphan flag includes the filename and suggestion (past sprint / renamed AC)
- [ ] Orphan tests do not cause a GAPS FOUND verdict on their own
- [ ] Overall verdict reflects AC coverage only
- [ ] No `0%` (or any percentage) coverage figure is reported
- [ ] Coverage reads `NOT ASSESSED — no test files found`
- [ ] `/test-setup` is named as the producing skill
- [ ] Any manifest write still requires the "May I write/update" ask
- [ ] Verdict is NOT ASSESSED — never COMPLETE over a coverage figure that was not computed
---
### Case 5: Director Gate Check — No gate; regression-suite is a QA utility
### Case 5: Director Gate Check — None; Report Mode Is Read-Only
**Fixture:**
- Sprint with stories and test files
- `qa.level: standard`; `project.stage: Polish`
- `tests/regression-suite.md`, tests and closed bugs exist
- Any review mode
**Input:** `/regression-suite`
**Input:** `/regression-suite report`
**Expected behavior:**
1. Skill produces coverage report and writes it
2. No director agents are spawned
3. No gate IDs appear in output
1. Skill produces the Regression Suite Status report in conversation
2. No director or other agent is spawned; no gate IDs appear
3. `report` mode writes nothing and asks no write question
**Assertions:**
- [ ] No director gate is invoked
- [ ] No gate skip messages appear
- [ ] Verdict is FULL COVERAGE, GAPS FOUND, or CRITICAL GAPS — no gate verdict
- [ ] No director gate is invoked and no gate skip message appears
- [ ] No subagent is spawned
- [ ] The status report is shown in conversation
- [ ] No file is written and no "May I write" ask appears
---
## Protocol Compliance
- [ ] Reads story ACs from sprint files before scanning tests
- [ ] Matches ACs to tests by system name and scenario (not file name alone)
- [ ] Flags critical-priority untested ACs as CRITICAL GAPS
- [ ] Flags orphan tests (exist in tests/ but no AC matches)
- [ ] Asks "May I write" before persisting the coverage report
- [ ] Verdict is FULL COVERAGE, GAPS FOUND, or CRITICAL GAPS
- [ ] Stops before scanning at `qa.level: minimal`
- [ ] Reports `NOT ASSESSED` instead of a percentage when GDDs or tests are absent
- [ ] Asks "May I write/update `tests/regression-suite.md`" before any write
- [ ] `update` only appends; existing manifest entries are never removed without approval
- [ ] `report` writes nothing
- [ ] Verdict, first match: NOT ASSESSED at the `qa.level: minimal` stop or when coverage was not computed (even after a write); else BLOCKED if the user declines the write; else COMPLETE
---
## Coverage Notes
- The heuristic for matching an AC to a test (by system name + scenario keywords)
is approximate; exact matching logic is defined in the skill body.
- Integration test coverage is mapped the same way as unit test coverage; no
distinction in verdicts is made between the two.
- This skill does not run the tests — it maps AC text to test assertions. Test
execution is handled by the CI pipeline.
- No-argument behaviour (runs `update` when a sprint is clearly active, else
asks A/B/C) is not tested.
- At workflow tier `minimal`, `audit` maps coverage against the latest
`production/qa/smoke-*.md` report instead of GDD criteria, and stops if none
exists; not separately tested.
- Coverage-drift detection (new systems, stale manifest date) is not tested by a
dedicated case.
- The skill does not run tests; it maps criteria and bugs to test files by name.