mirror of
https://github.com/Donchitos/Claude-Code-Game-Studios.git
synced 2026-10-16 10:17:06 +00:00
Release 1.1.2 — the default path works end to end
Fix release for 1.1.1: the minimal path end to end, engine commands that run on Godot, Unity and Unreal, hook warnings you can see, and verdicts that say when they could not check. Details in CHANGELOG.md; upgrade steps in UPGRADING.md (v1.1.1 → v1.1.2). Fixes #67 Fixes #131 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WuJn2FdynEybfL5KC697nJ
This commit is contained in:
Donchitos
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
7ed2c3e9c4
commit
b21fa0f7f2
@@ -2,174 +2,293 @@
|
||||
|
||||
## Skill Summary
|
||||
|
||||
`/test-evidence-review` performs a quality review of test files in `tests/`,
|
||||
checking test naming conventions, determinism, isolation, and absence of
|
||||
hardcoded magic numbers — all against the project's test standards defined in
|
||||
`coding-standards.md`. Findings may be flagged for qa-lead review. No director
|
||||
gates are invoked. The skill does not write without user approval. Verdicts:
|
||||
PASS, WARNINGS, or FAIL.
|
||||
`/test-evidence-review` reviews the **quality** of the test files and evidence
|
||||
documents behind stories — `/smoke-check` covers existence and passing. It
|
||||
takes a scope: a single story path, `sprint` (the most recent plan in
|
||||
`production/sprints/`), or a system name (`production/epics/[system]/story-*.md`);
|
||||
with no argument it asks which. If the resolved scope holds zero stories it stops
|
||||
with `NOT ASSESSED — no stories in scope` and routes to the skill that creates
|
||||
them. For each story it collects Story Type, evidence path and acceptance
|
||||
criteria with targeted greps, locates the evidence (`tests/unit/[system]/`,
|
||||
`tests/integration/[system]/`, `production/qa/evidence/`,
|
||||
`production/qa/smoke-*.md`), and reviews automated tests for assertion count,
|
||||
edge cases, naming and formula traceability, and manual evidence for criterion
|
||||
linkage, sign-offs, retained screenshots and freshness. It resolves `qa.level`
|
||||
and `testing.strict` the way `/story-done` does: at `qa.level: minimal` a Logic,
|
||||
Integration or Config/Data story with no test evidence is
|
||||
`WAIVED (qa.level: minimal)`, not MISSING, and a gap in a test that does exist is
|
||||
ADVISORY; Visual/Feel and UI evidence is never waived; otherwise each gap takes
|
||||
its story type's gate level from `testing.strict.<type>` (unset → a
|
||||
plain-boolean `testing.strict` legacy value applies to every type; else the
|
||||
`coding-standards.md` default). Each story gets ADEQUATE, INCOMPLETE, MISSING,
|
||||
NOT ASSESSED or WAIVED; the overall verdict is the worst present, with NOT
|
||||
ASSESSED outranking ADEQUATE only, and WAIVED below ADEQUATE. The report is presented in
|
||||
conversation, and the skill asks "May I write this test evidence review to
|
||||
`production/qa/evidence-review-[date].md`?" as an optional step. It never
|
||||
modifies test files or evidence docs and invokes no director gates. The run ends
|
||||
COMPLETE, or CONCERNS when BLOCKING items were found.
|
||||
|
||||
---
|
||||
|
||||
## Static Assertions (Structural)
|
||||
|
||||
Verified automatically by `/skill-test static` — no fixture needed.
|
||||
Checked against the SKILL.md by `/skill-test spec` — no fixture needed.
|
||||
|
||||
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
|
||||
- [ ] Has ≥2 phase headings
|
||||
- [ ] Contains verdict keywords: PASS, WARNINGS, FAIL
|
||||
- [ ] Does NOT require "May I write" language (read-only; write is optional flagging report)
|
||||
- [ ] Has a next-step handoff (what to do after findings are reviewed)
|
||||
- [ ] Contains verdict keywords: ADEQUATE, INCOMPLETE, MISSING, NOT ASSESSED, and the run-level COMPLETE / CONCERNS
|
||||
- [ ] Contains "May I write" language for the optional `production/qa/evidence-review-[date].md`
|
||||
- [ ] Has a next-step handoff (Section 7: BLOCKING items before `/story-done`, `/test-helpers`, sign-off routing)
|
||||
- [ ] The `resolve_config` injection names `qa.level` and `testing.strict`, and the skill states the `WAIVED (qa.level: minimal)` result and the per-type `testing.strict.<type>` gate level
|
||||
|
||||
---
|
||||
|
||||
## Director Gate Checks
|
||||
|
||||
None. Test evidence review is an advisory quality skill; QL-TEST-COVERAGE gate
|
||||
is a separate skill invocation and is NOT triggered here.
|
||||
None. Test evidence review is an advisory quality skill; no gates are invoked.
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Case 1: Happy Path — Tests follow all standards
|
||||
### Case 1: Happy Path — Single Logic story with a strong test
|
||||
|
||||
**Fixture:**
|
||||
- `tests/unit/combat/health_system_take_damage_test.gd` exists with:
|
||||
- Naming: `test_health_system_take_damage_reduces_health()` (follows `test_[system]_[scenario]_[expected]`)
|
||||
- Arrange/Act/Assert structure present
|
||||
- No `sleep()`, `await` with time values, or random seeds
|
||||
- No calls to external APIs or file I/O
|
||||
- No inline magic numbers (uses constants from `tests/unit/combat/fixtures/`)
|
||||
- `project.yaml`: `engine.name: godot`, so the unit-test root is `tests/unit/`; `modes.automation` unset (collaborative)
|
||||
- `production/epics/combat/story-001-damage-calc.md` (slug `damage-calc`): `## Test Evidence` gives Story Type Logic and the path `tests/unit/combat/damage-calc_test.gd`; `## Acceptance Criteria` includes "damage is never below 1" and "critical hits multiply damage by 2.0"
|
||||
- `tests/unit/combat/damage-calc_test.gd` opens with `# Story: production/epics/combat/story-001-damage-calc.md`, has 4 test functions with 3+ assertions each, named like `test_damage_calc_zero_armor_returns_base_damage` and `test_damage_calc_max_crit_doubles_damage`, and names the GDD's `damage_formula` in a comment
|
||||
|
||||
**Input:** `/test-evidence-review tests/unit/combat/`
|
||||
**Input:** `/test-evidence-review production/epics/combat/story-001-damage-calc.md`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads test standards from `coding-standards.md`
|
||||
2. Skill reads the test file; checks all 5 standards
|
||||
3. All checks pass: naming, structure, determinism, isolation, no hardcoded data
|
||||
4. Verdict is PASS
|
||||
1. Single-story mode: skill reads the story's Test Evidence and Acceptance Criteria
|
||||
2. Section 3 finds the test file under `tests/unit/combat/`
|
||||
3. Section 4: assertion coverage is normal (3+ per function); "zero" and "max" cases cover the numeric criteria; names follow `test_[scenario]_[expected]`; the formula is referenced
|
||||
4. Story verdict: **ADEQUATE**; overall verdict ADEQUATE; 0 BLOCKING items
|
||||
5. Report is presented in conversation, then the skill asks "May I write this test evidence review to `production/qa/evidence-review-[date].md`?" as optional
|
||||
6. Verdict: **COMPLETE**
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Each of the 5 test standards is checked and reported
|
||||
- [ ] All checks show PASS when standards are met
|
||||
- [ ] Verdict is PASS
|
||||
- [ ] No files are written
|
||||
- [ ] The test file is located under `tests/unit/combat/` for the Logic story
|
||||
- [ ] Assertion coverage, edge cases, naming and formula traceability are each reported
|
||||
- [ ] Story verdict is ADEQUATE
|
||||
- [ ] The report is shown before the optional "May I write" prompt
|
||||
- [ ] Run ends with COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 2: Fail — Timing dependency detected
|
||||
### Case 2: Vacuous and Thin Tests — INCOMPLETE with a BLOCKING item
|
||||
|
||||
**Fixture:**
|
||||
- `tests/unit/ui/hud_update_test.gd` contains:
|
||||
```gdscript
|
||||
await get_tree().create_timer(1.0).timeout
|
||||
assert_eq(label.text, "Ready")
|
||||
```
|
||||
- Real-time wait of 1 second used instead of mock or signal-based assertion
|
||||
- `project.yaml`: `engine.name: godot`, `modes.rigor: standard` (so `qa.level: standard` — tests are required); no `testing.strict` block
|
||||
- `production/epics/combat/story-002-crit-roll.md` (slug `crit-roll`): Story Type Logic
|
||||
- `tests/unit/combat/crit-roll_test.gd` has `func test_1()` with no assertions and `func test_crit_roll_applies_multiplier()` with one assertion
|
||||
|
||||
**Input:** `/test-evidence-review tests/unit/ui/hud_update_test.gd`
|
||||
**Input:** `/test-evidence-review production/epics/combat/story-002-crit-roll.md`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads the test file
|
||||
2. Skill detects real-time wait (`create_timer(1.0)`) — non-deterministic timing dependency
|
||||
3. Skill flags this as a FAIL-level finding
|
||||
4. Verdict is FAIL
|
||||
5. Skill recommends replacing the timer with a signal-based assertion or mock
|
||||
1. Skill finds the test file and counts assertions per function
|
||||
2. `test_1` has 0 assertions → flagged BLOCKING (passes vacuously, proves nothing); `test_1` is also flagged as a generic name
|
||||
3. The one-assertion function is noted as potentially thin
|
||||
4. Story verdict: **INCOMPLETE**
|
||||
5. Follow-ups: "These must be resolved before `/story-done` can mark the story Complete…" and "Consider running `/test-helpers [system]`…"
|
||||
6. Run ends with **CONCERNS** because BLOCKING items were found
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Real-time wait usage is detected as a non-deterministic timing dependency
|
||||
- [ ] Finding is classified as FAIL severity (blocking — violates determinism standard)
|
||||
- [ ] Verdict is FAIL
|
||||
- [ ] Remediation suggestion references signal-based or mock-based approach
|
||||
- [ ] Skill does not edit the test file
|
||||
- [ ] A test function with zero assertions is flagged BLOCKING
|
||||
- [ ] `test_1` is flagged as a naming issue
|
||||
- [ ] Story verdict is INCOMPLETE, not ADEQUATE
|
||||
- [ ] Output states BLOCKING items must be resolved before `/story-done` and suggests `/test-helpers`
|
||||
- [ ] Run ends with CONCERNS
|
||||
- [ ] The test file is not modified
|
||||
- [ ] Variant — with `testing.strict.logic: false` in `project.yaml`, the zero-assertion finding is ADVISORY, the story is still INCOMPLETE, and the run ends COMPLETE
|
||||
|
||||
---
|
||||
|
||||
### Case 3: Fail — Test calls external API directly
|
||||
### Case 3: Empty Sprint Scope at Minimal Workflow — NOT ASSESSED, not a clean report
|
||||
|
||||
**Fixture:**
|
||||
- `tests/unit/networking/auth_test.gd` contains:
|
||||
```gdscript
|
||||
var result = HTTPRequest.new().request("https://api.example.com/auth")
|
||||
```
|
||||
- Direct HTTP call to external API without a mock
|
||||
- `project.yaml`: `modes.rigor: minimal` (the resolved block shows `workflow: minimal`)
|
||||
- `production/sprints/` does not exist
|
||||
|
||||
**Input:** `/test-evidence-review tests/unit/networking/auth_test.gd`
|
||||
**Input:** `/test-evidence-review sprint`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reads the test file
|
||||
2. Skill detects direct external API call (HTTPRequest to live URL)
|
||||
3. Skill flags this as a FAIL-level finding — violates isolation standard
|
||||
4. Verdict is FAIL
|
||||
5. Skill recommends injecting a mock HTTP client
|
||||
1. Sprint mode finds no sprint plan, so the scope holds zero stories
|
||||
2. Skill stops and reports `NOT ASSESSED — no stories in scope`, naming the sprint scope and the empty `production/sprints/` path
|
||||
3. Because the workflow is minimal (no sprints), it routes to `/test-evidence-review [epic-slug]` instead of `/sprint-plan new`
|
||||
4. Skill does not continue to Section 3; no Summary table and no "BLOCKING items: 0" line are rendered
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Direct external API call is detected and flagged
|
||||
- [ ] Finding is classified as FAIL severity (violates isolation standard)
|
||||
- [ ] Verdict is FAIL
|
||||
- [ ] Remediation references dependency injection with a mock HTTP client
|
||||
- [ ] Skill does not modify the test file
|
||||
- [ ] Verdict is NOT ASSESSED — no stories in scope
|
||||
- [ ] Output names which scope was searched and which path was empty
|
||||
- [ ] At minimal workflow the route is `/test-evidence-review [epic-slug]`, not `/sprint-plan new`
|
||||
- [ ] No empty report reading "BLOCKING items: 0 / ADVISORY items: 0" is produced
|
||||
|
||||
---
|
||||
|
||||
### Case 4: Edge Case — No Test Files Found
|
||||
### Case 4: Story Type Unknown — Per-story NOT ASSESSED lifts the overall verdict
|
||||
|
||||
**Fixture:**
|
||||
- User calls `/test-evidence-review tests/unit/audio/`
|
||||
- `tests/unit/audio/` directory does not exist
|
||||
- `project.yaml`: `engine.name: godot`
|
||||
- `production/epics/combat/` contains `story-001-damage-calc.md` (Logic, test as in Case 1) and `story-003-hit-feedback.md`, whose `## Test Evidence` section states no Story Type and no evidence path
|
||||
- Full-reading `story-003-hit-feedback.md` does not reveal a type either
|
||||
|
||||
**Input:** `/test-evidence-review tests/unit/audio/`
|
||||
**Input:** `/test-evidence-review combat`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill attempts to read files in `tests/unit/audio/` — not found
|
||||
2. Skill outputs: "No test files found at `tests/unit/audio/` — run `/test-setup` to scaffold test directories"
|
||||
3. No verdict is emitted
|
||||
1. System mode globs `production/epics/combat/story-*.md` and finds both stories
|
||||
2. `story-001-damage-calc` is ADEQUATE
|
||||
3. `story-003-hit-feedback` is **NOT ASSESSED**, with the reason stated: the story type cannot be determined, so the required evidence is unknown
|
||||
4. It is not reported as MISSING
|
||||
5. Overall verdict: **NOT ASSESSED** — it outranks ADEQUATE
|
||||
|
||||
**Assertions:**
|
||||
- [ ] Skill does not crash when path does not exist
|
||||
- [ ] Output names the attempted path in the message
|
||||
- [ ] Output recommends `/test-setup` for scaffolding
|
||||
- [ ] No verdict is emitted when there is nothing to review
|
||||
- [ ] The untyped story is NOT ASSESSED, not MISSING
|
||||
- [ ] The NOT ASSESSED reason (type cannot be determined) is stated for that story
|
||||
- [ ] Overall verdict is NOT ASSESSED, not ADEQUATE
|
||||
- [ ] The typed story is still reviewed and reported ADEQUATE
|
||||
|
||||
---
|
||||
|
||||
### Case 5: Gate Compliance — No gate; QL-TEST-COVERAGE is a separate skill
|
||||
### Case 5: Visual/Feel Evidence Without a Retained Screenshot, Full Review Mode — No gates
|
||||
|
||||
**Fixture:**
|
||||
- Test file has 1 WARNINGS-level finding (magic number in a non-boundary test)
|
||||
- `review-mode.txt` contains `full`
|
||||
- `project.yaml`: `engine.name: godot`, `modes.review_mode: full`
|
||||
- `production/epics/combat/story-004-hit-flash.md` (slug `hit-flash`): Story Type Visual/Feel
|
||||
- `production/qa/evidence/hit-flash-evidence.md` references every acceptance criterion, has every sign-off filled, and is dated after the sprint start
|
||||
- No `*.png`, `*.jpg` or `*.gif` for this story exists in `production/qa/evidence/`
|
||||
|
||||
**Input:** `/test-evidence-review tests/unit/combat/`
|
||||
**Input:** `/test-evidence-review production/epics/combat/story-004-hit-flash.md`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Skill reviews tests; finds 1 WARNINGS-level finding
|
||||
2. No director gate is invoked (QL-TEST-COVERAGE is invoked separately, not here)
|
||||
3. Verdict is WARNINGS
|
||||
4. Output notes: "For full test coverage gate, run `/gate-check` which invokes QL-TEST-COVERAGE"
|
||||
5. Skill offers optional report write; asks "May I write" if user opts in
|
||||
1. Section 3 finds `production/qa/evidence/hit-flash-evidence.md`
|
||||
2. Section 5: criterion linkage and sign-offs are complete, but no retained image exists for the story
|
||||
3. The doc describes a visual check without evidence → story verdict **INCOMPLETE**, flagged BLOCKING
|
||||
4. No director gate is invoked regardless of review mode
|
||||
5. The evidence document is not edited
|
||||
6. Run ends with **CONCERNS**
|
||||
|
||||
**Assertions:**
|
||||
- [ ] A Visual/Feel evidence doc with no retained image is INCOMPLETE, even with complete sign-offs
|
||||
- [ ] The missing screenshot is listed as a BLOCKING issue
|
||||
- [ ] No director gate is invoked in any review mode
|
||||
- [ ] Output distinguishes this skill from the QL-TEST-COVERAGE gate invocation
|
||||
- [ ] Optional report requires "May I write" before writing
|
||||
- [ ] Verdict is WARNINGS for advisory-level test quality issues
|
||||
- [ ] The evidence document is not modified
|
||||
|
||||
---
|
||||
|
||||
### Case 6: UI Story Closed by Its Screenshots — No Sign-Off Required
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml`: `engine.name: godot`
|
||||
- `production/epics/ui/story-007-inventory-screen.md` (slug `inventory-screen`): Story Type UI; its
|
||||
`## Test Evidence` names `production/qa/evidence/inventory-screen-populated.png` and
|
||||
`production/qa/evidence/inventory-screen-empty.png` — one per screen state it touches
|
||||
- Both images exist and are dated after the sprint start
|
||||
- No `inventory-screen-evidence.md`, no sign-off record and no walkthrough log exist
|
||||
|
||||
**Input:** `/test-evidence-review production/epics/ui/story-007-inventory-screen.md`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Section 3 uses the two paths the story states before searching
|
||||
2. Section 5: a UI story is closed by the retained screenshot of each screen it touched
|
||||
3. Story verdict **ADEQUATE**; run ends with **COMPLETE**
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The story's stated evidence paths are checked first, and both images are found
|
||||
- [ ] The missing sign-off does not make the story INCOMPLETE — a UI story needs none
|
||||
- [ ] No walkthrough log is demanded
|
||||
- [ ] Story verdict is ADEQUATE
|
||||
|
||||
---
|
||||
|
||||
### Case 7: Unity Logic Story — Evidence at the Stated Engine Path
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml`: `engine.name: unity`
|
||||
- `production/epics/inventory/story-003-stack-merge.md`: Story Type Logic; its
|
||||
`## Test Evidence` names `Assets/Tests/EditMode/InventoryTests.cs`
|
||||
- That file exists and holds one `[Test]` per acceptance criterion (4 in all), each with 3 or more `Assert.That` calls, named in the `[Scenario]_[Expected]` form (`MergeStacks_FullStack_SpillsToNewSlot`), with the empty-stack and max-stack cases covered
|
||||
- The inventory GDD has no Formulas section, so formula traceability does not apply
|
||||
- Nothing named `stack-merge_test.*` exists anywhere, and `tests/unit/` does not exist
|
||||
|
||||
**Input:** `/test-evidence-review production/epics/inventory/story-003-stack-merge.md`
|
||||
|
||||
**Expected behavior:**
|
||||
1. Section 3 finds the test at the stated path under `Assets/Tests/EditMode/`
|
||||
2. Section 4 reviews its assertions; story verdict **ADEQUATE**
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The test is found at the stated Unity path — the story is not reported MISSING
|
||||
- [ ] The review does not require a `[story-slug]_test.*` name or a `tests/unit/` location
|
||||
|
||||
---
|
||||
|
||||
### Case 8: No Test for a Logic Story — MISSING, and it outranks NOT ASSESSED
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml`: `engine.name: godot`, `modes.rigor: standard` (so `qa.level: standard` — a Logic story's test is required)
|
||||
- `production/epics/combat/` contains `story-003-hit-feedback.md` (no Story Type, as in Case 4) and `story-005-knockback.md`: Story Type Logic, no evidence path stated
|
||||
- No test exists for it: `tests/unit/combat/` does not exist, and no other test file names the slug `knockback` or references the story
|
||||
|
||||
**Input:** `/test-evidence-review combat`
|
||||
|
||||
**Expected behavior:**
|
||||
1. `story-005-knockback` is a Logic story, so a unit test is required; the search of the unit-test root finds none
|
||||
2. `story-005-knockback` is **MISSING** — the review ran and found nothing — and the gap is a BLOCKING item
|
||||
3. `story-003-hit-feedback` is **NOT ASSESSED** (its type cannot be determined), not MISSING
|
||||
4. Overall verdict: **MISSING** — a known failure outranks the story that could not be assessed
|
||||
5. Run ends with **CONCERNS** because a BLOCKING item was found
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The Logic story with no test file is MISSING, not NOT ASSESSED
|
||||
- [ ] The untyped story is NOT ASSESSED, not MISSING — the two are kept apart
|
||||
- [ ] Overall verdict is MISSING, not NOT ASSESSED
|
||||
- [ ] The missing test is listed as BLOCKING and the run ends with CONCERNS
|
||||
|
||||
---
|
||||
|
||||
### Case 9: `qa.level: minimal` — a Logic Story With No Test Is WAIVED; a UI Screenshot Is Not
|
||||
|
||||
**Fixture:**
|
||||
- `project.yaml`: `engine.name: godot`, no `modes` block (so `qa.level: minimal` — tests are waived)
|
||||
- `production/epics/combat/` contains `story-005-knockback.md` (Logic, no test anywhere, as in Case 8) and `story-008-pause-menu.md` (UI, no retained image for it in `production/qa/evidence/`)
|
||||
|
||||
**Input:** `/test-evidence-review combat`
|
||||
|
||||
**Expected behavior:**
|
||||
1. `story-005-knockback` is **WAIVED (qa.level: minimal)** — not MISSING — and raises no BLOCKING item
|
||||
2. `story-008-pause-menu` is **MISSING** and BLOCKING — the retained screenshot a UI story needs is never waived
|
||||
3. Overall verdict: **MISSING**; the report's header shows `qa.level: minimal`
|
||||
4. Run ends with **CONCERNS** because of the UI story alone
|
||||
5. Variant — without the UI story, the overall verdict is **WAIVED**, there are 0 BLOCKING items, and the run ends **COMPLETE**
|
||||
|
||||
**Assertions:**
|
||||
- [ ] The Logic story with no test is WAIVED (qa.level: minimal), not MISSING, and is not listed as BLOCKING
|
||||
- [ ] The UI story with no screenshot is still MISSING and BLOCKING at `qa.level: minimal`
|
||||
- [ ] The waived story is named as waived in the report, not dropped from it
|
||||
- [ ] Variant: an all-waived scope reads WAIVED, with no "must be resolved before `/story-done`" prompt
|
||||
|
||||
---
|
||||
|
||||
## Protocol Compliance
|
||||
|
||||
- [ ] Reads `coding-standards.md` test standards before reviewing test files
|
||||
- [ ] Checks naming, Arrange/Act/Assert structure, determinism, isolation, no hardcoded data
|
||||
- [ ] Does not edit any test files (read-only skill)
|
||||
- [ ] Collects Story Type, evidence path and acceptance criteria with targeted `## Test Evidence` / `## Acceptance Criteria` greps, full-reading a story only when its Test Evidence section is missing or ambiguous
|
||||
- [ ] An empty scope stops with `NOT ASSESSED — no stories in scope`, naming the scope and routing onward
|
||||
- [ ] Each story gets ADEQUATE, INCOMPLETE, MISSING or NOT ASSESSED, and a NOT ASSESSED story states its reason
|
||||
- [ ] Overall verdict is the worst story verdict; NOT ASSESSED outranks ADEQUATE but not INCOMPLETE or MISSING; WAIVED ranks below ADEQUATE
|
||||
- [ ] At `qa.level: minimal` a Logic, Integration or Config/Data story with no test evidence is WAIVED, never MISSING or BLOCKING; Visual/Feel and UI evidence is never waived
|
||||
- [ ] BLOCKING vs ADVISORY per story type comes from `testing.strict.<type>` in the resolved block, falling back to a legacy plain-boolean `testing.strict` value applied to every type, then to the `coding-standards.md` default
|
||||
- [ ] The report is presented before the optional "May I write" for `production/qa/evidence-review-[date].md`
|
||||
- [ ] Does not modify test files or evidence documents
|
||||
- [ ] No director gates are invoked
|
||||
- [ ] Verdict is one of: PASS, WARNINGS, FAIL
|
||||
- [ ] Ends with COMPLETE, or CONCERNS when BLOCKING items were found — except an empty scope, which stops at `NOT ASSESSED — no stories in scope` before any report (Case 3)
|
||||
|
||||
---
|
||||
|
||||
## Coverage Notes
|
||||
|
||||
- Batch review of all test files in `tests/` is not explicitly tested; behavior
|
||||
is assumed to apply the same checks file by file and aggregate the verdict.
|
||||
- The QL-TEST-COVERAGE director gate (which checks test coverage percentage) is
|
||||
a separate concern and is intentionally NOT invoked by this skill.
|
||||
- Integration stories (playtest records in `production/session-logs/`) and
|
||||
Config/Data stories (`production/qa/smoke-*.md`) are not tested here.
|
||||
- The no-argument path (the skill asks which scope to review) is not tested.
|
||||
- The POTENTIALLY STALE freshness flag is not tested separately.
|
||||
|
||||
Reference in New Issue
Block a user