ksh999000/Claude-Code-Game-Studios
1
0
Fork 0
mirror of https://github.com/Donchitos/Claude-Code-Game-Studios.git synced 2026-10-16 10:17:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

Release 1.1.2 — the default path works end to end

Browse Source 
Fix release for 1.1.1: the minimal path end to end, engine commands that
run on Godot, Unity and Unreal, hook warnings you can see, and verdicts
that say when they could not check. Details in CHANGELOG.md; upgrade
steps in UPGRADING.md (v1.1.1 → v1.1.2).

Fixes #67
Fixes #131

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WuJn2FdynEybfL5KC697nJ
This commit is contained in:
Donchitos
2026-09-29 21:32:19 +10:00
co-authored by Claude Opus 5.5
parent 7ed2c3e9c4
commit b21fa0f7f2
383 changed files with 22888 additions and 10618 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CCGS Skill Testing Framework/skills/team/team-qa.md
+201 -85
View File
@@ -2,29 +2,47 @@
## Skill Summary
Orchestrates the QA team through a 7-phase structured testing cycle. Coordinates
qa-lead (strategy, test plan, sign-off report) and qa-tester (test case writing,
bug report writing). Covers scope detection, story classification, QA plan
generation, smoke check gate, test case writing, manual QA execution with bug
filing, and a final sign-off report with an APPROVED / APPROVED WITH CONDITIONS /
NOT APPROVED verdict. Parallel qa-tester spawning is used in Phase 5 for
independent stories.
Orchestrates the QA team through a 6-phase structured testing cycle. Coordinates
qa-lead (strategy, smoke check assessment, sign-off report) and qa-tester (test case
writing, bug report writing). Phase 1 loads scope (sprint file, `feature:` form, or
inferred from session state) and the project stage; Phase 2 classifies stories and
takes the smoke check verdict from the latest `production/qa/smoke-*.md` report
(UNKNOWN when none exists; NOT ASSESSED when the report says so — both continue
with a warning and rule out APPROVED; FAIL stops the cycle); at `qa.level:
minimal`, where tests are waived, a Logic, Integration or Config/Data story
without a test is WAIVED and its evidence is the acceptance-criteria walk in
manual QA; Phase 3 writes the QA plan after
asking; Phase 4 spawns qa-tester with a named output path per manual-QA story —
one spawn per story in parallel at `team.size: studio`, one for the whole group
below it; Phase 5 walks manual QA, files bugs, has the tester retain screenshots
and writes evidence artifacts after asking; Phase 6 produces a sign-off report,
with a row for every story in scope, and a NOT ASSESSED / APPROVED / APPROVED WITH
CONDITIONS / NOT APPROVED verdict. Phase 0 resolves `team.size` and the skill announces the
active set before Phase 1. The orchestrator-level verdict is COMPLETE or BLOCKED.
---
## Static Assertions (Structural)
- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
- [ ] Has ≥2 phase headings
- [ ] Has ≥2 phase headings (Phase 1 through Phase 6)
- [ ] Contains verdict keywords: COMPLETE, BLOCKED
- [ ] Contains verdict keywords for sign-off report: APPROVED, APPROVED WITH CONDITIONS, NOT APPROVED
- [ ] Contains "May I write" language for both the QA plan and the sign-off report
- [ ] Contains sign-off verdict keywords: NOT ASSESSED, APPROVED, APPROVED WITH CONDITIONS, NOT APPROVED
- [ ] Contains "May I write" language for every orchestrator write: the QA plan (Phase 3), the evidence artifacts (Phase 5) and the sign-off report (Phase 6)
- [ ] Has an Error Recovery Protocol section
- [ ] Uses `AskUserQuestion` at phase transitions to capture user approval before proceeding
- [ ] Phase 4 (smoke check) is a hard gate: FAIL stops the cycle
- [ ] Bug reports are written to `production/qa/bugs/` with `BUG-[NNN]-[short-slug].md` naming
- [ ] Next-step guidance differs by verdict (APPROVED / APPROVED WITH CONDITIONS / NOT APPROVED)
- [ ] Independent qa-tester tasks in Phase 5 are spawned in parallel
- [ ] Phase 2 takes the smoke verdict from the most recent `production/qa/smoke-*.md` report, and a FAIL stops the cycle before Phase 3
- [ ] Phase 4 names each qa-tester's output path: `production/qa/test-cases/[story-slug]-cases.md`
- [ ] Bug reports are written to `production/qa/bugs/` with `BUG-[NNNN].md` naming — `/bug-report`'s numbering, so `/bug-report verify` finds them
- [ ] Visual/Feel and UI stories that pass get a retained screenshot of each screen or effect touched under `production/qa/evidence/[story-slug]/`, confirmed on disk, and an evidence artifact at `production/qa/evidence/[story-slug]-evidence.md` that references it — the screenshot closes a UI story; a Visual/Feel story also needs the lead sign-off
- [ ] Next-step guidance differs by verdict (NOT ASSESSED / APPROVED / APPROVED WITH CONDITIONS / NOT APPROVED)
- [ ] Phase 0 resolves `team.size`, and the skill requires a one-line active-set announcement before Phase 1
- [ ] Phase 0 resolves `qa.level`; at `minimal` a Logic, Integration or Config/Data story without a test gets its acceptance criteria walked in Phase 4–5 — tests WAIVED, never counted as missing evidence
- [ ] The Phase 2 smoke verdict list includes NOT ASSESSED, which continues with a warning like UNKNOWN and makes the sign-off NOT ASSESSED
- [ ] Phase 1 reads the Open bugs in scope from `production/qa/bugs/` (open unless `Closed` or `Verified Fixed`) and passes them into the Phase 2 and Phase 6 briefs
- [ ] The Phase 6 brief carries the evidence and the verdict rules and names no verdict; the orchestrator checks the qa-lead's verdict against the rules before writing
- [ ] Phase 4 spawns one qa-tester per story, in parallel, at `team.size: studio`; at `small` and `individual` one qa-tester spawn covers the whole group
- [ ] The Phase 6 sign-off gives every story in scope a Test Coverage Summary row — BLOCKED and unexecuted stories included — and its template shows a BLOCKED row
---
@@ -33,30 +51,33 @@ independent stories.
### Case 1: Happy Path — All stories pass manual QA, APPROVED verdict
**Fixture:**
- `production/sprints/sprint-03/` exists with 4 story files
- Stories are a mix of types: 1 Logic, 1 Integration, 2 Visual/Feel
- Resolved config block: `team.size: small`, `automation: collaborative`
- `production/sprints/sprint-03.md` lists 4 stories whose story files exist
- Stories are a mix of types: 1 Logic (its unit test exists and passes), 1 Integration (no automated test), 2 Visual/Feel
- All stories have acceptance criteria populated
- `tests/smoke/` contains a smoke test list; all items are verifiable
- No existing bugs in `production/qa/bugs/`
- The most recent `production/qa/smoke-[date].md` report has verdict PASS
- `production/qa/bugs/` contains no bugs
**Input:** `/team-qa sprint-03`
**Expected behavior:**
1. Phase 1: Reads all story files in `production/sprints/sprint-03/`; reads `production/stage.txt`; reports "Found 4 stories. Current stage: [stage]. Ready to begin QA strategy?"
2. Phase 2: Spawns `qa-lead` via Task; produces strategy table classifying all 4 stories; no blockers flagged; presents to user; AskUserQuestion: user selects "Looks good — proceed to test plan"
3. Phase 3: Produces QA plan document; asks "May I write the QA plan to `production/qa/qa-plan-sprint-03-[date].md`?"; writes after approval
4. Phase 4: Spawns `qa-lead` via Task; reviews `tests/smoke/`; returns PASS; reports "Smoke check passed. Proceeding to test case writing."
5. Phase 5: Spawns `qa-tester` via Task for each Visual/Feel and Integration story (2–3 stories); run in parallel; presents test cases grouped by story; AskUserQuestion per group; user approves
6. Phase 6: Walks through each approved story; user marks all as PASS; result summary: "Stories PASS: 4, FAIL: 0, BLOCKED: 0"
7. Phase 7: Spawns `qa-lead` via Task to produce sign-off report; report shows all stories PASS; no bugs filed; Verdict: APPROVED; asks "May I write this QA sign-off report to `production/qa/qa-signoff-sprint-03-[date].md`?"; writes after approval
1. Before any spawn, one line announces the active set for `team.size: small`
2. Phase 1: Globs `production/sprints/sprint-*.md`, matches sprint number 3 (`sprint-03.md`) and reads it; reads `project.stage` from `project.yaml` (fallback `production/stage.txt`); reads the open bugs in scope (none on file); reports "QA cycle starting for sprint-03. Found 4 stories. Open bugs in scope: none on file. Current stage: [stage]. Ready to begin QA strategy?"
3. Phase 2: Spawns `qa-lead` via Agent; classifies all 4 stories in the strategy table; reads the smoke report and records `Smoke Check: PASS — source: production/qa/smoke-[date].md` without re-interviewing the user; presents the strategy; `AskUserQuestion` "QA Strategy Review": user selects "Looks good — proceed to test plan"
4. Phase 3: Produces QA plan (scope, classification table, automated/manual scope, entry and exit criteria); asks "May I write the QA plan to `production/qa/qa-plan-sprint-03-[date].md`?"; writes after approval
5. Phase 4: Spawns one `qa-tester` via Agent for the Integration and both Visual/Feel stories (per-story parallel spawns are the `studio` shape), its prompt naming `production/qa/test-cases/[story-slug]-cases.md` for each story; test cases presented grouped by story; `AskUserQuestion` per group; user approves
6. Phase 5: `AskUserQuestion` per story (PASS / PASS WITH NOTES / FAIL / BLOCKED); user marks all as PASS; for each Visual/Feel story the tester saves a screenshot under `production/qa/evidence/[story-slug]/` and the orchestrator confirms it is on disk; after "May I write the evidence for [story titles] to `production/qa/evidence/`?" it writes `production/qa/evidence/[story-slug]-evidence.md`, referencing the screenshots, with sign-off rows left unchecked unless signed off in this session; result summary: "Stories PASS: 4, PASS WITH NOTES: 0, FAIL: 0, BLOCKED: 0"
7. Phase 6: Spawns `qa-lead` via Agent to produce sign-off report; every story has executed evidence; no bugs filed; Verdict: APPROVED; asks "May I write this QA sign-off report to `production/qa/qa-signoff-sprint-03-[date].md`?"; writes after approval
8. Verdict: COMPLETE — QA cycle finished
**Assertions:**
- [ ] Active-set line naming `team.size: small` appears before the first agent is spawned
- [ ] Phase 1 correctly counts and reports 4 stories with current stage
- [ ] Strategy table in Phase 2 classifies all 4 stories with correct types
- [ ] Smoke check verdict is taken from the existing `production/qa/smoke-*.md` report and its source is shown
- [ ] QA plan written only after "May I write?" approval
- [ ] Smoke check PASS allows pipeline to continue without user intervention
- [ ] Phase 5 qa-tester tasks for independent stories are issued in parallel
- [ ] Phase 4 spawns a single qa-tester at `team.size: small`, and its prompt names each story's `production/qa/test-cases/` path
- [ ] Each passing Visual/Feel story has a retained screenshot on disk under `production/qa/evidence/[story-slug]/`, and its evidence artifact references it and is written only after the "May I write" ask
- [ ] Sign-off report includes Test Coverage Summary table and Verdict: APPROVED
- [ ] Sign-off report written only after "May I write?" approval
- [ ] Verdict: COMPLETE appears in final output
@@ -64,59 +85,62 @@ independent stories.
---
### Case 2: Smoke Check Fail — QA cycle stops at Phase 4
### Case 2: Smoke Check Fail — QA cycle stops at Phase 2
**Fixture:**
- `production/sprints/sprint-04/` exists with 3 story files
- `tests/smoke/` exists with 5 smoke test items; 2 items cannot be verified (e.g., build is unstable, core navigation broken)
- Resolved config block: `team.size: small`, `automation: collaborative`
- `production/sprints/sprint-04.md` lists 3 stories
- The most recent `production/qa/smoke-[date].md` report has verdict FAIL with 2 listed failures (build unstable on launch, core navigation broken)
**Input:** `/team-qa sprint-04`
**Expected behavior:**
1. Phases 1–3 complete normally; QA plan is written
2. Phase 4: Spawns `qa-lead` via Task; smoke check returns FAIL; two specific failures are identified
3. Skill reports: "Smoke check failed. QA cannot begin until these issues are resolved: [list of 2 failures]. Fix them and re-run `/smoke-check`, or re-run `/team-qa` once resolved."
4. Skill stops immediately after Phase 4 — no Phase 5, 6, or 7 is executed
5. No sign-off report is produced; no "May I write?" for a sign-off is issued
1. Phase 1 completes normally
2. Phase 2: Spawns `qa-lead` via Agent; qa-lead reads the smoke report and records `Smoke Check: FAIL` with both failures listed prominently
3. `AskUserQuestion` "QA Strategy Review" is presented, including the option "Smoke check failed — fix issues and re-run /team-qa"
4. Skill does not proceed to Phase 3: it surfaces the failures from the smoke report and stops — the user must fix them, re-run `/smoke-check sprint`, and then re-run `/team-qa`
5. No QA plan, test cases, manual QA or sign-off report is produced; a partial report is given
6. Verdict: BLOCKED — smoke check failed
**Assertions:**
- [ ] Smoke check FAIL causes the pipeline to halt at Phase 4 — Phases 5, 6, 7 are NOT executed
- [ ] Smoke check verdict comes from the `production/qa/smoke-*.md` report — qa-lead does not re-derive it or re-interview the user
- [ ] Smoke check FAIL halts the pipeline at Phase 2 — Phases 3, 4, 5, 6 are NOT executed
- [ ] Failure list is shown to the user explicitly (not summarized vaguely)
- [ ] Skill recommends `/smoke-check` and `/team-qa` re-run as remediation steps
- [ ] No QA sign-off report is written or offered
- [ ] Skill does NOT produce a COMPLETE verdict
- [ ] Any QA plan already written in Phase 3 is preserved (not deleted)
- [ ] Skill recommends `/smoke-check sprint` and a `/team-qa` re-run as remediation steps
- [ ] No QA plan and no QA sign-off report is written or offered
- [ ] Orchestrator verdict is BLOCKED, not COMPLETE
---
### Case 3: Bug Found — Visual/Feel story fails manual QA, bug report filed
**Fixture:**
- `production/sprints/sprint-05/` exists with 2 story files: 1 Logic (passes automated tests), 1 Visual/Feel
- `tests/smoke/` smoke check passes
- Resolved config block: `team.size: small`, `automation: collaborative`
- `production/sprints/sprint-05.md` lists 2 stories: 1 Logic (its automated test passes), 1 Visual/Feel
- The most recent `production/qa/smoke-[date].md` report has verdict PASS
- The Visual/Feel story's animation timing is visibly wrong (acceptance criterion not met)
- `production/qa/bugs/` directory exists (empty or with existing bugs)
- `production/qa/bugs/` already contains `BUG-0001.md` and `BUG-0002.md`
**Input:** `/team-qa sprint-05`
**Expected behavior:**
1. Phases 1–5 complete normally; test cases are written for the Visual/Feel story
2. Phase 6: User marks Visual/Feel story as FAIL; AskUserQuestion collects failure description: "Animation plays at 2x speed — jitter visible on every loop"
3. Phase 6: Spawns `qa-tester` via Task to write a formal bug report; bug report written to `production/qa/bugs/BUG-001-animation-speed-jitter.md` (or next increment if bugs exist); report includes severity field
4. Result summary: "Stories PASS: 1, FAIL: 1 — bugs filed: BUG-001"
5. Phase 7: Spawns `qa-lead` to produce sign-off report; Bugs Found table lists BUG-001 with severity and status Open; Verdict: NOT APPROVED (S1/S2 bug open, or FAIL without documented workaround)
1. Phases 1–4 complete normally; test cases are written for the Visual/Feel story
2. Phase 5: User marks Visual/Feel story as FAIL; `AskUserQuestion` collects failure description: "Animation plays at 2x speed — jitter visible on every loop"
3. Phase 5: Spawns `qa-tester` via Agent to write a formal bug report at `production/qa/bugs/BUG-0003.md`; report includes severity field
4. Result summary: "Stories PASS: 1, PASS WITH NOTES: 0, FAIL: 1 — bugs filed: BUG-0003, BLOCKED: 0"
5. Phase 6: Spawns `qa-lead` to produce sign-off report; every story has executed evidence; Bugs Found table lists BUG-0003 with story, severity and status Open; Verdict: NOT APPROVED (a story FAILed without documented workaround)
6. Sign-off report write is offered; writes after approval
7. Next step: "Resolve S1/S2 bugs and re-run `/team-qa` or targeted manual QA before advancing."
**Assertions:**
- [ ] FAIL result in Phase 6 triggers AskUserQuestion to collect the failure description before the bug report is written
- [ ] `qa-tester` is spawned via Task to write the bug report — orchestrator does not write it directly
- [ ] Bug report follows naming convention: `BUG-[NNN]-[short-slug].md` in `production/qa/bugs/`
- [ ] Bug report NNN is incremented correctly from existing bugs in the directory
- [ ] Phase 7 sign-off report Bugs Found table includes the bug ID, story name, severity, and status
- [ ] FAIL result in Phase 5 triggers `AskUserQuestion` to collect the failure description before the bug report is written
- [ ] `qa-tester` is spawned via Agent to write the bug report — orchestrator does not write it directly
- [ ] Bug report follows `/bug-report`'s naming: `BUG-[NNNN].md` in `production/qa/bugs/`
- [ ] The ID is one past the highest existing bug, zero-padded to four digits (BUG-0003)
- [ ] Phase 6 sign-off report Bugs Found table includes the bug ID, story name, severity, and status
- [ ] Verdict in sign-off report is NOT APPROVED
- [ ] Next step explicitly mentions re-running `/team-qa`
- [ ] Verdict: COMPLETE is still issued by the orchestrator (the QA cycle finished — the verdict is NOT APPROVED, but the skill completed its pipeline)
- [ ] Verdict: COMPLETE is still issued by the orchestrator (the QA cycle finished — the sign-off verdict is NOT APPROVED, but the skill completed its pipeline)
---
@@ -125,27 +149,29 @@ independent stories.
**Fixture (variant A — state files present):**
- `production/session-state/active.md` exists and contains a reference to `sprint-06`
- `production/sprint-status.yaml` exists and identifies `sprint-06` as active
- `production/sprints/sprint-06.md` exists
**Fixture (variant B — state files absent):**
**Fixture (variant B — no sprint, as at `rigor: minimal`):**
- `production/session-state/active.md` does NOT exist
- `production/sprint-status.yaml` does NOT exist
- No sprint files exist; epics exist under `production/epics/`
**Input:** `/team-qa` (no argument)
**Expected behavior (variant A):**
1. Phase 1: No argument provided; reads `production/session-state/active.md`; reads `production/sprint-status.yaml`
2. Detects `sprint-06` as the active sprint from both sources
3. Proceeds as if `/team-qa sprint-06` was the input; reports "No sprint argument provided — inferred sprint-06 from session state. Found [N] stories."
1. Phase 1: No argument provided; reads `production/session-state/active.md` and `production/sprint-status.yaml`
2. Detects `sprint-06` as the active sprint
3. Proceeds as if `/team-qa sprint-06` was the input; the Phase 1 report line names it: "QA cycle starting for sprint-06. Found [N] stories. Open bugs in scope: [N or none on file]. Current stage: [stage]. Ready to begin QA strategy?"
**Expected behavior (variant B):**
1. Phase 1: No argument provided; attempts to read `production/session-state/active.md` — file missing; attempts to read `production/sprint-status.yaml` — file missing
2. Cannot infer sprint; uses AskUserQuestion: "Which sprint or feature should QA cover?" with options to type a sprint identifier or cancel
1. Phase 1: No argument provided; attempts to read both state files — both missing; no sprint exists
2. Asks the user which epic to cover, then scopes the run as `feature: [epic-slug]` — it does not invent a sprint
**Assertions:**
- [ ] Skill does NOT default to a hardcoded sprint name when no argument is provided
- [ ] Skill reads both `production/session-state/active.md` AND `production/sprint-status.yaml` before asking the user (variant A)
- [ ] When both state files are absent, skill uses AskUserQuestion rather than guessing (variant B)
- [ ] Inferred sprint is reported to the user before proceeding (variant A transparency)
- [ ] When no sprint can be inferred, skill asks which epic to cover and uses the `feature: [epic-slug]` form rather than guessing (variant B)
- [ ] The inferred sprint is named in the Phase 1 report line before Phase 2 begins (variant A)
- [ ] Skill does NOT error out when state files are missing — it falls back to asking (variant B)
---
@@ -153,52 +179,142 @@ independent stories.
### Case 5: Mixed Results — Some PASS, one FAIL with S1 bug, one BLOCKED
**Fixture:**
- `production/sprints/sprint-07/` exists with 4 story files
- Smoke check passes
- Resolved config block: `team.size: small`, `automation: collaborative`
- `production/sprints/sprint-07.md` lists 4 stories
- The most recent `production/qa/smoke-[date].md` report has verdict PASS
- Story A (Logic): automated test passes — PASS
- Story B (UI): manual QA — PASS WITH NOTES (minor text overflow)
- Story C (Visual/Feel): manual QA — FAIL; tester identifies S1 crash on ability activation
- Story D (Integration): cannot test — BLOCKED (dependency system not yet implemented)
- `production/qa/bugs/` is empty
**Input:** `/team-qa sprint-07`
**Expected behavior:**
1. Phases 1–5 proceed; Phase 5 test cases cover stories B, C, D
2. Phase 6: User marks Story A as implicitly PASS (automated); Story B: PASS WITH NOTES; Story C: FAIL; Story D: BLOCKED
3. After Story C FAIL: qa-tester spawned to write bug report `BUG-001-crash-ability-activation.md` with S1 severity
4. Result summary presented: "Stories PASS: 1, PASS WITH NOTES: 1, FAIL: 1 — bugs filed: BUG-001 (S1), BLOCKED: 1"
5. Phase 7: qa-lead produces sign-off report covering all 4 stories; BUG-001 listed as S1/Open; Story D listed as BLOCKED; Verdict: NOT APPROVED
6. Sign-off report written after "May I write?" approval
7. Next step: "Resolve S1/S2 bugs and re-run `/team-qa` or targeted manual QA before advancing."
1. Phases 1–4 proceed; Phase 4 test cases cover stories B, C, D
2. Phase 5: Story A is PASS from its automated test; user marks Story B: PASS WITH NOTES; Story C: FAIL; Story D: BLOCKED
3. Story B (UI) gets a retained screenshot of each screen it touched, saved by the tester under `production/qa/evidence/[story-slug]/` and confirmed on disk; its evidence artifact at `production/qa/evidence/[story-slug]-evidence.md` references them and is written only after the "May I write" ask
4. After Story C FAIL: failure description collected; qa-tester spawned to write bug report `BUG-0001.md` with S1 severity
5. Result summary presented: "Stories PASS: 1, PASS WITH NOTES: 1, FAIL: 1 — bugs filed: BUG-0001, BLOCKED: 1"
6. Phase 6: qa-lead produces sign-off report covering all 4 stories; BUG-0001 listed as S1/Open; Story D listed as BLOCKED with no executed evidence; Verdict: NOT APPROVED — the open S1 is a known failure and is reported as such, not demoted to NOT ASSESSED by the unexecuted Story D
7. Sign-off report written after "May I write?" approval
8. Next step: "Resolve S1/S2 bugs and re-run `/team-qa` or targeted manual QA before advancing."
**Assertions:**
- [ ] All 4 stories appear in the Phase 7 sign-off report Test Coverage Summary table — none are silently omitted
- [ ] Story D (BLOCKED) is listed in the report with a BLOCKED status, not silently dropped
- [ ] S1 bug causes Verdict: NOT APPROVED regardless of the other stories passing
- [ ] All 4 stories appear in the Phase 6 sign-off report Test Coverage Summary table — none are silently omitted
- [ ] Story D (BLOCKED) is listed in the report with a BLOCKED status and named as lacking executed evidence, not silently dropped
- [ ] S1 bug causes Verdict: NOT APPROVED regardless of the other stories passing, and regardless of Story D being unexecuted
- [ ] PASS WITH NOTES stories do not downgrade to FAIL — they are tracked separately
- [ ] BUG-001 severity is listed as S1 in the Bugs Found table
- [ ] Story B's evidence is its retained screenshots — a UI story needs no lead sign-off to close
- [ ] BUG-0001 severity is listed as S1 in the Bugs Found table
- [ ] Partial results are preserved — the sign-off report is still produced even with failures and blocks
- [ ] Verdict: COMPLETE is issued by the orchestrator (pipeline completed); sign-off verdict is NOT APPROVED
---
### Case 6: `qa.level: minimal` — a Logic story with no test is evidenced by its acceptance-criteria walk
**Fixture:**
- Resolved config block: `team.size: small`, `automation: collaborative`, `qa.level: minimal (rigor:minimal)`
- No sprint files and no `production/sprint-status.yaml`; `production/epics/combat/` holds 2 story files, both `In Review`: story-001 (Logic, no test file anywhere) and story-002 (UI)
- The most recent `production/qa/smoke-[date].md` report has verdict PASS (its automated row WAIVED)
- `production/qa/bugs/` contains no bugs
**Input:** `/team-qa feature: combat`
**Expected behavior:**
1. Phase 2: qa-lead's strategy table marks story-001's Automated Required as `WAIVED` — tests are waived at `qa.level: minimal` — and puts it in manual QA; it is not flagged as missing test evidence or as a blocker
2. Phase 3: the QA plan lists no automated test requirements; its entry criteria take the stories' status from the story files (feature scope), not from `production/sprint-status.yaml`
3. Phase 4: test cases are written for both stories — story-001's walk its acceptance criteria
4. Phase 5: the tester answers PASS for both; story-002's retained screenshot is saved and confirmed on disk
5. Phase 6: the Test Coverage Summary shows story-001 as `WAIVED (qa.level: minimal)` under Auto Test with its criteria walked; every story has executed evidence, so the verdict is APPROVED
**Assertions:**
- [ ] story-001 is never reported as missing its test, and its acceptance-criteria walk is its executed evidence
- [ ] The sign-off verdict is APPROVED, not NOT ASSESSED
- [ ] The UI story still needs its retained screenshot — `qa.level: minimal` waives tests, not the look
- [ ] Entry criteria at `feature:` scope read the story files' Status, not `production/sprint-status.yaml`
- [ ] No next step tells the user to write Logic tests
---
### Case 7: Smoke report NOT ASSESSED — continue with a warning, sign-off NOT ASSESSED
**Fixture:**
- Resolved config block: `team.size: small`, `automation: collaborative`, `qa.level: standard`
- `production/sprints/sprint-008.md` lists 2 stories: 1 Logic (its unit test exists and passes) and 1 Visual/Feel
- The most recent `production/qa/smoke-[date].md` report has verdict NOT ASSESSED — Batch 3 was offered and skipped
- `production/qa/bugs/` contains no bugs
**Input:** `/team-qa sprint-8`
**Expected behavior:**
1. Phase 1 matches `sprint-008.md` by its number
2. Phase 2: records `Smoke Check: NOT ASSESSED — Batch 3 skipped — source: production/qa/smoke-[date].md` and warns that the last smoke check could not assess it, recommending a `/smoke-check sprint` re-run; the cycle continues
3. Phases 3–5 run; both stories PASS and the Visual/Feel story's evidence is retained
4. Phase 6: every story has executed evidence, but the smoke check was NOT ASSESSED, so the sign-off verdict is **NOT ASSESSED** — never APPROVED
**Assertions:**
- [ ] The NOT ASSESSED smoke verdict is taken from the report and shown with its source — not read as PASS or UNKNOWN
- [ ] The cycle does not stop at Phase 2 on NOT ASSESSED (only FAIL stops it)
- [ ] The sign-off verdict is NOT ASSESSED even though every story passed
- [ ] The next step names re-running `/smoke-check` before `/team-qa`
---
### Case 8: An open S1 already on file — known before sign-off, and the brief names no verdict
**Fixture:**
- Resolved config block: `team.size: small`, `automation: collaborative`, `qa.level: standard`
- `production/sprints/sprint-009.md` lists 2 stories in the Combat system: 1 Logic (its unit test passes) and 1 UI
- `production/qa/bugs/BUG-0004.md`: System Combat, `**Severity**: S1-Critical`, `**Status**: Open` — filed before this cycle; `BUG-0002.md` (Combat, S2) has `**Status**: Closed`
- The most recent `production/qa/smoke-[date].md` report has verdict PASS
**Input:** `/team-qa sprint-9`
**Expected behavior:**
1. Phase 1 greps the bug files and reports "Open bugs in scope: 1 — S1: 1"; the closed BUG-0002 is not counted
2. Phase 2's brief to qa-lead carries BUG-0004, and the strategy lists it under **Open bugs in scope** as a known failure
3. Phases 3–5 run; both stories PASS and the UI story's screenshots are retained
4. Phase 6's brief passes the strategy, the smoke verdict, both stories' results, BUG-0004 and the verdict rules — it names no verdict, pre-fills no `### Verdict:` line and does not say which fact decides it
5. The qa-lead returns NOT APPROVED (an open S1 bug in scope); the Bugs Found table lists BUG-0004; the orchestrator checks that verdict against the rules, then asks to write
**Assertions:**
- [ ] BUG-0004 is known from Phase 1 and appears in the Phase 2 strategy — not first discovered at sign-off, and never missed
- [ ] The sign-off verdict is NOT APPROVED although every story passed — never NOT ASSESSED or APPROVED
- [ ] The Phase 6 brief names no verdict and no deciding fact; the qa-lead reaches NOT APPROVED from the rules
- [ ] A closed bug is not counted as open
---
## Protocol Compliance
- [ ] `AskUserQuestion` used at Phase 2 (strategy review), Phase 5 (test case approval per group), and Phase 6 (per-story manual QA result)
- [ ] Phase 4 smoke check is a hard gate: FAIL halts the pipeline at Phase 4 with no exceptions
- [ ] "May I write?" asked separately for QA plan (Phase 3) and sign-off report (Phase 7)
- [ ] Bug reports are always written by `qa-tester` via Task — orchestrator does not write directly
- [ ] Phase 5 qa-tester tasks for independent stories are issued in parallel where possible
- [ ] A manual-QA result is recorded only as the tester gave it — never filled in for them; an unanswered story stays unexecuted
- [ ] In Phase 6 the qa-lead returns the sign-off text and writes no file; the orchestrator writes it after the ask
- [ ] A bug filed in Phase 5 uses `/bug-report`'s Severity and Priority labels exactly
- [ ] At the default `team.size: individual`, the run opens with an active-set line naming qa-tester as the always-active agent and stating that qa-lead is spawned only at phase gates (the decision points the pipeline lists, whatever the `automation` mode) — before any agent is spawned
- [ ] `AskUserQuestion` used at Phase 2 (strategy review), Phase 4 (test case approval per group), and Phase 5 (per-story manual QA result and failure description)
- [ ] Phase 2 smoke check is a hard gate: FAIL halts the pipeline before Phase 3
- [ ] When no `production/qa/smoke-*.md` report exists, the smoke status is UNKNOWN, the warning "No smoke check report found. Recommend running `/smoke-check sprint` before QA. Proceeding with caution." is shown, and the cycle continues
- [ ] A smoke report whose verdict is NOT ASSESSED is surfaced with what did not run, and the cycle continues as for UNKNOWN
- [ ] With no smoke report (UNKNOWN), or one that is NOT ASSESSED, the sign-off verdict is NOT ASSESSED even when every story passes — never APPROVED or APPROVED WITH CONDITIONS — unless an open S1/S2 bug, or a FAIL without a documented workaround, makes it NOT APPROVED
- [ ] "May I write?" is asked before every orchestrator write: the QA plan (Phase 3), each group's evidence artifacts (Phase 5) and the sign-off report (Phase 6)
- [ ] Bug reports are always written by `qa-tester` via Agent — orchestrator does not write them directly
- [ ] Every Phase 4 qa-tester prompt names each story's output path; per-story parallel spawns happen only at `team.size: studio`
- [ ] Evidence artifacts never carry pre-ticked sign-off rows for a sign-off that did not happen in the session
- [ ] Error recovery: any BLOCKED agent is surfaced immediately with AskUserQuestion options
- [ ] Partial report always produced — no work is discarded because one story failed or blocked
- [ ] Sign-off verdict rules are strictly applied: any S1/S2 bug open = NOT APPROVED; no exceptions
- [ ] Orchestrator-level Verdict: COMPLETE is distinct from the sign-off report's APPROVED/NOT APPROVED verdict
- [ ] Open bugs already on file are read in Phase 1 and reach both qa-lead briefs; no brief to qa-lead names the sign-off verdict
- [ ] Sign-off verdict rules are strictly applied: any S1/S2 bug open = NOT APPROVED; APPROVED and APPROVED WITH CONDITIONS require executed evidence for every story in scope
- [ ] Orchestrator-level Verdict: COMPLETE is distinct from the sign-off report's verdict
---
## Coverage Notes
- The "APPROVED WITH CONDITIONS" verdict path (S3/S4 bugs, PASS WITH NOTES) is covered implicitly by Case 5's PASS WITH NOTES story (Story B) — if no S1/S2 bugs existed, that case would produce APPROVED WITH CONDITIONS. A dedicated case is not required as the verdict logic is table-driven.
- The `feature: [system-name]` argument form is not separately tested — it follows the same Phase 1 logic as the sprint form, using glob instead of directory read. The no-argument inference path (Case 4) provides sufficient coverage of the detection logic.
- The "APPROVED WITH CONDITIONS" verdict path (S3/S4 bugs, PASS WITH NOTES with every story executed) is not given a dedicated case; Case 5's Story B would produce it if no S1/S2 bug were open and Story D had been executed.
- The NOT ASSESSED verdict with no known failure is covered for a NOT ASSESSED smoke report (Case 7); one story BLOCKED with the rest PASS, or no smoke report at all, is not given a dedicated case — the smoke-UNKNOWN rule is asserted in Protocol Compliance.
- The `studio` shape (one qa-tester per story, in parallel) is asserted structurally but not given a case; Cases 1, 3 and 5 run at `small`.
- The `feature: [system-name]` argument form is exercised by Case 6 — it follows the same Phase 1 logic as the sprint form, using glob instead of the sprint file. Case 4 variant B exercises the epic fallback.
- Logic stories with passing automated tests do not need manual QA — this is validated implicitly by Case 5 (Story A) where the Logic story receives no manual QA phase.
- Parallel qa-tester spawning in Phase 5 is validated implicitly by Case 1 (multiple Visual/Feel stories issued simultaneously); no dedicated parallelism case is required beyond the Static Assertions check.
- The session-state append to `production/session-state/active.md` after the final phase is not asserted.