Claude-Code-Game-Studios/CCGS Skill Testing Framework/skills/team/team-qa.md

Skill Test Spec: /team-qa

Skill Summary

Orchestrates the QA team through a 6-phase structured testing cycle. Coordinates qa-lead (strategy, smoke check assessment, sign-off report) and qa-tester (test case writing, bug report writing). Phase 1 loads scope (sprint file, feature: form, or inferred from session state) and the project stage; Phase 2 classifies stories and takes the smoke check verdict from the latest production/qa/smoke-*.md report (UNKNOWN when none exists; NOT ASSESSED when the report says so — both continue with a warning and rule out APPROVED; FAIL stops the cycle); at qa.level: minimal, where tests are waived, a Logic, Integration or Config/Data story without a test is WAIVED and its evidence is the acceptance-criteria walk in manual QA; Phase 3 writes the QA plan after asking; Phase 4 spawns qa-tester with a named output path per manual-QA story — one spawn per story in parallel at team.size: studio, one for the whole group below it; Phase 5 walks manual QA, files bugs, has the tester retain screenshots and writes evidence artifacts after asking; Phase 6 produces a sign-off report, with a row for every story in scope, and a NOT ASSESSED / APPROVED / APPROVED WITH CONDITIONS / NOT APPROVED verdict. Phase 0 resolves team.size and the skill announces the active set before Phase 1. The orchestrator-level verdict is COMPLETE or BLOCKED.

---

Static Assertions (Structural)

• Has required frontmatter fields: name, description, argument-hint, user-invocable, allowed-tools
• Has ≥2 phase headings (Phase 1 through Phase 6)
• Contains verdict keywords: COMPLETE, BLOCKED
• Contains sign-off verdict keywords: NOT ASSESSED, APPROVED, APPROVED WITH CONDITIONS, NOT APPROVED
• Contains "May I write" language for every orchestrator write: the QA plan (Phase 3), the evidence artifacts (Phase 5) and the sign-off report (Phase 6)
• Has an Error Recovery Protocol section
• Uses AskUserQuestion at phase transitions to capture user approval before proceeding
• Phase 2 takes the smoke verdict from the most recent production/qa/smoke-*.md report, and a FAIL stops the cycle before Phase 3
• Phase 4 names each qa-tester's output path: production/qa/test-cases/[story-slug]-cases.md
• Bug reports are written to production/qa/bugs/ with BUG-[NNNN].md naming — /bug-report's numbering, so /bug-report verify finds them
• Visual/Feel and UI stories that pass get a retained screenshot of each screen or effect touched under production/qa/evidence/[story-slug]/, confirmed on disk, and an evidence artifact at production/qa/evidence/[story-slug]-evidence.md that references it — the screenshot closes a UI story; a Visual/Feel story also needs the lead sign-off
• Next-step guidance differs by verdict (NOT ASSESSED / APPROVED / APPROVED WITH CONDITIONS / NOT APPROVED)
• Phase 0 resolves team.size, and the skill requires a one-line active-set announcement before Phase 1
• Phase 0 resolves qa.level; at minimal a Logic, Integration or Config/Data story without a test gets its acceptance criteria walked in Phase 4–5 — tests WAIVED, never counted as missing evidence
• The Phase 2 smoke verdict list includes NOT ASSESSED, which continues with a warning like UNKNOWN and makes the sign-off NOT ASSESSED
• Phase 1 reads the Open bugs in scope from production/qa/bugs/ (open unless Closed or Verified Fixed) and passes them into the Phase 2 and Phase 6 briefs
• The Phase 6 brief carries the evidence and the verdict rules and names no verdict; the orchestrator checks the qa-lead's verdict against the rules before writing
• Phase 4 spawns one qa-tester per story, in parallel, at team.size: studio; at small and individual one qa-tester spawn covers the whole group
• The Phase 6 sign-off gives every story in scope a Test Coverage Summary row — BLOCKED and unexecuted stories included — and its template shows a BLOCKED row

---

Test Cases

Case 1: Happy Path — All stories pass manual QA, APPROVED verdict

Fixture:

• Resolved config block: team.size: small, automation: collaborative
• production/sprints/sprint-03.md lists 4 stories whose story files exist
• Stories are a mix of types: 1 Logic (its unit test exists and passes), 1 Integration (no automated test), 2 Visual/Feel
• All stories have acceptance criteria populated
• The most recent production/qa/smoke-[date].md report has verdict PASS
• production/qa/bugs/ contains no bugs

Input: /team-qa sprint-03

Expected behavior:

1. Before any spawn, one line announces the active set for team.size: small
2. Phase 1: Globs production/sprints/sprint-*.md, matches sprint number 3 (sprint-03.md) and reads it; reads project.stage from project.yaml (fallback production/stage.txt); reads the open bugs in scope (none on file); reports "QA cycle starting for sprint-03. Found 4 stories. Open bugs in scope: none on file. Current stage: [stage]. Ready to begin QA strategy?"
3. Phase 2: Spawns qa-lead via Agent; classifies all 4 stories in the strategy table; reads the smoke report and records Smoke Check: PASS — source: production/qa/smoke-[date].md without re-interviewing the user; presents the strategy; AskUserQuestion "QA Strategy Review": user selects "Looks good — proceed to test plan"
4. Phase 3: Produces QA plan (scope, classification table, automated/manual scope, entry and exit criteria); asks "May I write the QA plan to production/qa/qa-plan-sprint-03-[date].md?"; writes after approval
5. Phase 4: Spawns one qa-tester via Agent for the Integration and both Visual/Feel stories (per-story parallel spawns are the studio shape), its prompt naming production/qa/test-cases/[story-slug]-cases.md for each story; test cases presented grouped by story; AskUserQuestion per group; user approves
6. Phase 5: AskUserQuestion per story (PASS / PASS WITH NOTES / FAIL / BLOCKED); user marks all as PASS; for each Visual/Feel story the tester saves a screenshot under production/qa/evidence/[story-slug]/ and the orchestrator confirms it is on disk; after "May I write the evidence for [story titles] to production/qa/evidence/?" it writes production/qa/evidence/[story-slug]-evidence.md, referencing the screenshots, with sign-off rows left unchecked unless signed off in this session; result summary: "Stories PASS: 4, PASS WITH NOTES: 0, FAIL: 0, BLOCKED: 0"
7. Phase 6: Spawns qa-lead via Agent to produce sign-off report; every story has executed evidence; no bugs filed; Verdict: APPROVED; asks "May I write this QA sign-off report to production/qa/qa-signoff-sprint-03-[date].md?"; writes after approval
8. Verdict: COMPLETE — QA cycle finished

Assertions:

• Active-set line naming team.size: small appears before the first agent is spawned
• Phase 1 correctly counts and reports 4 stories with current stage
• Strategy table in Phase 2 classifies all 4 stories with correct types
• Smoke check verdict is taken from the existing production/qa/smoke-*.md report and its source is shown
• QA plan written only after "May I write?" approval
• Phase 4 spawns a single qa-tester at team.size: small, and its prompt names each story's production/qa/test-cases/ path
• Each passing Visual/Feel story has a retained screenshot on disk under production/qa/evidence/[story-slug]/, and its evidence artifact references it and is written only after the "May I write" ask
• Sign-off report includes Test Coverage Summary table and Verdict: APPROVED
• Sign-off report written only after "May I write?" approval
• Verdict: COMPLETE appears in final output
• Next step: "Run /gate-check to validate advancement."

---

Case 2: Smoke Check Fail — QA cycle stops at Phase 2

Fixture:

• Resolved config block: team.size: small, automation: collaborative
• production/sprints/sprint-04.md lists 3 stories
• The most recent production/qa/smoke-[date].md report has verdict FAIL with 2 listed failures (build unstable on launch, core navigation broken)

Input: /team-qa sprint-04

Expected behavior:

1. Phase 1 completes normally
2. Phase 2: Spawns qa-lead via Agent; qa-lead reads the smoke report and records Smoke Check: FAIL with both failures listed prominently
3. AskUserQuestion "QA Strategy Review" is presented, including the option "Smoke check failed — fix issues and re-run /team-qa"
4. Skill does not proceed to Phase 3: it surfaces the failures from the smoke report and stops — the user must fix them, re-run /smoke-check sprint, and then re-run /team-qa
5. No QA plan, test cases, manual QA or sign-off report is produced; a partial report is given
6. Verdict: BLOCKED — smoke check failed

Assertions:

• Smoke check verdict comes from the production/qa/smoke-*.md report — qa-lead does not re-derive it or re-interview the user
• Smoke check FAIL halts the pipeline at Phase 2 — Phases 3, 4, 5, 6 are NOT executed
• Failure list is shown to the user explicitly (not summarized vaguely)
• Skill recommends /smoke-check sprint and a /team-qa re-run as remediation steps
• No QA plan and no QA sign-off report is written or offered
• Orchestrator verdict is BLOCKED, not COMPLETE

---

Case 3: Bug Found — Visual/Feel story fails manual QA, bug report filed

Fixture:

• Resolved config block: team.size: small, automation: collaborative
• production/sprints/sprint-05.md lists 2 stories: 1 Logic (its automated test passes), 1 Visual/Feel
• The most recent production/qa/smoke-[date].md report has verdict PASS
• The Visual/Feel story's animation timing is visibly wrong (acceptance criterion not met)
• production/qa/bugs/ already contains BUG-0001.md and BUG-0002.md

Input: /team-qa sprint-05

Expected behavior:

1. Phases 1–4 complete normally; test cases are written for the Visual/Feel story
2. Phase 5: User marks Visual/Feel story as FAIL; AskUserQuestion collects failure description: "Animation plays at 2x speed — jitter visible on every loop"
3. Phase 5: Spawns qa-tester via Agent to write a formal bug report at production/qa/bugs/BUG-0003.md; report includes severity field
4. Result summary: "Stories PASS: 1, PASS WITH NOTES: 0, FAIL: 1 — bugs filed: BUG-0003, BLOCKED: 0"
5. Phase 6: Spawns qa-lead to produce sign-off report; every story has executed evidence; Bugs Found table lists BUG-0003 with story, severity and status Open; Verdict: NOT APPROVED (a story FAILed without documented workaround)
6. Sign-off report write is offered; writes after approval
7. Next step: "Resolve S1/S2 bugs and re-run /team-qa or targeted manual QA before advancing."

Assertions:

• FAIL result in Phase 5 triggers AskUserQuestion to collect the failure description before the bug report is written
• qa-tester is spawned via Agent to write the bug report — orchestrator does not write it directly
• Bug report follows /bug-report's naming: BUG-[NNNN].md in production/qa/bugs/
• The ID is one past the highest existing bug, zero-padded to four digits (BUG-0003)
• Phase 6 sign-off report Bugs Found table includes the bug ID, story name, severity, and status
• Verdict in sign-off report is NOT APPROVED
• Next step explicitly mentions re-running /team-qa
• Verdict: COMPLETE is still issued by the orchestrator (the QA cycle finished — the sign-off verdict is NOT APPROVED, but the skill completed its pipeline)

---

Case 4: No Argument — Skill infers active sprint or asks user

Fixture (variant A — state files present):

• production/session-state/active.md exists and contains a reference to sprint-06
• production/sprint-status.yaml exists and identifies sprint-06 as active
• production/sprints/sprint-06.md exists

Fixture (variant B — no sprint, as at rigor: minimal):

• production/session-state/active.md does NOT exist
• production/sprint-status.yaml does NOT exist
• No sprint files exist; epics exist under production/epics/

Input: /team-qa (no argument)

Expected behavior (variant A):

1. Phase 1: No argument provided; reads production/session-state/active.md and production/sprint-status.yaml
2. Detects sprint-06 as the active sprint
3. Proceeds as if /team-qa sprint-06 was the input; the Phase 1 report line names it: "QA cycle starting for sprint-06. Found [N] stories. Open bugs in scope: [N or none on file]. Current stage: [stage]. Ready to begin QA strategy?"

Expected behavior (variant B):

1. Phase 1: No argument provided; attempts to read both state files — both missing; no sprint exists
2. Asks the user which epic to cover, then scopes the run as feature: [epic-slug] — it does not invent a sprint

Assertions:

• Skill does NOT default to a hardcoded sprint name when no argument is provided
• Skill reads both production/session-state/active.md AND production/sprint-status.yaml before asking the user (variant A)
• When no sprint can be inferred, skill asks which epic to cover and uses the feature: [epic-slug] form rather than guessing (variant B)
• The inferred sprint is named in the Phase 1 report line before Phase 2 begins (variant A)
• Skill does NOT error out when state files are missing — it falls back to asking (variant B)

---

Case 5: Mixed Results — Some PASS, one FAIL with S1 bug, one BLOCKED

Fixture:

• Resolved config block: team.size: small, automation: collaborative
• production/sprints/sprint-07.md lists 4 stories
• The most recent production/qa/smoke-[date].md report has verdict PASS
• Story A (Logic): automated test passes — PASS
• Story B (UI): manual QA — PASS WITH NOTES (minor text overflow)
• Story C (Visual/Feel): manual QA — FAIL; tester identifies S1 crash on ability activation
• Story D (Integration): cannot test — BLOCKED (dependency system not yet implemented)
• production/qa/bugs/ is empty

Input: /team-qa sprint-07

Expected behavior:

1. Phases 1–4 proceed; Phase 4 test cases cover stories B, C, D
2. Phase 5: Story A is PASS from its automated test; user marks Story B: PASS WITH NOTES; Story C: FAIL; Story D: BLOCKED
3. Story B (UI) gets a retained screenshot of each screen it touched, saved by the tester under production/qa/evidence/[story-slug]/ and confirmed on disk; its evidence artifact at production/qa/evidence/[story-slug]-evidence.md references them and is written only after the "May I write" ask
4. After Story C FAIL: failure description collected; qa-tester spawned to write bug report BUG-0001.md with S1 severity
5. Result summary presented: "Stories PASS: 1, PASS WITH NOTES: 1, FAIL: 1 — bugs filed: BUG-0001, BLOCKED: 1"
6. Phase 6: qa-lead produces sign-off report covering all 4 stories; BUG-0001 listed as S1/Open; Story D listed as BLOCKED with no executed evidence; Verdict: NOT APPROVED — the open S1 is a known failure and is reported as such, not demoted to NOT ASSESSED by the unexecuted Story D
7. Sign-off report written after "May I write?" approval
8. Next step: "Resolve S1/S2 bugs and re-run /team-qa or targeted manual QA before advancing."

Assertions:

• All 4 stories appear in the Phase 6 sign-off report Test Coverage Summary table — none are silently omitted
• Story D (BLOCKED) is listed in the report with a BLOCKED status and named as lacking executed evidence, not silently dropped
• S1 bug causes Verdict: NOT APPROVED regardless of the other stories passing, and regardless of Story D being unexecuted
• PASS WITH NOTES stories do not downgrade to FAIL — they are tracked separately
• Story B's evidence is its retained screenshots — a UI story needs no lead sign-off to close
• BUG-0001 severity is listed as S1 in the Bugs Found table
• Partial results are preserved — the sign-off report is still produced even with failures and blocks
• Verdict: COMPLETE is issued by the orchestrator (pipeline completed); sign-off verdict is NOT APPROVED

---

Case 6: qa.level: minimal — a Logic story with no test is evidenced by its acceptance-criteria walk

Fixture:

• Resolved config block: team.size: small, automation: collaborative, qa.level: minimal (rigor:minimal)
• No sprint files and no production/sprint-status.yaml; production/epics/combat/ holds 2 story files, both In Review: story-001 (Logic, no test file anywhere) and story-002 (UI)
• The most recent production/qa/smoke-[date].md report has verdict PASS (its automated row WAIVED)
• production/qa/bugs/ contains no bugs

Input: /team-qa feature: combat

Expected behavior:

1. Phase 2: qa-lead's strategy table marks story-001's Automated Required as WAIVED — tests are waived at qa.level: minimal — and puts it in manual QA; it is not flagged as missing test evidence or as a blocker
2. Phase 3: the QA plan lists no automated test requirements; its entry criteria take the stories' status from the story files (feature scope), not from production/sprint-status.yaml
3. Phase 4: test cases are written for both stories — story-001's walk its acceptance criteria
4. Phase 5: the tester answers PASS for both; story-002's retained screenshot is saved and confirmed on disk
5. Phase 6: the Test Coverage Summary shows story-001 as WAIVED (qa.level: minimal) under Auto Test with its criteria walked; every story has executed evidence, so the verdict is APPROVED

Assertions:

• story-001 is never reported as missing its test, and its acceptance-criteria walk is its executed evidence
• The sign-off verdict is APPROVED, not NOT ASSESSED
• The UI story still needs its retained screenshot — qa.level: minimal waives tests, not the look
• Entry criteria at feature: scope read the story files' Status, not production/sprint-status.yaml
• No next step tells the user to write Logic tests

---

Case 7: Smoke report NOT ASSESSED — continue with a warning, sign-off NOT ASSESSED

Fixture:

• Resolved config block: team.size: small, automation: collaborative, qa.level: standard
• production/sprints/sprint-008.md lists 2 stories: 1 Logic (its unit test exists and passes) and 1 Visual/Feel
• The most recent production/qa/smoke-[date].md report has verdict NOT ASSESSED — Batch 3 was offered and skipped
• production/qa/bugs/ contains no bugs

Input: /team-qa sprint-8

Expected behavior:

1. Phase 1 matches sprint-008.md by its number
2. Phase 2: records Smoke Check: NOT ASSESSED — Batch 3 skipped — source: production/qa/smoke-[date].md and warns that the last smoke check could not assess it, recommending a /smoke-check sprint re-run; the cycle continues
3. Phases 3–5 run; both stories PASS and the Visual/Feel story's evidence is retained
4. Phase 6: every story has executed evidence, but the smoke check was NOT ASSESSED, so the sign-off verdict is NOT ASSESSED — never APPROVED

Assertions:

• The NOT ASSESSED smoke verdict is taken from the report and shown with its source — not read as PASS or UNKNOWN
• The cycle does not stop at Phase 2 on NOT ASSESSED (only FAIL stops it)
• The sign-off verdict is NOT ASSESSED even though every story passed
• The next step names re-running /smoke-check before /team-qa

---

Case 8: An open S1 already on file — known before sign-off, and the brief names no verdict

Fixture:

• Resolved config block: team.size: small, automation: collaborative, qa.level: standard
• production/sprints/sprint-009.md lists 2 stories in the Combat system: 1 Logic (its unit test passes) and 1 UI
• production/qa/bugs/BUG-0004.md: System Combat, **Severity**: S1-Critical, **Status**: Open — filed before this cycle; BUG-0002.md (Combat, S2) has **Status**: Closed
• The most recent production/qa/smoke-[date].md report has verdict PASS

Input: /team-qa sprint-9

Expected behavior:

1. Phase 1 greps the bug files and reports "Open bugs in scope: 1 — S1: 1"; the closed BUG-0002 is not counted
2. Phase 2's brief to qa-lead carries BUG-0004, and the strategy lists it under Open bugs in scope as a known failure
3. Phases 3–5 run; both stories PASS and the UI story's screenshots are retained
4. Phase 6's brief passes the strategy, the smoke verdict, both stories' results, BUG-0004 and the verdict rules — it names no verdict, pre-fills no ### Verdict: line and does not say which fact decides it
5. The qa-lead returns NOT APPROVED (an open S1 bug in scope); the Bugs Found table lists BUG-0004; the orchestrator checks that verdict against the rules, then asks to write

Assertions:

• BUG-0004 is known from Phase 1 and appears in the Phase 2 strategy — not first discovered at sign-off, and never missed
• The sign-off verdict is NOT APPROVED although every story passed — never NOT ASSESSED or APPROVED
• The Phase 6 brief names no verdict and no deciding fact; the qa-lead reaches NOT APPROVED from the rules
• A closed bug is not counted as open

---

Protocol Compliance

• A manual-QA result is recorded only as the tester gave it — never filled in for them; an unanswered story stays unexecuted
• In Phase 6 the qa-lead returns the sign-off text and writes no file; the orchestrator writes it after the ask
• A bug filed in Phase 5 uses /bug-report's Severity and Priority labels exactly
• At the default team.size: individual, the run opens with an active-set line naming qa-tester as the always-active agent and stating that qa-lead is spawned only at phase gates (the decision points the pipeline lists, whatever the automation mode) — before any agent is spawned
• AskUserQuestion used at Phase 2 (strategy review), Phase 4 (test case approval per group), and Phase 5 (per-story manual QA result and failure description)
• Phase 2 smoke check is a hard gate: FAIL halts the pipeline before Phase 3
• When no production/qa/smoke-*.md report exists, the smoke status is UNKNOWN, the warning "No smoke check report found. Recommend running /smoke-check sprint before QA. Proceeding with caution." is shown, and the cycle continues
• A smoke report whose verdict is NOT ASSESSED is surfaced with what did not run, and the cycle continues as for UNKNOWN
• With no smoke report (UNKNOWN), or one that is NOT ASSESSED, the sign-off verdict is NOT ASSESSED even when every story passes — never APPROVED or APPROVED WITH CONDITIONS — unless an open S1/S2 bug, or a FAIL without a documented workaround, makes it NOT APPROVED
• "May I write?" is asked before every orchestrator write: the QA plan (Phase 3), each group's evidence artifacts (Phase 5) and the sign-off report (Phase 6)
• Bug reports are always written by qa-tester via Agent — orchestrator does not write them directly
• Every Phase 4 qa-tester prompt names each story's output path; per-story parallel spawns happen only at team.size: studio
• Evidence artifacts never carry pre-ticked sign-off rows for a sign-off that did not happen in the session
• Error recovery: any BLOCKED agent is surfaced immediately with AskUserQuestion options
• Partial report always produced — no work is discarded because one story failed or blocked
• Open bugs already on file are read in Phase 1 and reach both qa-lead briefs; no brief to qa-lead names the sign-off verdict
• Sign-off verdict rules are strictly applied: any S1/S2 bug open = NOT APPROVED; APPROVED and APPROVED WITH CONDITIONS require executed evidence for every story in scope
• Orchestrator-level Verdict: COMPLETE is distinct from the sign-off report's verdict

---

Coverage Notes

• The "APPROVED WITH CONDITIONS" verdict path (S3/S4 bugs, PASS WITH NOTES with every story executed) is not given a dedicated case; Case 5's Story B would produce it if no S1/S2 bug were open and Story D had been executed.
• The NOT ASSESSED verdict with no known failure is covered for a NOT ASSESSED smoke report (Case 7); one story BLOCKED with the rest PASS, or no smoke report at all, is not given a dedicated case — the smoke-UNKNOWN rule is asserted in Protocol Compliance.
• The studio shape (one qa-tester per story, in parallel) is asserted structurally but not given a case; Cases 1, 3 and 5 run at small.
• The feature: [system-name] argument form is exercised by Case 6 — it follows the same Phase 1 logic as the sprint form, using glob instead of the sprint file. Case 4 variant B exercises the epic fallback.
• Logic stories with passing automated tests do not need manual QA — this is validated implicitly by Case 5 (Story A) where the Logic story receives no manual QA phase.
• The session-state append to production/session-state/active.md after the final phase is not asserted.