Add comprehensive QA and testing framework (52→56 skills)

Introduces a full shift-left QA pipeline with Story Type classification
as the backbone of the Definition of Done:

New skills:
- /test-setup: scaffold test framework + CI/CD per engine (Godot/Unity/Unreal)
- /qa-plan: generate sprint test plan classifying stories by type
- /smoke-check: critical path gate (PASS/PASS WITH WARNINGS/FAIL) before QA hand-off
- /team-qa: orchestrate qa-lead + qa-tester through full QA cycle

Story Type classification (Logic/Integration/Visual/Feel/UI/Config/Data):
- Logic and Integration: BLOCKING DoD gate — unit/integration test required
- Visual/Feel and UI: ADVISORY — screenshot + sign-off evidence required
- Config/Data: ADVISORY — smoke check pass sufficient

Updated skills: story-done (test evidence gate), story-readiness (Story Type
check), gate-check (test framework at Technical Setup, test evidence at
Polish/Release), create-epics-stories (Type field + Test Evidence section)

Updated agents: qa-lead (shift-left philosophy + evidence table),
qa-tester (automated test patterns for Godot/Unity/Unreal)

New templates: test-evidence.md (manual sign-off record), test-plan.md
(sprint-oriented QA plan replacing generic feature template)

Updated coding-standards.md: Testing Standards section with DoD table,
test rules, what NOT to automate, and engine-specific CI/CD commands

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Donchitos
2026-03-16 13:48:32 +11:00
parent a2f8ed93ff
commit 168ac96c3a
13 changed files with 1704 additions and 87 deletions

View File

@@ -23,3 +23,43 @@
7. **Tuning Knobs** -- configurable values identified
8. **Acceptance Criteria** -- testable success conditions
- Balance values must link to their source formula or rationale
# Testing Standards
## Test Evidence by Story Type
All stories must have appropriate test evidence before they can be marked Done:
| Story Type | Required Evidence | Location | Gate Level |
|---|---|---|---|
| **Logic** (formulas, AI, state machines) | Automated unit test — must pass | `tests/unit/[system]/` | BLOCKING |
| **Integration** (multi-system) | Integration test OR documented playtest | `tests/integration/[system]/` | BLOCKING |
| **Visual/Feel** (animation, VFX, feel) | Screenshot + lead sign-off | `production/qa/evidence/` | ADVISORY |
| **UI** (menus, HUD, screens) | Manual walkthrough doc OR interaction test | `production/qa/evidence/` | ADVISORY |
| **Config/Data** (balance tuning) | Smoke check pass | `production/qa/smoke-[date].md` | ADVISORY |
## Automated Test Rules
- **Naming**: `[system]_[feature]_test.[ext]` for files; `test_[scenario]_[expected]` for functions
- **Determinism**: Tests must produce the same result every run — no random seeds, no time-dependent assertions
- **Isolation**: Each test sets up and tears down its own state; tests must not depend on execution order
- **No hardcoded data**: Test fixtures use constant files or factory functions, not inline magic numbers
(exception: boundary value tests where the exact number IS the point)
- **Independence**: Unit tests do not call external APIs, databases, or file I/O — use dependency injection
## What NOT to Automate
- Visual fidelity (shader output, VFX appearance, animation curves)
- "Feel" qualities (input responsiveness, perceived weight, timing)
- Platform-specific rendering (test on target hardware, not headlessly)
- Full gameplay sessions (covered by playtesting, not automation)
## CI/CD Rules
- Automated test suite runs on every push to main and every PR
- No merge if tests fail — tests are a blocking gate in CI
- Never disable or skip failing tests to make CI pass — fix the underlying issue
- Engine-specific CI commands:
- **Godot**: `godot --headless --script tests/gdunit4_runner.gd`
- **Unity**: `game-ci/unity-test-runner@v4` (GitHub Actions)
- **Unreal**: headless runner with `-nullrhi` flag