Claude-Code-Game-Studios/CCGS Skill Testing Framework/skills/utility/soak-test.md

# Skill Test Spec: /soak-test

## Skill Summary

`/soak-test [duration] [focus]` generates an observation protocol for an
extended play session — the human plays, the skill writes the plan. Duration is
`30m`, `1h`, `2h` or `4h` (default `1h`); focus is `memory`, `stability`,
`balance` or `all` (default `all`). It reads `engine.name` and `performance.*`
from `project.yaml` (falling back to `technical-preferences.md`), the game
concept or brief, the latest playtest and QA plan, then builds timed
checkpoints for the duration and engine-specific memory guidance whose alert
thresholds are relative to the session's own T+0 baseline, in units recorded as
displayed. The engine tool and counter names it gives are not covered by
`docs/engine-reference/`, so they are marked NOT SOURCEABLE — pointers the
tester confirms at T+0, recording the tool actually used.

The protocol document carries a Verdict section with PASS / PASS WITH CONCERNS /
NOT ASSESSED / FAIL for the tester to fill in after the session; a short soak
cannot return PASS. The skill asks "May I write this soak test protocol to
`production/qa/soak-test-[date]-[duration].md`?" before writing, and never runs
the soak itself. No director gates apply.

---

## Static Assertions (Structural)

Checked against the SKILL.md by `/skill-test spec` — no fixture needed.

- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
- [ ] Has ≥2 phase headings
- [ ] Contains verdict keywords: PASS, PASS WITH CONCERNS, NOT ASSESSED, FAIL
- [ ] Contains "May I write" collaborative protocol language before writing the protocol
- [ ] Has a next-step handoff (`/bug-triage sprint` after the session; `/smoke-check` after fixing a FAIL)

---

## Director Gate Checks

None. `/soak-test` is a QA planning utility. No director gates apply.

---

## Test Cases

### Case 1: Happy Path — 2-hour, all-focus soak on Godot

**Fixture:**
- `project.yaml` has `engine.name: Godot` and `performance.target_framerate: 60`
- `design/gdd/game-concept.md` states the intended session length
- The user approves the write

**Input:** `/soak-test 2h all`

**Expected behavior:**
1. Skill loads context: engine, performance budgets, concept, latest playtest and QA plan
2. Checkpoints for `2h`: T+0, T+20, T+40, T+60, T+80, T+100, T+120
3. Godot memory items: the Debugger's memory monitors, Static Memory (unit as
   displayed), Object Count, Orphan Nodes — names marked NOT SOURCEABLE, since
   `docs/engine-reference/godot/` does not cover them, for the tester to confirm
   at T+0; alert when memory grows > 20% from T+0 after the first 15 minutes;
   Orphan Nodes must return to its T+0 value after a scene unload
4. Stability items (60 fps target) and balance/fatigue items are both included
5. Protocol document has Pre-Session Setup, Baseline (T+0), a Checkpoint Log per
   checkpoint, Post-Session Analysis, and a Verdict section left for the tester
6. Skill asks "May I write this soak test protocol to
   `production/qa/soak-test-[date]-2h.md`?"
7. After writing, prints the run steps ending with `/bug-triage sprint`

**Assertions:**
- [ ] Checkpoints are exactly T+0, T+20, T+40, T+60, T+80, T+100, T+120
- [ ] Memory thresholds are relative to T+0, and the unit is recorded as displayed (no unit assumed)
- [ ] The Godot tool and counter names are marked NOT SOURCEABLE, not presented as verified, and Pre-Session Setup asks for the tool actually used
- [ ] Memory, stability and balance observation items are all present (focus `all`)
- [ ] "May I write" is asked with `production/qa/soak-test-[date]-2h.md`
- [ ] The skill issues no verdict itself — the protocol's Verdict section is left for the tester

---

### Case 2: No Arguments — Defaults applied

**Fixture:**
- No arguments provided; engine configured

**Input:** `/soak-test`

**Expected behavior:**
1. Duration defaults to `1h`, focus to `all` — no questions are needed to start
2. Checkpoints: T+0, T+15, T+30, T+45, T+60
3. Protocol header records `Duration: 1h` and `Focus: all`
4. Skill asks "May I write this soak test protocol to `production/qa/soak-test-[date]-1h.md`?"

**Assertions:**
- [ ] Duration `1h` and focus `all` are used when no argument is given
- [ ] Checkpoints are exactly T+0, T+15, T+30, T+45, T+60
- [ ] The output path ends `-1h.md`
- [ ] Nothing is written before the "May I write" approval

---

### Case 3: Narrow Focus on Unreal — 4-hour memory soak

**Fixture:**
- `project.yaml` has `engine.name: Unreal`

**Input:** `/soak-test 4h memory`

**Expected behavior:**
1. Checkpoints for `4h`: T+0, T+30, T+60, T+90, T+120, T+180, T+240
2. Unreal memory items: a memory readout at each checkpoint — `stat memory`,
   marked NOT SOURCEABLE since `docs/engine-reference/unreal/` does not cover
   it; record Physical Memory Used and Available (units as displayed)
3. Alert threshold: Physical Memory Used growth > 20% over the full soak,
   against this run's T+0 — no absolute MB threshold
4. Balance/fatigue observation items are not generated (focus is `memory`)
5. Output path `production/qa/soak-test-[date]-4h.md`

**Assertions:**
- [ ] Checkpoints are exactly T+0, T+30, T+60, T+90, T+120, T+180, T+240
- [ ] The Unreal threshold is a relative > 20% growth, not an absolute size
- [ ] `stat memory` is marked NOT SOURCEABLE, a pointer to confirm rather than a verified command
- [ ] Balance/fatigue items are omitted for focus `memory`
- [ ] "May I write" names the `-4h.md` path

---

### Case 4: No Performance Budgets — "not set", never invented

**Fixture:**
- `project.yaml` has `engine.name: Unity` and no `performance` block
- `technical-preferences.md` Performance Budgets are `[TO BE CONFIGURED]`

**Input:** `/soak-test 30m memory`

**Expected behavior:**
1. Skill falls back from `project.yaml` to `technical-preferences.md` for budgets
2. Budget notes read `Memory ceiling: not set`, `Target FPS: not set`,
   `Frame budget: not set` — no number is supplied in their place
3. Checkpoints: T+0, T+10, T+20, T+30
4. Unity memory items: a memory profiler, marked NOT SOURCEABLE (not covered by
   `docs/engine-reference/unity/`); record Total Reserved Memory, GC Allocated,
   Object Count (units as displayed); alert when GC Allocated grows
   monotonically across 3+ checkpoints
5. Skill asks "May I write" before writing `production/qa/soak-test-[date]-30m.md`

**Assertions:**
- [ ] Missing budgets are recorded as "not set", not filled with defaults
- [ ] Checkpoints are exactly T+0, T+10, T+20, T+30
- [ ] The Unity threshold is the unit-free monotonic-growth check
- [ ] "May I write" is asked before the file is written

---

### Case 5: Director Gate Check — No gate; soak-test is a planning utility

**Fixture:**
- Engine configured; valid duration and focus provided

**Input:** `/soak-test 1h stability`

**Expected behavior:**
1. Skill generates and writes the soak test protocol
2. No director agents are spawned
3. No gate IDs appear in output

**Assertions:**
- [ ] No director gate is invoked
- [ ] No gate skip messages appear
- [ ] The run ends at "Protocol written." with the run steps — no gate verdict, and no soak verdict issued by the skill

---

## Protocol Compliance

- [ ] Parses `[duration] [focus]`, defaulting to `1h` and `all`
- [ ] Uses the Phase 3 checkpoint table for the chosen duration
- [ ] Memory alert thresholds are ratios or deltas against T+0; units are recorded as displayed
- [ ] Engine tool names not covered by `docs/engine-reference/` are marked NOT SOURCEABLE; a memory tool the tester cannot find leaves the verdict NOT ASSESSED
- [ ] The protocol's Verdict section offers PASS / PASS WITH CONCERNS / NOT ASSESSED / FAIL and states a short soak cannot return PASS
- [ ] Asks "May I write" before creating `production/qa/soak-test-[date]-[duration].md`
- [ ] Never attempts to run the soak itself

---

## Coverage Notes

- The `stability` and `balance` focus values follow Case 3's pattern with the
  other observation groups; not separately tested.
- At `rigor: minimal` the skill reads `design/game-brief.md` instead of the
  game concept; not separately tested.
- Executing the protocol is outside this skill's scope — it generates the plan,
  a human runs it.