Claude-Code-Game-Studios/CCGS Skill Testing Framework/skills/analysis/security-audit.md

# Skill Test Spec: /security-audit

## Skill Summary

`/security-audit` audits the codebase for common game security failures across
six categories: save/serialization, network/multiplayer, input validation, data
exposure, cheat/anti-tamper vectors, and dependencies. It takes a scope
(`full`, `network`, `save`, `input`, `quick`; no argument runs `full`), reads
`engine.name`, `engine.language`, `platform.targets`, `platform.multiplayer` and
`platform.online` from `project.yaml`, and spawns the `security-engineer` agent
with a fixed brief template — only its slots filled: the scope, the engine, the
`platform.*` values, the resolved code root, `assets/data/` and config files,
the categories and the report path — whose own text tells the agent to rate
every finding from its severity table and that game type never lowers a
rating. Grep patterns are chosen per engine from the
Phase 3 table; a category the table marks NOT SOURCEABLE for the engine is
NOT ASSESSED, and an absent `platform.multiplayer` is never read as `false`.
A skipped category is named with its reason, and an unresolved code root makes
the code categories NOT ASSESSED. Findings are classified CRITICAL / HIGH /
MEDIUM / LOW, each with file, line, attack scenario and remediation. The
report's release recommendation, first match wins, is DO NOT SHIP, FIX BEFORE
SHIPPING, NOT ASSESSED (e.g. INSUFFICIENT IMPLEMENTATION) or CLEAR TO SHIP.
The skill presents the executive summary and CRITICAL/HIGH
findings in conversation, then asks "May I write the full security audit report
to `production/security/security-audit-[date]-[scope].md`?" At
`workflow: minimal` the Phase 7 message says the release gate does not read the
report. No director gates are invoked.

---

## Static Assertions (Structural)

Checked against the SKILL.md by `/skill-test spec` — no fixture needed.

- [ ] Has required frontmatter fields: `name`, `description`, `argument-hint`, `user-invocable`, `allowed-tools`
- [ ] Has ≥2 phase headings
- [ ] Contains verdict keywords: CLEAR TO SHIP, FIX BEFORE SHIPPING, DO NOT SHIP, NOT ASSESSED, and the severities CRITICAL / HIGH / MEDIUM / LOW
- [ ] Contains "May I write" language naming `production/security/security-audit-[date]-[scope].md`
- [ ] Has a next-step handoff (Phase 7: `/security-audit quick`, `/gate-check release`, `/launch-checklist`)

---

## Director Gate Checks

None. The skill spawns the `security-engineer` specialist to run the audit;
that is its core mechanism, not a director gate. No director gate is invoked in
any review mode.

---

## Test Cases

### Case 1: Happy Path — Single-player Godot project, no findings

**Fixture:**
- `project.yaml`: `engine.name: godot`, `engine.language: gdscript`, `platform.multiplayer: false`, `platform.online: false`, `modes.rigor: standard` (so `workflow` resolves to `standard`); `modes.automation` unset (collaborative)
- `src/` holds the implemented game: 42 `.gd` files across core, gameplay, UI and save systems
- `src/core/save_system.gd` opens saves with `FileAccess`, verifies a checksum, and bounds-checks every loaded value before use
- No API keys, secrets, passwords or tokens anywhere in `src/` or `assets/`
- `addons/gdUnit4/` is the only third-party dependency

**Input:** `/security-audit`

**Expected behavior:**
1. No argument → `full` scope
2. Skill reads engine, language and platform keys from `project.yaml`
3. Skill spawns `security-engineer` via `Agent` with the fixed brief template: the scope, Godot/GDScript, `platform.multiplayer: false`, `platform.online: false`, code root `src/`, `assets/data/` and config files, the categories to run, and the report path — nothing added outside the slots
4. Category 1 uses the Godot pattern list, including the Godot 4 name `FileAccess`; Category 2 is skipped because both platform keys are explicitly `false`, and the Executive Summary names the skip and its reason
5. Category 6 lists `addons/gdUnit4/` in the Dependency Inventory
6. Executive Summary shows zero findings at every severity over 42 source files; Release recommendation: **CLEAR TO SHIP**
7. Skill presents the summary, then asks "May I write the full security audit report to `production/security/security-audit-[date]-full.md`?"
8. After writing: "✅ No blocking security findings. Report written to `production/security/`. Include this path when running `/gate-check release`."

**Assertions:**
- [ ] `security-engineer` is spawned with the fixed brief template — scope, engine/language, the `platform.*` values and a source manifest rooted at `src/` — and the brief adds no severity and no remark on the game's type
- [ ] Save handling is checked with Godot 4 names (`FileAccess`), not only the Godot 3 `File.open`
- [ ] Hardcoded-credential patterns (`api_key`, `secret`, `password`, `token`, `private_key`) are scanned
- [ ] The Executive Summary says Category 2 was skipped because `platform.multiplayer` and `platform.online` are `false` — the skip is not silent
- [ ] Release recommendation is CLEAR TO SHIP
- [ ] "May I write" precedes the report write

---

### Case 1b: Single-player, One HIGH Finding — FIX BEFORE SHIPPING

**Fixture:**
- Same as Case 1, except `src/core/save_system.gd` loads save values with no
  checksum or bounds check, so an edited save file unlocks late-game progression
- No other findings

**Input:** `/security-audit`

**Expected behavior:**
1. The save finding is classified **HIGH** (save tampering that bypasses progression)
   and is **not** raised to CRITICAL, because `platform.multiplayer` is `false`
2. Release recommendation: **FIX BEFORE SHIPPING** — an open HIGH, no CRITICAL
3. After the write: "⚠️ 1 HIGH finding(s) are open. The Polish → Release gate's
   security audit item requires none — fix them and re-run `/security-audit quick`
   before `/gate-check release`."

**Assertions:**
- [ ] The brief sent to `security-engineer` is the fixed template: it does not call the save finding HIGH (or any severity), and does not say a single-player game lowers the bar — the agent rates it from its own table
- [ ] The finding is HIGH, not CRITICAL, in a single-player game
- [ ] Release recommendation is FIX BEFORE SHIPPING — never CLEAR TO SHIP with a HIGH open, and not DO NOT SHIP without a CRITICAL
- [ ] The ⚠️ HIGH message names the release gate's security audit item as the requirement, and names `/security-audit quick` and `/gate-check release`
- [ ] Variant — at `modes.rigor: minimal` the recommendation is still FIX BEFORE SHIPPING, and the message says the `minimal` release gate does not read this report instead of naming the gate item

---

### Case 2: Multiplayer Critical — Client-authoritative RPC and a hardcoded key

**Fixture:**
- `project.yaml`: `engine.name: godot`, `platform.multiplayer: true`, `platform.online: true`
- `src/net/combat_sync.gd` line 42: `@rpc("any_peer") func apply_damage(target_id, amount)` applies `amount` with no sender or range validation
- `src/online/leaderboard_client.gd` line 8: `const API_KEY = "sk_live_..."`

**Input:** `/security-audit full`

**Expected behavior:**
1. Category 2 runs (multiplayer is `true`) with the Godot network patterns (`MultiplayerPeer`, `rpc`, `@rpc`, …)
2. The unvalidated `apply_damage` RPC is a CRITICAL finding (client dictates outcomes, breaking multiplayer integrity)
3. The hardcoded key is a credential exposure (HIGH), treated as CRITICAL because the game is multiplayer
4. Each finding is written as `SEC-NNN` with Category, File and line, Description, Attack scenario, Remediation and Effort
5. Release recommendation is DO NOT SHIP — an open CRITICAL decides it
6. Skill presents the executive summary and CRITICAL/HIGH findings, then asks "May I write"
7. Phase 7: "⛔ CRITICAL security findings must be resolved before any public release. Do not proceed to `/launch-checklist` until these are addressed." and re-run `/security-audit quick` before `/gate-check release`

**Assertions:**
- [ ] The RPC finding cites `src/net/combat_sync.gd` with its line and an attack scenario
- [ ] The hardcoded key is raised to CRITICAL in the multiplayer context
- [ ] Every finding carries a specific remediation
- [ ] Release recommendation is DO NOT SHIP (open CRITICAL findings) — never FIX BEFORE SHIPPING or CLEAR TO SHIP
- [ ] The ⛔ message is shown and `/launch-checklist` is held back

---

### Case 3: Multiplayer Scope Unset — Absent is not `false`

**Fixture:**
- `project.yaml`: `engine.name: godot`; no `platform` block at all
- `src/networking/lobby.gd` defines `@rpc` functions `join_lobby()` and `send_chat()`
- `modes.automation` unset (collaborative)

**Input:** `/security-audit`

**Expected behavior:**
1. Skill finds `platform.multiplayer` and `platform.online` absent
2. Skill does not assume single-player and does not skip Category 2
3. Skill asks the user whether the game has multiplayer or online features
4. On "yes", Category 2 runs with the Godot network patterns and checks each `@rpc` call site for validation

**Assertions:**
- [ ] Absent `platform.multiplayer` / `platform.online` are not treated as `false`
- [ ] Skill asks the user whether the game has multiplayer or online features
- [ ] Category 2 is not skipped on the absent keys
- [ ] Unvalidated `@rpc` entry points in `lobby.gd` are checked and any gap is reported with a remediation
- [ ] Variant — with `modes.automation: autonomous` the question cannot be asked: Category 2 still runs, is marked `NOT ASSESSED — multiplayer scope unconfirmed`, and the recommendation is never CLEAR TO SHIP

---

### Case 4: No Implemented Surface — NOT ASSESSED — INSUFFICIENT IMPLEMENTATION

**Fixture:**
- `project.yaml`: `engine.name: godot`, `platform.multiplayer: false`, `platform.online: false`
- `src/` exists but contains no source files; `assets/data/` is empty

**Input:** `/security-audit`

**Expected behavior:**
1. The code root resolves to `src/`, but there is nothing in it to audit
2. Every code-scanning category has zero files to scan, so each is NOT ASSESSED rather than clean
3. Release recommendation: **NOT ASSESSED — INSUFFICIENT IMPLEMENTATION**
4. The report names what was missing (no source under `src/`) and which skill produces it
5. CLEAR TO SHIP is not reported

**Assertions:**
- [ ] The empty code root leaves every code-scanning category NOT ASSESSED; none is reported clean or as zero findings
- [ ] Release recommendation is NOT ASSESSED — INSUFFICIENT IMPLEMENTATION
- [ ] Output names what was missing and which skill produces it
- [ ] CLEAR TO SHIP is not reachable from a scan with nothing to look at
- [ ] Variant — with no `engine.name`, `technical-preferences.md` at `[TO BE CONFIGURED]`, and both `src/` and `Source/` present, the code root is unresolved: no root is guessed, the code categories read `NOT ASSESSED — code root unresolved`, and the recommendation is never CLEAR TO SHIP

---

### Case 5: Unity Project in Full Review Mode — NOT SOURCEABLE category, no gates

**Fixture:**
- `project.yaml`: `engine.name: unity`, `engine.language: csharp`, `platform.multiplayer: true`, `platform.online: true`, `modes.review_mode: full`
- `Assets/Scripts/Save/SaveManager.cs` reads and writes save data
- `Assets/Scripts/Net/PlayerNet.cs` has `[ServerRpc]` methods that validate their inputs
- No findings in the categories that can run

**Input:** `/security-audit`

**Expected behavior:**
1. The manifest passed to `security-engineer` is rooted at `Assets/`, not `src/`
2. Category 1 is `NOT SOURCEABLE` for Unity → `NOT ASSESSED`; no Unity save-API names are written from memory
3. Categories 2 and 3 use the Unity names from the Phase 3 table (`ServerRpc`, `ClientRpc`, `NetworkVariable`, …; `InputSystem`, `PlayerInput`, …), not the Godot lists
4. The Executive Summary names Category 1 as NOT ASSESSED; Release recommendation: `NOT ASSESSED — Save / serialization has no sourced pattern set for Unity`
5. The Phase 7 NOT ASSESSED message names Category 1 as NOT SOURCEABLE on Unity and its fix — a save/serialization module in `docs/engine-reference/unity/modules/` — not `engine.name`, the code root or the multiplayer scope, which were all supplied
6. No director gate is invoked regardless of review mode
7. Skill asks "May I write" before writing the report

**Assertions:**
- [ ] The source manifest uses the Unity code root `Assets/`
- [ ] Category 1 is NOT ASSESSED for Unity and no save-API pattern list is invented
- [ ] Category 2 uses the Unity network names, not the Godot list
- [ ] CLEAR TO SHIP is unreachable; the recommendation names the unsourced category
- [ ] The NOT ASSESSED message points at a serialization module under `docs/engine-reference/unity/modules/` as the fix
- [ ] No director gate is invoked in any review mode

---

### Case 5b: Ranking — An open HIGH outranks an unsourced category

**Fixture:**
- `project.yaml`: `engine.name: unity`, `engine.language: csharp`, `platform.multiplayer: false`, `platform.online: false`
- `Assets/Scripts/` holds the implemented game (35 `.cs` files), including `Assets/Scripts/Save/SaveManager.cs`
- `Assets/Scripts/Analytics/Telemetry.cs` line 12: `const string ApiKey = "sk_live_...";`
- No other findings

**Input:** `/security-audit`

**Expected behavior:**
1. Category 1 is `NOT SOURCEABLE` for Unity, so it is NOT ASSESSED; Category 2 is skipped (both platform keys `false`) and the skip is named
2. The hardcoded key is a credential exposure: HIGH, and not raised to CRITICAL because the game is single-player
3. Release recommendation: **FIX BEFORE SHIPPING** — the open HIGH is matched before NOT ASSESSED
4. The Executive Summary still names Category 1 as NOT ASSESSED

**Assertions:**
- [ ] Release recommendation is FIX BEFORE SHIPPING, not NOT ASSESSED — a known failure is not buried behind an unscanned category
- [ ] Category 1 is still reported NOT ASSESSED in the Executive Summary
- [ ] CLEAR TO SHIP is not reported

---

## Protocol Compliance

- [ ] Reads `engine.name`, `engine.language` and the `platform.*` keys from `project.yaml` before auditing
- [ ] Spawns `security-engineer` via `Agent` with the fixed brief template — only its named slots filled (scope, engine/language, `platform.*` values, the resolved code root, `assets/data/` and config files, the categories, the report path) — and no sentence of the orchestrator's own: no pre-rated finding, no "single-player lowers the stakes"; an unresolved code root makes the code categories NOT ASSESSED, never clean
- [ ] Uses the Phase 3 pattern set for the resolved engine; a NOT SOURCEABLE category is NOT ASSESSED and makes CLEAR TO SHIP unreachable
- [ ] Never treats an absent `platform.multiplayer` / `platform.online` as `false`
- [ ] Names every skipped category and why in the Executive Summary
- [ ] Each finding carries severity, category, file and line, attack scenario and remediation
- [ ] Release recommendation, first match: DO NOT SHIP, FIX BEFORE SHIPPING, NOT ASSESSED, CLEAR TO SHIP
- [ ] Presents the executive summary and CRITICAL/HIGH findings before asking "May I write" for `production/security/security-audit-[date]-[scope].md`; writes only after approval
- [ ] No director gates are invoked
- [ ] Ends with the Phase 7 gate-integration guidance

---

## Coverage Notes

- The `network`, `save`, `input` and `quick` scopes are not tested individually;
  they restrict the categories run by the same rules.
- An absent `engine.name` with a resolvable tree (every grep category NOT
  ASSESSED, with no fallback to the Godot lists) is not tested separately.
- Unreal follows the same engine table as Case 5 and is not tested separately.