mirror of
https://github.com/Donchitos/Claude-Code-Game-Studios.git
synced 2026-10-16 01:37:05 +00:00
Release 1.1.0 — one config file, and a process weight you choose
`project.yaml` at the repo root is now the single source of truth for a project's configuration, replacing `production/stage.txt`, `production/review-mode.txt` and `.claude/docs/technical-preferences.md` (all three still read as a fallback; nothing is force-deleted). **`modes.rigor` — one question instead of six.** `minimal`, `standard` or `full` fronts six knobs: `modes.workflow`, `docs.density`, `qa.level`, `modes.story_granularity`, `modes.review_mode` and `team.size`. It defaults to `minimal`, which is a change from 1.0's behaviour: the heavier tier was measured costing several times more to reach working code without producing a better result, and returning nothing when a fresh developer inherited the project. Pin `modes.rigor: standard` to keep 1.0's process level. Also in this release: - `/settings` to view or change any setting, and `project.local.yaml` for per-developer preferences that should not be committed. - `system_overrides` to hold one system to a higher standard than the rest of the project, and `testing.strict` to set per test type whether missing evidence blocks a story or warns. - A story that changes anything the player sees is not closed until the game has been launched, observed, and a screenshot retained in `production/qa/evidence/`. A parse check is not a run. This holds at every rigor level, including `minimal`, where automated tests are waived but the look is not. - The code root is engine-specific and enforced as such: Godot `src/`, Unity `Assets/`, Unreal `Source/<Module>/`. Two of the three will not build a project laid out any other way. - Migration tooling for existing projects: `.claude/scripts/migrate-v1-config.sh` converts a 1.0 layout, non-destructively by default, with a report to review before `--finalize` deletes anything. - Ask-before-write holds regardless of personal automation settings. Upgrading from 1.0 or earlier: see UPGRADING.md. Full list of changes: CHANGELOG.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016xvzG3i7pWQg4LjyDwWMXd
This commit is contained in:
Donchitos
co-authored by
Claude Opus 5
Claude Opus 5
parent
984023ddac
commit
d05699707f
@@ -1,12 +1,16 @@
|
||||
---
|
||||
name: smoke-check
|
||||
description: "Run the critical path smoke test gate before QA hand-off. Executes the automated test suite, verifies core functionality, and produces a PASS/FAIL report. Run after a sprint's stories are implemented and before manual QA begins. A failed smoke check means the build is not ready for QA."
|
||||
description: "Critical-path smoke gate before QA hand-off — runs the automated suite. A failed check means the build is not QA-ready."
|
||||
argument-hint: "[sprint | quick | --platform pc|console|mobile|all]"
|
||||
user-invocable: true
|
||||
allowed-tools: Read, Glob, Grep, Bash, Write, AskUserQuestion
|
||||
model: sonnet
|
||||
---
|
||||
|
||||
!`source "${CLAUDE_PROJECT_DIR:-.}/.claude/hooks/yaml-helper.sh" 2>/dev/null && resolve_config --keys automation,qa.level,testing.strict`
|
||||
|
||||
|
||||
|
||||
# Smoke Check
|
||||
|
||||
This skill is the gate between "implementation done" and "ready for QA
|
||||
@@ -21,6 +25,16 @@ Handing a broken build to QA wastes their time and demoralises the team.
|
||||
|
||||
---
|
||||
|
||||
Every `AskUserQuestion` call follows `.claude/docs/automation-modes.md`
|
||||
(collaborative asks always · guided major-only · autonomous logs and proceeds;
|
||||
`automation_always_ask` categories always prompt).
|
||||
|
||||
**`qa.level`**: at `minimal`, smoke-check is **optional** — if
|
||||
run, a FAIL is advisory and never blocks hand-off; at `standard`, it is required
|
||||
before a phase transition; at `full`, before every commit. This sits in front of
|
||||
the Phase 6 `testing.strict.config` resolution (which only matters once a smoke run
|
||||
gates). Distinct axis from `workflow`.
|
||||
|
||||
## Parse Arguments
|
||||
|
||||
Arguments can be combined: `/smoke-check sprint --platform console`
|
||||
@@ -46,17 +60,50 @@ Phase 5 outputs a per-platform verdict table in addition to the overall verdict.
|
||||
|
||||
Before running anything, understand the environment:
|
||||
|
||||
1. **Test framework check**: verify `tests/` directory exists.
|
||||
If it does not: "No test directory found at `tests/`. Run `/test-setup`
|
||||
to scaffold the testing infrastructure, or create the directory manually
|
||||
if tests live elsewhere." Then stop.
|
||||
0. **Config coherence**: run `bash .claude/scripts/project-coherence.sh`.
|
||||
|
||||
It compares what `project.yaml` declares against the real project file, the
|
||||
installed engine binary, and the files `commands.*` name. This runs first
|
||||
because two of its checks are about *this skill's own inputs*: a
|
||||
`commands.test` naming a runner that does not exist, or a `commands.build`
|
||||
naming an export preset with no `export_presets.cfg`, will fail here and read
|
||||
as a broken build rather than as broken config.
|
||||
|
||||
Report any `[DIFFERS]` lines in the report's Environment section. They do not
|
||||
by themselves decide the verdict -- but a smoke check run against a project
|
||||
whose declared engine is not the installed one is worth saying out loud.
|
||||
|
||||
1. **Test framework check**: verify that **game** test files exist — not merely
|
||||
that `tests/` does. Check `tests/unit/`, `tests/integration/` and
|
||||
`tests/smoke/` for actual test files.
|
||||
If none are found, **deliver a NOT ASSESSED verdict — do not merely stop.**
|
||||
"Smoke check: **NOT ASSESSED — no game tests found** under
|
||||
`tests/unit|integration|smoke`. Run `/test-setup` to scaffold the testing
|
||||
infrastructure, or point me at where tests live." Then stop.
|
||||
|
||||
> A bare halt is the wrong shape here.
|
||||
> This is the state with the **least** information about build health, so it is
|
||||
> the last one that should exit without a verdict: the caller gets no
|
||||
> machine-readable outcome, and "the skill said nothing" is easy to read as
|
||||
> "nothing was wrong". Replacing a *wrong* verdict with *no* verdict is not an
|
||||
> improvement either — the honest result is the one that names what could not
|
||||
> be established.
|
||||
|
||||
> **Do not gate on `tests/` existing.** `/test-setup` creates `tests/unit/`
|
||||
> and `tests/integration/` with placeholder files, so the directory tree is
|
||||
> present on any project that ran setup — whether or not a single game test
|
||||
> was ever written. Count actual test files instead. That is not
|
||||
> hypothetical: a fixture with no build and zero game tests passed this step
|
||||
> and went on to score **PASS WITH WARNINGS**.
|
||||
|
||||
2. **CI check**: check whether `.github/workflows/` contains a workflow file
|
||||
referencing tests. Note in the report whether CI is configured.
|
||||
|
||||
3. **Engine detection**: read `.claude/docs/technical-preferences.md` and
|
||||
extract the `Engine:` value. Store this for test command selection in
|
||||
Phase 2.
|
||||
3. **Engine detection**: read `engine.name` from `project.yaml`; if that key
|
||||
is absent or empty (including when `project.yaml` has no `engine:` block),
|
||||
fall back to the `Engine:` value in `.claude/docs/technical-preferences.md`
|
||||
(a `[TO BE CONFIGURED]` value means not configured). Store this for test
|
||||
command selection in Phase 2.
|
||||
|
||||
4. **Smoke test list**: check whether `production/qa/smoke-tests.md` or
|
||||
`tests/smoke/` exists. If a smoke test list is found, load it for use in
|
||||
@@ -90,16 +137,26 @@ If neither path exists, note: "GDUnit4 runner not found — confirm the runner
|
||||
path for your test framework."
|
||||
|
||||
**Unity:**
|
||||
Unity tests require the editor and cannot be run headlessly via shell in most
|
||||
environments. Check for recent test result artifacts:
|
||||
Unity **can** run tests headlessly via shell. Do not skip to reading artifacts.
|
||||
|
||||
First confirm the Unity Test Framework is installed. A project without it does
|
||||
not fail cleanly — it **hangs** until something kills it, which is the
|
||||
observation the old "Unity cannot test headlessly" advice was generalised from:
|
||||
```bash
|
||||
# List most recent test results (bash) — on Windows PowerShell use the fallback below
|
||||
ls -t test-results/ 2>/dev/null | head -5 \
|
||||
|| powershell -Command "Get-ChildItem test-results/ -ErrorAction SilentlyContinue | Sort-Object LastWriteTime -Descending | Select-Object -First 5 -ExpandProperty Name"
|
||||
grep -q 'com.unity.test-framework' Packages/manifest.json && echo present || echo ABSENT
|
||||
```
|
||||
If test result files exist (XML or JSON), read the most recent one and parse
|
||||
PASS/FAIL counts. If no artifacts exist: "Unity tests must be run from the
|
||||
editor or CI pipeline. Please confirm test status manually before proceeding."
|
||||
If ABSENT, report `NOT ASSESSED — Unity Test Framework not installed` and
|
||||
give the one-line fix (add `com.unity.test-framework` to `Packages/manifest.json`).
|
||||
**Do not fall through to reading stale artifacts** — an unknown-age XML
|
||||
reported as a pass is worse than no gate.
|
||||
|
||||
If present, run the suite with an explicit timeout:
|
||||
```bash
|
||||
timeout 900 "<Unity.exe>" -batchmode -runTests -projectPath . -testPlatform EditMode -testResults test-results/results.xml
|
||||
```
|
||||
Parse `test-results/results.xml` for the `passed` and `failed` counts on the
|
||||
`<test-run>` element. **A timeout (exit 124) is a gate FAILURE, never a pass** —
|
||||
the run never completed and nothing was verified.
|
||||
|
||||
**Unreal Engine:**
|
||||
```bash
|
||||
@@ -111,19 +168,23 @@ If no matching log found: "UE automation tests must be run via the Session
|
||||
Frontend or CI pipeline. Please confirm test status manually."
|
||||
|
||||
**Unknown engine / not configured:**
|
||||
"Engine not configured in `.claude/docs/technical-preferences.md`. Run
|
||||
`/setup-engine` to specify the engine, then re-run `/smoke-check`."
|
||||
"Engine not configured in `project.yaml` or
|
||||
`.claude/docs/technical-preferences.md`. Run `/setup-engine` to specify the
|
||||
engine, then re-run `/smoke-check`."
|
||||
|
||||
**If the test runner is not available in this environment** (engine binary not
|
||||
on PATH, runner script not found, etc.), report clearly:
|
||||
|
||||
"Automated tests could not be executed — engine binary not found on PATH.
|
||||
Status will be recorded as NOT RUN. Confirm test results from your local IDE
|
||||
or CI pipeline. Unconfirmed NOT RUN is treated as PASS WITH WARNINGS, not
|
||||
FAIL — the developer must manually confirm results."
|
||||
or CI pipeline. Until you do, the verdict is NOT ASSESSED — not FAIL, and not
|
||||
a pass either: nothing has been observed about this build yet."
|
||||
|
||||
Do not treat NOT RUN as an automatic FAIL. Record it as a warning. The
|
||||
developer's manual confirmation in Phase 4 can resolve it.
|
||||
Do not treat NOT RUN as an automatic FAIL. Record it, and let the developer's
|
||||
manual confirmation in Phase 4 resolve it. Until that confirmation arrives the
|
||||
verdict is **NOT ASSESSED** (see the verdict rules in Phase 5), which ranks
|
||||
above both pass values and below FAIL. An unrun suite is not a healthy build; it
|
||||
is an unknown one, and the two need different follow-ups.
|
||||
|
||||
Parse runner output and extract:
|
||||
- Total tests run
|
||||
@@ -178,6 +239,15 @@ Draw the smoke test checklist from, in priority order:
|
||||
3. `tests/smoke/` directory contents (if it exists)
|
||||
4. The standard fallback list below (used only when none of the above exist)
|
||||
|
||||
**Name the source you used in the report**, on its own line — *"Checklist source:
|
||||
`production/qa/smoke-tests.md`"*, or *"Checklist source: standard fallback list —
|
||||
no QA plan scope, no `production/qa/smoke-tests.md`, no `tests/smoke/`."* The
|
||||
fallback is a real fallback, so nothing is silently skipped here; what was
|
||||
missing is that a report drawn from the generic list and one drawn from this
|
||||
project's own smoke definitions were indistinguishable. `/gate-check` now
|
||||
validates a smoke report's claims against the repo, and it cannot weigh them
|
||||
without knowing what the checklist was drawn from.
|
||||
|
||||
Tailor batches 2 and 3 to the actual systems identified from the sprint or QA
|
||||
plan. Replace bracketed placeholders with real mechanic names from the current
|
||||
sprint's stories.
|
||||
@@ -301,7 +371,7 @@ will determine whether the automated test row contributes to a FAIL verdict."
|
||||
| Story | Type | Test File | Coverage Status |
|
||||
|-------|------|-----------|----------------|
|
||||
| [title] | Logic | `tests/unit/[system]/[slug]_test.[ext]` | COVERED |
|
||||
| [title] | Visual/Feel | `tests/evidence/[slug]-screenshots.md` | MANUAL |
|
||||
| [title] | Visual/Feel | `production/qa/evidence/[slug]-screenshots.md` | MANUAL |
|
||||
| [title] | Logic | — | MISSING ⚠ |
|
||||
| [title] | Config/Data | — | EXPECTED |
|
||||
|
||||
@@ -346,7 +416,7 @@ Any platform with one or more FAIL checks contributes to the overall FAIL verdic
|
||||
|
||||
---
|
||||
|
||||
### Verdict: [PASS | PASS WITH WARNINGS | FAIL]
|
||||
### Verdict: [PASS | PASS WITH WARNINGS | NOT ASSESSED | FAIL]
|
||||
|
||||
[Verdict rules — first matching rule wins:]
|
||||
|
||||
@@ -355,8 +425,21 @@ Any platform with one or more FAIL checks contributes to the overall FAIL verdic
|
||||
- Any Batch 1 (core stability) check returned FAIL
|
||||
- Any Batch 2 (primary sprint mechanic or regression check) returned FAIL
|
||||
|
||||
**NOT ASSESSED** if ANY of:
|
||||
- The automated suite is **unconfirmed NOT RUN** — nobody has reported a result
|
||||
- A Batch 1 or Batch 2 check could not be executed (no build, engine not
|
||||
configured, platform unavailable) as opposed to executing and failing
|
||||
- **Any story's coverage row is `UNKNOWN`** (Phase 3: story file missing or
|
||||
unreadable). A story nobody could read is not a story with no gaps — without
|
||||
this line, a run where *every* row is UNKNOWN and the suite passes matches
|
||||
**PASS**, because PASS only requires "no MISSING entries"
|
||||
- **Batch 3 was offered and skipped** ("Performance not checked this session").
|
||||
It is not a FAIL, not an execution failure, and not "PASS or N/A", so without
|
||||
this line it matches no rule at all and renders as `[-]` beside a PASS
|
||||
|
||||
**PASS WITH WARNINGS** if ALL of:
|
||||
- Automated tests PASS or NOT RUN (developer has not yet confirmed)
|
||||
- Automated tests PASS, or NOT RUN **and the developer has confirmed the result
|
||||
from their local IDE or CI**
|
||||
- All Batch 1 and Batch 2 smoke checks PASS
|
||||
- One or more Logic/Integration stories have MISSING test evidence
|
||||
|
||||
@@ -366,6 +449,19 @@ Any platform with one or more FAIL checks contributes to the overall FAIL verdic
|
||||
- No MISSING test evidence entries
|
||||
````
|
||||
|
||||
**`NOT ASSESSED` — the build nobody could check.** Rank: it **outranks PASS and
|
||||
PASS WITH WARNINGS** and **ranks below FAIL**. A suite that never ran has not
|
||||
shown the build is healthy; a suite that ran and failed is the more actionable
|
||||
finding and must not be demoted behind one that did not run.
|
||||
|
||||
This is a change in where unconfirmed `NOT RUN` lands, and it is deliberate. The
|
||||
rule below — **never treat NOT RUN as an automatic FAIL** — is unchanged and
|
||||
still correct: NOT ASSESSED is not a FAIL, and it ranks below one. What changes
|
||||
is that an unrun suite no longer resolves to a *pass* verdict while waiting for a
|
||||
confirmation that may never come. Confirmed NOT RUN (the developer reports the
|
||||
result from their own IDE or CI) still lands at PASS WITH WARNINGS, because
|
||||
somebody did look.
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Write and Gate
|
||||
@@ -376,9 +472,36 @@ Present the full report in conversation, then ask:
|
||||
|
||||
Write only after approval.
|
||||
|
||||
**First apply `qa.level` (resolved earlier).** At `qa.level: minimal`, a FAIL is
|
||||
**advisory** regardless of `testing.strict.config` — skip the resolution below and
|
||||
deliver the advisory-FAIL outcome (smoke-check is optional at minimal and never
|
||||
blocks hand-off). Otherwise:
|
||||
|
||||
**Resolve the gate enforcement level.** A FAIL verdict either *blocks* QA
|
||||
hand-off or is *flagged while hand-off proceeds*, governed by the
|
||||
`testing.strict` block **resolved in the resolved-config block at the top of this skill** (which merges
|
||||
`project.local.yaml` over `project.yaml` — read that block, not the file, or a
|
||||
developer's local override is silently ignored):
|
||||
|
||||
1. Take `testing.strict.config` from that resolved block. If its value is `true`
|
||||
(case-insensitive) → blocking; if `false` → advisory; `unset` → fall through.
|
||||
2. Else read `testing.strict` as a plain boolean (legacy single-value form) — if
|
||||
its value is `true` or `false`, it applies.
|
||||
3. Else default to **blocking** — an unset `testing.strict.config` keeps
|
||||
smoke-check's FAIL gate blocking (behavior unchanged from before this setting
|
||||
existed). Smoke check is a build-health gate, so its unset default is strict
|
||||
even though the `config` test type defaults to advisory elsewhere. The
|
||||
reciprocal carve-out is recorded in `.claude/skills/story-done/SKILL.md` and
|
||||
`.claude/docs/coding-standards.md`, which own the per-story evidence table.
|
||||
|
||||
Only `true` and `false` (case-insensitive) are recognized at steps 1–2. A key
|
||||
that is present but holds any other value — `maybe`, `1`, `yes`, etc. — is
|
||||
treated as unset: continue to the next step, and surface the unrecognized value
|
||||
to the user.
|
||||
|
||||
After writing, deliver the gate verdict:
|
||||
|
||||
**If verdict is FAIL:**
|
||||
**If verdict is FAIL and the gate is blocking:**
|
||||
|
||||
"The smoke check failed. Do not hand off to QA until these failures are
|
||||
resolved:
|
||||
@@ -387,6 +510,37 @@ resolved:
|
||||
|
||||
Fix the failures and run `/smoke-check` again to re-gate before QA hand-off."
|
||||
|
||||
**If verdict is FAIL and the gate is advisory** (`testing.strict.config: false`):
|
||||
|
||||
"The smoke check failed, but `testing.strict.config` is set to advisory — QA
|
||||
hand-off is not blocked. Resolve these before release:
|
||||
|
||||
[List each failing automated test or smoke check with a one-line description]
|
||||
|
||||
QA hand-off: share `production/qa/qa-plan-[sprint].md` with the qa-tester
|
||||
agent to begin manual verification. Re-run `/smoke-check` once the failures
|
||||
are fixed."
|
||||
|
||||
**If verdict is NOT ASSESSED:**
|
||||
|
||||
"The smoke check could not establish build health — it did not fail, it did not
|
||||
run. Do not hand off to QA on this result:
|
||||
|
||||
[Name each check that could not execute, and why: suite unconfirmed NOT RUN,
|
||||
engine binary absent, no build, platform unavailable]
|
||||
|
||||
[For each, the one thing that would make it runnable — e.g. 'confirm the suite
|
||||
result from your IDE or CI', 'run `/setup-engine`', 'produce a build'.]
|
||||
|
||||
Re-run `/smoke-check` once any of those is resolved."
|
||||
|
||||
This outcome is **not governed by `testing.strict.config`**. That setting decides
|
||||
whether a *failure* blocks hand-off; it has nothing to say about a check that
|
||||
never produced a result, and reading an unrun check as advisory-therefore-fine is
|
||||
the exact substitution this verdict exists to prevent. Say what could not be
|
||||
checked and let the user decide — do not resolve it to either pass or FAIL on
|
||||
their behalf.
|
||||
|
||||
**If verdict is PASS WITH WARNINGS:**
|
||||
|
||||
"Smoke check passed with warnings. The build is ready for manual QA.
|
||||
@@ -408,9 +562,14 @@ agent to begin manual verification."
|
||||
|
||||
## Collaborative Protocol
|
||||
|
||||
**Applies in `collaborative` mode (the default).** For `guided` and
|
||||
`autonomous` modes, see `.claude/docs/automation-modes.md` — the rules below
|
||||
describe what collaborative mode requires, not universal behavior.
|
||||
|
||||
- **Never treat NOT RUN as automatic FAIL** — record it as NOT RUN and let
|
||||
the developer confirm status manually. Unconfirmed NOT RUN contributes to
|
||||
PASS WITH WARNINGS, not FAIL.
|
||||
the developer confirm status manually. Unconfirmed NOT RUN yields **NOT
|
||||
ASSESSED**, not FAIL — and not a pass verdict either, which is what it used to
|
||||
yield.
|
||||
- **Never auto-fix failures** — report them and state what must be resolved.
|
||||
Do not attempt to edit source code or test files.
|
||||
- **PASS WITH WARNINGS does not block QA hand-off** — it records advisory
|
||||
|
||||
Reference in New Issue
Block a user