mirror of
https://github.com/Donchitos/Claude-Code-Game-Studios.git
synced 2026-10-16 10:17:06 +00:00
Release 1.1.2 — the default path works end to end
Fix release for 1.1.1: the minimal path end to end, engine commands that run on Godot, Unity and Unreal, hook warnings you can see, and verdicts that say when they could not check. Details in CHANGELOG.md; upgrade steps in UPGRADING.md (v1.1.1 → v1.1.2). Fixes #67 Fixes #131 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WuJn2FdynEybfL5KC697nJ
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
7ed2c3e9c4
commit
b21fa0f7f2
@@ -7,7 +7,7 @@ allowed-tools: Read, Glob, Grep, Bash, Write, AskUserQuestion, Bash(bash "*/.cla
|
||||
model: sonnet
|
||||
---
|
||||
|
||||
!`bash "${CLAUDE_SKILL_DIR}/../../hooks/yaml-helper.sh" resolve_config --keys automation,qa.level,testing.strict`
|
||||
!`bash "${CLAUDE_SKILL_DIR}/../../hooks/yaml-helper.sh" resolve_config --keys automation,workflow,qa.level,testing.strict`
|
||||
|
||||
|
||||
|
||||
@@ -30,7 +30,8 @@ Every `AskUserQuestion` call follows `.claude/docs/automation-modes.md`
|
||||
`automation_always_ask` categories always prompt).
|
||||
|
||||
**`qa.level`**: at `minimal`, smoke-check is **optional** — if
|
||||
run, a FAIL is advisory and never blocks hand-off; at `standard`, it is required
|
||||
run, a FAIL is advisory and never blocks hand-off, and a project with no game
|
||||
tests is checked by its build, launch and critical paths alone (Phase 1 step 1); at `standard`, it is required
|
||||
before a phase transition; at `full`, before every commit. This sits in front of
|
||||
the Phase 6 `testing.strict.config` resolution (which only matters once a smoke run
|
||||
gates). Distinct axis from `workflow`.
|
||||
@@ -74,11 +75,25 @@ Before running anything, understand the environment:
|
||||
whose declared engine is not the installed one is worth saying out loud.
|
||||
|
||||
1. **Test framework check**: verify that **game** test files exist — not merely
|
||||
that `tests/` does. Check `tests/unit/`, `tests/integration/` and
|
||||
`tests/smoke/` for actual test files.
|
||||
If none are found, **deliver a NOT ASSESSED verdict — do not merely stop.**
|
||||
that `tests/` does. Check the engine's **test root** (from step 3; the table
|
||||
is in `.claude/docs/directory-structure.md` — Godot `tests/unit/` and
|
||||
`tests/integration/`, Unity `Assets/Tests/`, Unreal
|
||||
`Source/*/Private/Tests/`) and `tests/smoke/` for actual test files. A
|
||||
**test file** is one the engine's test runner executes — a `*_test.gd` suite,
|
||||
a C# test class, an Unreal automation test source. Markdown checklists do not
|
||||
count: `/test-setup` always writes `tests/smoke/critical-paths.md`, and
|
||||
counting it would hide exactly the zero-test state this step exists to catch.
|
||||
If none are found **at `qa.level: minimal`**, tests are waived at this level:
|
||||
record the automated row as `WAIVED — qa.level: minimal, no game tests`, skip
|
||||
the test run in Phase 2 and all of Phase 3 (say so in one line each), run
|
||||
Phase 2's **build check**, and go on to Phase 4 — the build check and the
|
||||
launch and critical-path checks are the smoke check at this level, so
|
||||
a project with no tests can still reach PASS, but never without a run: a
|
||||
Batch 1 answer that the build was not launched this session leaves the
|
||||
verdict NOT ASSESSED.
|
||||
If none are found **at `standard` or `full`**, **deliver a NOT ASSESSED verdict — do not merely stop.**
|
||||
"Smoke check: **NOT ASSESSED — no game tests found** under
|
||||
`tests/unit|integration|smoke`. Run `/test-setup` to scaffold the testing
|
||||
[the test root] or `tests/smoke/`. Run `/test-setup` to scaffold the testing
|
||||
infrastructure, or point me at where tests live." Then stop.
|
||||
|
||||
> A bare halt is the wrong shape here.
|
||||
@@ -92,9 +107,8 @@ Before running anything, understand the environment:
|
||||
> **Do not gate on `tests/` existing.** `/test-setup` creates `tests/unit/`
|
||||
> and `tests/integration/` with placeholder files, so the directory tree is
|
||||
> present on any project that ran setup — whether or not a single game test
|
||||
> was ever written. Count actual test files instead. That is not
|
||||
> hypothetical: a fixture with no build and zero game tests passed this step
|
||||
> and went on to score **PASS WITH WARNINGS**.
|
||||
> was ever written. Count actual test files instead; gated on the directory,
|
||||
> a project with no build and zero game tests passes this step.
|
||||
|
||||
2. **CI check**: check whether `.github/workflows/` contains a workflow file
|
||||
referencing tests. Note in the report whether CI is configured.
|
||||
@@ -105,6 +119,17 @@ Before running anything, understand the environment:
|
||||
(a `[TO BE CONFIGURED]` value means not configured). Store this for test
|
||||
command selection in Phase 2.
|
||||
|
||||
Then find the **editor executable** — none of the three engines installs
|
||||
onto `PATH` by default on Windows or macOS, so a bare `godot` that does not
|
||||
resolve says nothing about whether the engine is installed. Take the first
|
||||
that resolves (the file exists, or `command -v` finds it): the executable
|
||||
`commands.test` names (its quoted path or first word), else the editor
|
||||
`engine.path` records, else the engine's name on `PATH` (`godot`). Use it
|
||||
wherever Phase 2 writes `godot` or `"<Unity editor>"`, quoting a path with a
|
||||
space; on Unreal, `<UE root>` is `engine.path`. If none resolves, the test
|
||||
runner is not available (Phase 2) — on Godot,
|
||||
`NOT ASSESSED — Godot executable not found (commands.test, engine.path, PATH)`.
|
||||
|
||||
4. **Smoke test list**: check whether `production/qa/smoke-tests.md` or
|
||||
`tests/smoke/` exists. If a smoke test list is found, load it for use in
|
||||
Phase 4. If neither exists, smoke tests will be drawn from the current QA
|
||||
@@ -113,28 +138,57 @@ Before running anything, understand the environment:
|
||||
5. **QA plan check**: glob `production/qa/qa-plan-*.md` and take the most
|
||||
recently modified file. If found, note the path — it will be used in
|
||||
Phase 3 and Phase 4. If not found, note: "No QA plan found. Run
|
||||
`/qa-plan sprint` before smoke-checking for best results."
|
||||
`/qa-plan sprint` before smoke-checking for best results." (At `workflow: minimal`,
|
||||
which has no sprints, a QA plan is optional — say so rather than recommend one.)
|
||||
|
||||
Report findings before proceeding: "Environment: [engine]. Test directory:
|
||||
Report findings before proceeding: "Environment: [engine]. Editor: [the
|
||||
resolved executable / not found]. Test directory:
|
||||
[found / not found]. CI configured: [yes / no]. QA plan: [path / not found]."
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Run Automated Tests
|
||||
|
||||
Attempt to run the test suite via Bash. Select the command based on the engine
|
||||
detected in Phase 1:
|
||||
**Run `commands.test` from `project.yaml` when it is set** — it is the
|
||||
project's own test command, the one `/setup-engine` wrote and CI runs. Use the
|
||||
per-engine default below only when `commands.test` is absent, and say in the
|
||||
report which of the two you ran. Wrap it in a timeout (`timeout 900 …`): **a
|
||||
timeout (exit 124) is a gate FAILURE, never a pass** — the run never completed
|
||||
and nothing was verified. macOS has no `timeout`: use `gtimeout 900 …` from
|
||||
Homebrew's `coreutils` (same exit 124), or, with nothing installed,
|
||||
`perl -e 'alarm shift; exec @ARGV' 900 …`, which exits 142 when time runs out —
|
||||
a FAILURE the same way. Then read the result for the engine. On every engine
|
||||
one outcome looks like success and is not; each is named below.
|
||||
|
||||
**Godot 4:**
|
||||
**Godot 4** — first check that `addons/gdUnit4/bin/GdUnitCmdTool.gd` exists —
|
||||
the folder has a **capital U**. If it does not, report
|
||||
`NOT ASSESSED — gdUnit4 not installed at addons/gdUnit4/` instead of running
|
||||
anything. Then import, whether you run `commands.test` or the
|
||||
default. A fresh clone has no `.godot/` class cache, and without it gdUnit4's
|
||||
runner does not load; a cache that predates a new `class_name` fails valid
|
||||
code with 105. The import takes seconds when nothing changed (`godot` below is
|
||||
the executable Phase 1 step 3 resolved):
|
||||
```bash
|
||||
godot --headless --script tests/gdunit4_runner.gd 2>&1
|
||||
godot --headless --path . --import
|
||||
godot --headless -s -d --remote-debug tcp://127.0.0.1:0 res://addons/gdUnit4/bin/GdUnitCmdTool.gd -a res://tests --ignoreHeadlessMode 2>&1
|
||||
```
|
||||
If the GDUnit4 runner script does not exist at that path, try:
|
||||
```bash
|
||||
godot --headless -s addons/gdunit4/GdUnitRunner.gd 2>&1
|
||||
```
|
||||
If neither path exists, note: "GDUnit4 runner not found — confirm the runner
|
||||
path for your test framework."
|
||||
gdUnit4's exit code gives the verdict (6.1.3):
|
||||
- **0** — every test passed: PASS. **But exit 0 over zero tests is not a pass:**
|
||||
gdUnit4 aborts and still exits 0, so if the output says `No test cases found`,
|
||||
the verdict is `NOT ASSESSED — no tests found`.
|
||||
- **101** — every test passed but a test leaked nodes (orphans): PASS WITH
|
||||
WARNINGS, naming the orphan count from the output. Not a failure.
|
||||
- **100** — a test failed: FAIL. **105** — a test script does not parse: FAIL.
|
||||
- **103** — gdUnit4 refused to run headless (`--ignoreHeadlessMode` missing);
|
||||
**104** — Godot older than 4.3: `NOT ASSESSED`, with that reason.
|
||||
- **1** — Godot's own error: `GdUnitCmdTool.gd` did not load (`Failed to load
|
||||
script`), so no test ran. Usually the import above was skipped; after it,
|
||||
exit 1 is a FAIL.
|
||||
|
||||
Every run opens with two `ERROR:` lines about the remote port
|
||||
(`127.0.0.1:0`) — that is the `--remote-debug` flag working, not a failure.
|
||||
Never drop the flag: without it a script error opens Godot's interactive
|
||||
debugger and the run hangs at a `debug>` prompt until the timeout.
|
||||
|
||||
**Unity:**
|
||||
Unity **can** run tests headlessly via shell. Do not skip to reading artifacts.
|
||||
@@ -150,33 +204,106 @@ give the one-line fix (add `com.unity.test-framework` to `Packages/manifest.json
|
||||
**Do not fall through to reading stale artifacts** — an unknown-age XML
|
||||
reported as a pass is worse than no gate.
|
||||
|
||||
If present, run the suite with an explicit timeout:
|
||||
If present, the default is (the **editor** executable — `Unity` on `PATH` may be
|
||||
Unity's separate CLI). Delete the old results file first, whether you run
|
||||
`commands.test` or this default:
|
||||
```bash
|
||||
timeout 900 "<Unity.exe>" -batchmode -runTests -projectPath . -testPlatform EditMode -testResults test-results/results.xml
|
||||
rm -f test-results/editmode.xml
|
||||
timeout 900 "<Unity editor>" -batchmode -runTests -projectPath . -testPlatform EditMode -testResults test-results/editmode.xml
|
||||
```
|
||||
Parse `test-results/results.xml` for the `passed` and `failed` counts on the
|
||||
`<test-run>` element. **A timeout (exit 124) is a gate FAILURE, never a pass** —
|
||||
the run never completed and nothing was verified.
|
||||
Exit 0 when every test passes, 2 on a failure. **Exit 1 or 3 is a FAIL**: 1
|
||||
means the project did not compile (the editor log has the `error CS…` lines),
|
||||
3 that the run itself failed; 4 (an unknown `-testPlatform`) or any other
|
||||
non-zero exit is a FAIL too. A compile error writes no results file — which
|
||||
is why the old one is deleted first: left in place, the last run's pass is read
|
||||
as this run's. After any exit, no results file is a FAIL. Parse the `<test-run>`
|
||||
element of the results file for `testcasecount`, `passed` and `failed`. **`testcasecount="0"`
|
||||
is NOT ASSESSED, never a pass**: tests outside `Assets/` are never compiled, and
|
||||
Unity then reports `result="Passed"` with exit 0 over zero tests.
|
||||
|
||||
**Unreal Engine:**
|
||||
**Play Mode is a second run.** `-testPlatform EditMode` — this default, and
|
||||
`commands.test` as `/setup-engine` writes it — never runs the integration tests
|
||||
`/test-setup` scaffolds under `Assets/Tests/PlayMode/`. When that folder holds a
|
||||
test file and the command you ran did not already pass `-testPlatform PlayMode`,
|
||||
run Play Mode the same way, into its own results file, and read it by the same
|
||||
rules:
|
||||
```bash
|
||||
# List most recent Unreal automation logs (bash) — on Windows PowerShell use the fallback below
|
||||
ls -t Saved/Logs/ 2>/dev/null | grep -i "test\|automation" | head -5 \
|
||||
|| powershell -Command "Get-ChildItem Saved/Logs/ -ErrorAction SilentlyContinue | Where-Object { $_.Name -match 'test|automation' } | Sort-Object LastWriteTime -Descending | Select-Object -First 5 -ExpandProperty Name"
|
||||
rm -f test-results/playmode.xml
|
||||
timeout 900 "<Unity editor>" -batchmode -runTests -projectPath . -testPlatform PlayMode -testResults test-results/playmode.xml
|
||||
```
|
||||
If no matching log found: "UE automation tests must be run via the Session
|
||||
Frontend or CI pipeline. Please confirm test status manually."
|
||||
Report each platform on its own line — `EditMode: PASS (12/12)`,
|
||||
`PlayMode: FAIL (1 failure)`. A Play Mode run that could not be made is
|
||||
reported by name, `PlayMode: NOT RUN — [reason]`: the Edit Mode result does not
|
||||
cover those tests, so it cannot make the automated-test row a PASS.
|
||||
|
||||
**Unreal Engine** — build the editor target first, whether you then run
|
||||
`commands.test` or the default. `Binaries/` is not committed, and without a
|
||||
built game module the editor stops with "The game module could not be found",
|
||||
exit 1, before running any test. The build is incremental — seconds when
|
||||
nothing changed; a failed build is a FAIL.
|
||||
|
||||
**Pick the commands for this machine** with `uname -s`: `Linux` → Linux,
|
||||
`Darwin` → macOS, anything else (`MINGW*`, `MSYS*`, `CYGWIN*`) → Windows. The
|
||||
Windows forms below were run on UE 5.7; the Linux and macOS forms come from
|
||||
Epic's documentation — `docs/engine-reference/unreal/current-best-practices.md`,
|
||||
"Command Line", has them with their sources. On Windows use
|
||||
`UnrealBuildTool.exe`, not `Build.bat`: run from bash, `Build.bat` fails on an
|
||||
engine path with spaces.
|
||||
```bash
|
||||
# Windows
|
||||
timeout 1800 "<UE root>/Engine/Binaries/DotNET/UnrealBuildTool/UnrealBuildTool.exe" <Project>Editor Win64 Development -Project="$(pwd -W 2>/dev/null || pwd)/<Project>.uproject" 2>&1
|
||||
# Linux
|
||||
timeout 1800 "<UE root>/Engine/Build/BatchFiles/Linux/Build.sh" <Project>Editor Linux Development -Project="$(pwd -W 2>/dev/null || pwd)/<Project>.uproject" 2>&1
|
||||
# macOS (timeout: see the macOS note above)
|
||||
gtimeout 1800 "<UE root>/Engine/Build/BatchFiles/Mac/Build.sh" <Project>Editor Mac Development -Project="$(pwd -W 2>/dev/null || pwd)/<Project>.uproject" 2>&1
|
||||
```
|
||||
Then the default. Keep `-stdout -FullStdOutLogOutput`; when `commands.test`
|
||||
lacks them, add them to the command for this run only: without them the editor
|
||||
prints nothing, and the lines below are only in `Saved/Logs/<Project>.log`.
|
||||
Give the project as an **absolute** path — when `commands.test` passes a
|
||||
relative one, run it with the absolute path instead: UE 5.7
|
||||
does not find a relative `<Project>.uproject` and exits 1 (`Project file not
|
||||
found`). `$(pwd -W 2>/dev/null || pwd)` gives the `C:/…` form in Git Bash and
|
||||
the plain path elsewhere; `$PWD` alone fails when `MSYS_NO_PATHCONV` is set.
|
||||
Either way, say in the report what you changed and tell the user to fix
|
||||
`commands.test` in `project.yaml` — never edit `project.yaml` without asking.
|
||||
`<Project>.` in the filter is the root the tests are named under: the project
|
||||
name, or the distinct root `/setup-engine` chose when the project's name is
|
||||
also an engine area — read one test's name string under
|
||||
`Source/*/Private/Tests/` if unsure.
|
||||
```bash
|
||||
# Windows
|
||||
timeout 1800 "<UE root>/Engine/Binaries/Win64/UnrealEditor-Cmd.exe" "$(pwd -W 2>/dev/null || pwd)/<Project>.uproject" -ExecCmds="Automation RunTests <Project>.; Quit" -unattended -nullrhi -stdout -FullStdOutLogOutput 2>&1
|
||||
# Linux
|
||||
timeout 1800 "<UE root>/Engine/Binaries/Linux/UnrealEditor" "$(pwd -W 2>/dev/null || pwd)/<Project>.uproject" -ExecCmds="Automation RunTests <Project>.; Quit" -unattended -nullrhi -stdout -FullStdOutLogOutput 2>&1
|
||||
```
|
||||
**macOS has no sourced test command.** Epic documents only the
|
||||
`UnrealEditor.app` bundle, not a command-line editor inside it. Run a
|
||||
`commands.test` the user has set and confirmed; with none, report
|
||||
`NOT ASSESSED — Unreal test command on macOS not sourced; set commands.test to the editor command you use`
|
||||
instead of guessing a path.
|
||||
Read the output, not the exit code: a failing test and a run that matched nothing both exit 255.
|
||||
- `Result={Fail}` on any line → FAIL.
|
||||
- `No automation tests matched` → `NOT ASSESSED — no tests found` (tests
|
||||
outside `Source/<Module>/` are never built), not FAIL.
|
||||
- Exit 0 with `Result={Success}` lines → PASS.
|
||||
- Any other non-zero exit (`Project file not found`, `The game module … could
|
||||
not be found`) → FAIL: no test ran. The exception: the editor never started (exit 126 or 127, `command not found`) — that is the runner not being
|
||||
available, below, and `NOT ASSESSED`.
|
||||
|
||||
**Unknown engine / not configured:**
|
||||
"Engine not configured in `project.yaml` or
|
||||
`.claude/docs/technical-preferences.md`. Run `/setup-engine` to specify the
|
||||
engine, then re-run `/smoke-check`."
|
||||
|
||||
**If the test runner is not available in this environment** (engine binary not
|
||||
on PATH, runner script not found, etc.), report clearly:
|
||||
**If the test runner is not available in this environment** (no editor
|
||||
executable resolved in Phase 1 step 3, runner script not found, etc.), report
|
||||
clearly:
|
||||
|
||||
"Automated tests could not be executed — engine binary not found on PATH.
|
||||
Status will be recorded as NOT RUN. Confirm test results from your local IDE
|
||||
"Automated tests could not be executed — engine executable not found (checked
|
||||
`commands.test`, `engine.path` and `PATH`; set `engine.path`, or put the
|
||||
editor's full path in `commands.test`). Status will be recorded as NOT RUN.
|
||||
Confirm test results from your local IDE
|
||||
or CI pipeline. Until you do, the verdict is NOT ASSESSED — not FAIL, and not
|
||||
a pass either: nothing has been observed about this build yet."
|
||||
|
||||
@@ -193,6 +320,21 @@ Parse runner output and extract:
|
||||
- Names of any failing tests (up to 10; if more, note the count)
|
||||
- Any crash or error output from the runner itself
|
||||
|
||||
**Build check — the WAIVED path only** (`qa.level: minimal`, no game tests;
|
||||
Phase 1 step 1). With no suite to run, run the project's own boot check:
|
||||
`commands.smoke` when set, else `commands.build` when set, with the executable
|
||||
Phase 1 step 3 resolved, under the same timeout rules (`timeout 900`, or
|
||||
`timeout 1800` for `commands.build`; a timeout is a FAIL). On Godot, run the
|
||||
import line above first. Read the result as `/setup-engine` documents it:
|
||||
Godot `smoke` exits 0 even on a parse error, so a `SCRIPT ERROR` line in its
|
||||
output is a FAIL, and so is `Can't run project: no main scene defined` — the
|
||||
game cannot launch; Unity `smoke` exits 1 on a compile error; any other
|
||||
non-zero exit is a FAIL. Record `Build check: PASS — [command] exit 0`,
|
||||
`FAIL — [reason]`, or `NOT RUN — [commands.smoke and commands.build unset |
|
||||
executable not found]`. A build check that did not run is named in the report
|
||||
and leaves the launch in Batch 1 to carry the verdict; it never passes on its
|
||||
own.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Check Test Coverage
|
||||
@@ -233,7 +375,7 @@ fully close those stories.
|
||||
|
||||
## Phase 4: Run Manual Smoke Checks
|
||||
|
||||
Draw the smoke test checklist from, in priority order:
|
||||
Draw the smoke test checklist from, in priority order (and name the one you used in the report):
|
||||
1. The QA plan's "Smoke Test Scope" section (if QA plan was found in Phase 1)
|
||||
2. `production/qa/smoke-tests.md` (if it exists)
|
||||
3. `tests/smoke/` directory contents (if it exists)
|
||||
@@ -254,8 +396,15 @@ sprint's stories.
|
||||
|
||||
Use `AskUserQuestion` to batch-verify. Keep to at most 3 calls.
|
||||
|
||||
**Batch 1 — Core stability (always run):**
|
||||
**Batch 1 — Core stability (always run):** one call, two questions — an empty
|
||||
failure list means nothing unless somebody launched the build.
|
||||
```
|
||||
question: "Did you launch the current build this session?"
|
||||
multiSelect: false
|
||||
options:
|
||||
- "Yes — I launched it and checked the items below"
|
||||
- "No — I have not launched the current build this session"
|
||||
|
||||
question: "Core stability — select any items that FAILED (leave all unselected if everything passed):"
|
||||
multiSelect: true
|
||||
options:
|
||||
@@ -267,6 +416,12 @@ options:
|
||||
|
||||
For any selected item, ask the user to briefly describe what failed before generating the report.
|
||||
|
||||
If the build was **not launched**, the Batch 1 checks could not be executed:
|
||||
record them as `NOT RUN — build not launched this session`, skip Batches 2 and
|
||||
3 and any platform batch (say so — they need a running build), and ignore the
|
||||
unselected failure list. The verdict is then NOT ASSESSED unless something else
|
||||
FAILED.
|
||||
|
||||
**Batch 2 — Sprint changes and regression (always run):**
|
||||
```
|
||||
question: "Sprint changes and regression — select any items that FAILED (leave all unselected if everything passed):"
|
||||
@@ -354,8 +509,16 @@ Assemble the full smoke check report:
|
||||
|
||||
### Automated Tests
|
||||
|
||||
**Status**: [PASS ([N] tests, [N] passing) | FAIL ([N] failures) |
|
||||
NOT RUN ([reason])]
|
||||
**Status**: [PASS ([N] tests, [N] passing) | PASS WITH WARNINGS ([N] tests
|
||||
passing; [the runner's warning, e.g. [N] orphan nodes leaked]) | FAIL ([N]
|
||||
failures, or did not complete: [timeout | compile, parse or build error | no
|
||||
results file]) | NOT ASSESSED ([reason, e.g. zero tests executed, runner could
|
||||
not run]) | NOT RUN ([reason]) | WAIVED (qa.level: minimal — no game tests)]
|
||||
|
||||
[Unity: one Status line per test platform — `EditMode:` and `PlayMode:`.]
|
||||
|
||||
[WAIVED only:] **Build check**: [PASS — `[command]` exit 0 | FAIL — [reason] |
|
||||
NOT RUN — [commands.smoke and commands.build unset | executable not found]]
|
||||
|
||||
[If FAIL, list failing tests:]
|
||||
- `[test name]` — [brief failure description from runner output]
|
||||
@@ -381,6 +544,7 @@ will determine whether the automated test row contributes to a FAIL verdict."
|
||||
|
||||
### Manual Smoke Checks
|
||||
|
||||
- [x] Build launched this session — yes [or: `NOT RUN — build not launched this session`; the batches below were not asked]
|
||||
- [x] Game launches without crash — PASS
|
||||
- [x] New game starts — PASS
|
||||
- [x] [Core mechanic] — PASS
|
||||
@@ -422,13 +586,23 @@ Any platform with one or more FAIL checks contributes to the overall FAIL verdic
|
||||
|
||||
**FAIL** if ANY of:
|
||||
- Automated test suite ran and reported one or more test failures
|
||||
- The automated suite did not complete: a timeout, a compile, parse or build
|
||||
error, or no results file after the run (Phase 2 names each engine's form)
|
||||
- The build check (WAIVED path) failed: a non-zero exit, a timeout, or a Godot
|
||||
`SCRIPT ERROR` or no-main-scene line
|
||||
- Any Batch 1 (core stability) check returned FAIL
|
||||
- Any Batch 2 (primary sprint mechanic or regression check) returned FAIL
|
||||
|
||||
**NOT ASSESSED** if ANY of:
|
||||
- The automated suite is **unconfirmed NOT RUN** — nobody has reported a result
|
||||
- The runner could not run (gdUnit4 103 or 104, gdUnit4 or the Unity Test
|
||||
Framework not installed), or it ran and executed zero tests
|
||||
(`No test cases found`, `testcasecount="0"`, `No automation tests matched`)
|
||||
- Unity Play Mode tests exist and are `PlayMode: NOT RUN`, unconfirmed — the
|
||||
Edit Mode result does not cover them
|
||||
- A Batch 1 or Batch 2 check could not be executed (no build, engine not
|
||||
configured, platform unavailable) as opposed to executing and failing
|
||||
configured, platform unavailable) as opposed to executing and failing —
|
||||
including a Batch 1 answer that the build was not launched this session
|
||||
- **Any story's coverage row is `UNKNOWN`** (Phase 3: story file missing or
|
||||
unreadable). A story nobody could read is not a story with no gaps — without
|
||||
this line, a run where *every* row is UNKNOWN and the suite passes matches
|
||||
@@ -438,13 +612,17 @@ Any platform with one or more FAIL checks contributes to the overall FAIL verdic
|
||||
this line it matches no rule at all and renders as `[-]` beside a PASS
|
||||
|
||||
**PASS WITH WARNINGS** if ALL of:
|
||||
- Automated tests PASS, or NOT RUN **and the developer has confirmed the result
|
||||
from their local IDE or CI**
|
||||
- Automated tests PASS or PASS WITH WARNINGS, WAIVED at `qa.level: minimal`, or
|
||||
NOT RUN **and the developer has confirmed the result from their local IDE or CI**
|
||||
- All Batch 1 and Batch 2 smoke checks PASS
|
||||
- One or more Logic/Integration stories have MISSING test evidence
|
||||
- At least one warning: a Logic/Integration story with MISSING test evidence,
|
||||
a runner warning on a passing suite (gdUnit4 101 — orphan nodes leaked), or
|
||||
a NOT RUN suite the developer confirmed
|
||||
|
||||
**PASS** if ALL of:
|
||||
- Automated tests PASS
|
||||
- Automated tests PASS, or WAIVED (`qa.level: minimal` and no game tests exist —
|
||||
the build check, when `commands.smoke` or `commands.build` is set, and the
|
||||
launch and critical-path batches below then carry the verdict)
|
||||
- All smoke checks in all batches PASS or N/A
|
||||
- No MISSING test evidence entries
|
||||
````
|
||||
@@ -527,10 +705,13 @@ are fixed."
|
||||
run. Do not hand off to QA on this result:
|
||||
|
||||
[Name each check that could not execute, and why: suite unconfirmed NOT RUN,
|
||||
engine binary absent, no build, platform unavailable]
|
||||
`PlayMode: NOT RUN`, zero tests executed, runner not installed, engine
|
||||
executable not found, no build, build not launched this session, platform
|
||||
unavailable]
|
||||
|
||||
[For each, the one thing that would make it runnable — e.g. 'confirm the suite
|
||||
result from your IDE or CI', 'run `/setup-engine`', 'produce a build'.]
|
||||
result from your IDE or CI', 'run `/setup-engine`', 'set `engine.path`',
|
||||
'produce a build', 'launch the build and play the checks'.]
|
||||
|
||||
Re-run `/smoke-check` once any of those is resolved."
|
||||
|
||||
@@ -546,7 +727,8 @@ their behalf.
|
||||
"Smoke check passed with warnings. The build is ready for manual QA.
|
||||
|
||||
Advisory items to resolve before running `/story-done` on affected stories:
|
||||
[list MISSING test evidence entries]
|
||||
[list MISSING test evidence entries, and any runner warning — e.g. the orphan
|
||||
count]
|
||||
|
||||
QA hand-off: share `production/qa/qa-plan-[sprint].md` with the qa-tester
|
||||
agent to begin manual verification."
|
||||
|
||||
Reference in New Issue
Block a user