# /smoke-check — Phase 2: Run Automated Tests > Part of `/smoke-check`. `SKILL.md` names when to read this file; the settings it > resolved and the rules under its later headings still apply here. ## Phase 2: Run Automated Tests **In this file:** - `commands.test` first - Godot 4 - Unity: Edit Mode, then Play Mode as a second run - Unreal Engine: build first, commands per OS, macOS - Unknown engine / not configured - Test runner not available here - Build check (the WAIVED path only) **Run `commands.test` from `project.yaml` when it is set** — it is the project's own test command, the one `/setup-engine` wrote and CI runs. Use the per-engine default below only when `commands.test` is absent, and say in the report which of the two you ran. Wrap it in a timeout (`timeout 900 …`): **a timeout (exit 124) is a gate FAILURE, never a pass** — the run never completed and nothing was verified. macOS has no `timeout`: use `gtimeout 900 …` from Homebrew's `coreutils` (same exit 124), or, with nothing installed, `perl -e 'alarm shift; exec @ARGV' 900 …`, which exits 142 when time runs out — a FAILURE the same way. Then read the result for the engine. On every engine one outcome looks like success and is not; each is named below. **Godot 4** — first check that `addons/gdUnit4/bin/GdUnitCmdTool.gd` exists — the folder has a **capital U**. If it does not, report `NOT ASSESSED — gdUnit4 not installed at addons/gdUnit4/` instead of running anything. Then import, whether you run `commands.test` or the default. A fresh clone has no `.godot/` class cache, and without it gdUnit4's runner does not load; a cache that predates a new `class_name` fails valid code with 105. The import takes seconds when nothing changed (`godot` below is the executable Phase 1 step 3 resolved): ```bash godot --headless --path . --import godot --headless -s -d --remote-debug tcp://127.0.0.1:0 res://addons/gdUnit4/bin/GdUnitCmdTool.gd -a res://tests --ignoreHeadlessMode 2>&1 ``` gdUnit4's exit code gives the verdict (6.1.3): - **0** — every test passed: PASS. **But exit 0 over zero tests is not a pass:** gdUnit4 aborts and still exits 0, so if the output says `No test cases found`, the verdict is `NOT ASSESSED — no tests found`. - **101** — every test passed but a test leaked nodes (orphans): PASS WITH WARNINGS, naming the orphan count from the output. Not a failure. - **100** — a test failed: FAIL. **105** — a test script does not parse: FAIL. - **103** — gdUnit4 refused to run headless (`--ignoreHeadlessMode` missing); **104** — Godot older than 4.3: `NOT ASSESSED`, with that reason. - **1** — Godot's own error: `GdUnitCmdTool.gd` did not load (`Failed to load script`), so no test ran. Usually the import above was skipped; after it, exit 1 is a FAIL. Every run opens with two `ERROR:` lines about the remote port (`127.0.0.1:0`) — that is the `--remote-debug` flag working, not a failure. Never drop the flag: without it a script error opens Godot's interactive debugger and the run hangs at a `debug>` prompt until the timeout. **Unity:** Unity **can** run tests headlessly via shell. Do not skip to reading artifacts. First confirm the Unity Test Framework is installed. A project without it does not fail cleanly — it **hangs** until something kills it, which is the observation the old "Unity cannot test headlessly" advice was generalised from: ```bash grep -q 'com.unity.test-framework' Packages/manifest.json && echo present || echo ABSENT ``` If ABSENT, report `NOT ASSESSED — Unity Test Framework not installed` and give the one-line fix (add `com.unity.test-framework` to `Packages/manifest.json`). **Do not fall through to reading stale artifacts** — an unknown-age XML reported as a pass is worse than no gate. If present, the default is (the **editor** executable — `Unity` on `PATH` may be Unity's separate CLI). Delete the old results file first, whether you run `commands.test` or this default: ```bash rm -f test-results/editmode.xml timeout 900 "" -batchmode -runTests -projectPath . -testPlatform EditMode -testResults test-results/editmode.xml ``` Exit 0 when every test passes, 2 on a failure. **Exit 1 or 3 is a FAIL**: 1 means the project did not compile (the editor log has the `error CS…` lines), 3 that the run itself failed; 4 (an unknown `-testPlatform`) or any other non-zero exit is a FAIL too. A compile error writes no results file — which is why the old one is deleted first: left in place, the last run's pass is read as this run's. After any exit, no results file is a FAIL. Parse the `` element of the results file for `testcasecount`, `passed` and `failed`. **`testcasecount="0"` is NOT ASSESSED, never a pass**: tests outside `Assets/` are never compiled, and Unity then reports `result="Passed"` with exit 0 over zero tests. **Play Mode is a second run.** `-testPlatform EditMode` — this default, and `commands.test` as `/setup-engine` writes it — never runs the integration tests `/test-setup` scaffolds under `Assets/Tests/PlayMode/`. When that folder holds a test file and the command you ran did not already pass `-testPlatform PlayMode`, run Play Mode the same way, into its own results file, and read it by the same rules: ```bash rm -f test-results/playmode.xml timeout 900 "" -batchmode -runTests -projectPath . -testPlatform PlayMode -testResults test-results/playmode.xml ``` Report each platform on its own line — `EditMode: PASS (12/12)`, `PlayMode: FAIL (1 failure)`. A Play Mode run that could not be made is reported by name, `PlayMode: NOT RUN — [reason]`: the Edit Mode result does not cover those tests, so it cannot make the automated-test row a PASS. **Unreal Engine** — build the editor target first, whether you then run `commands.test` or the default. `Binaries/` is not committed, and without a built game module the editor stops with "The game module could not be found", exit 1, before running any test. The build is incremental — seconds when nothing changed; a failed build is a FAIL. **Pick the commands for this machine** with `uname -s`: `Linux` → Linux, `Darwin` → macOS, anything else (`MINGW*`, `MSYS*`, `CYGWIN*`) → Windows. The Windows forms below were run on UE 5.7; the Linux and macOS forms come from Epic's documentation — `docs/engine-reference/unreal/current-best-practices.md`, "Command Line", has them with their sources. On Windows use `UnrealBuildTool.exe`, not `Build.bat`: run from bash, `Build.bat` fails on an engine path with spaces. ```bash # Windows timeout 1800 "/Engine/Binaries/DotNET/UnrealBuildTool/UnrealBuildTool.exe" Editor Win64 Development -Project="$(pwd -W 2>/dev/null || pwd)/.uproject" 2>&1 # Linux timeout 1800 "/Engine/Build/BatchFiles/Linux/Build.sh" Editor Linux Development -Project="$(pwd -W 2>/dev/null || pwd)/.uproject" 2>&1 # macOS (timeout: see the macOS note above) gtimeout 1800 "/Engine/Build/BatchFiles/Mac/Build.sh" Editor Mac Development -Project="$(pwd -W 2>/dev/null || pwd)/.uproject" 2>&1 ``` Then the default. Keep `-stdout -FullStdOutLogOutput`; when `commands.test` lacks them, add them to the command for this run only: without them the editor prints nothing, and the lines below are only in `Saved/Logs/.log`. Give the project as an **absolute** path — when `commands.test` passes a relative one, run it with the absolute path instead: UE 5.7 does not find a relative `.uproject` and exits 1 (`Project file not found`). `$(pwd -W 2>/dev/null || pwd)` gives the `C:/…` form in Git Bash and the plain path elsewhere; `$PWD` alone fails when `MSYS_NO_PATHCONV` is set. Either way, say in the report what you changed and tell the user to fix `commands.test` in `project.yaml` — never edit `project.yaml` without asking. `.` in the filter is the root the tests are named under: the project name, or the distinct root `/setup-engine` chose when the project's name is also an engine area — read one test's name string under `Source/*/Private/Tests/` if unsure. ```bash # Windows timeout 1800 "/Engine/Binaries/Win64/UnrealEditor-Cmd.exe" "$(pwd -W 2>/dev/null || pwd)/.uproject" -ExecCmds="Automation RunTests .; Quit" -unattended -nullrhi -stdout -FullStdOutLogOutput 2>&1 # Linux timeout 1800 "/Engine/Binaries/Linux/UnrealEditor" "$(pwd -W 2>/dev/null || pwd)/.uproject" -ExecCmds="Automation RunTests .; Quit" -unattended -nullrhi -stdout -FullStdOutLogOutput 2>&1 ``` **macOS has no sourced test command.** Epic documents only the `UnrealEditor.app` bundle, not a command-line editor inside it. Run a `commands.test` the user has set and confirmed; with none, report `NOT ASSESSED — Unreal test command on macOS not sourced; set commands.test to the editor command you use` instead of guessing a path. Read the output, not the exit code: a failing test and a run that matched nothing both exit 255. - `Result={Fail}` on any line → FAIL. - `No automation tests matched` → `NOT ASSESSED — no tests found` (tests outside `Source//` are never built), not FAIL. - Exit 0 with `Result={Success}` lines → PASS. - Any other non-zero exit (`Project file not found`, `The game module … could not be found`) → FAIL: no test ran. The exception: the editor never started (exit 126 or 127, `command not found`) — that is the runner not being available, below, and `NOT ASSESSED`. **Unknown engine / not configured:** "Engine not configured in `project.yaml` or `.claude/docs/technical-preferences.md`. Run `/setup-engine` to specify the engine, then re-run `/smoke-check`." **If the test runner is not available in this environment** (no editor executable resolved in Phase 1 step 3, runner script not found, etc.), report clearly: "Automated tests could not be executed — engine executable not found (checked `commands.test`, `engine.path` and `PATH`; set `engine.path`, or put the editor's full path in `commands.test`). Status will be recorded as NOT RUN. Confirm test results from your local IDE or CI pipeline. Until you do, the verdict is NOT ASSESSED — not FAIL, and not a pass either: nothing has been observed about this build yet." Do not treat NOT RUN as an automatic FAIL. Record it, and let the developer's manual confirmation in Phase 4 resolve it. Until that confirmation arrives the verdict is **NOT ASSESSED** (see the verdict rules in Phase 5), which ranks above both pass values and below FAIL. An unrun suite is not a healthy build; it is an unknown one, and the two need different follow-ups. Parse runner output and extract: - Total tests run - Passing count - Failing count - Names of any failing tests (up to 10; if more, note the count) - Any crash or error output from the runner itself **Build check — the WAIVED path only** (`qa.level: minimal`, no game tests; Phase 1 step 1). With no suite to run, run the project's own boot check: `commands.smoke` when set, else `commands.build` when set, with the executable Phase 1 step 3 resolved, under the same timeout rules (`timeout 900`, or `timeout 1800` for `commands.build`; a timeout is a FAIL). On Godot, run the import line above first. Read the result as `/setup-engine` documents it: Godot `smoke` exits 0 even on a parse error, so a `SCRIPT ERROR` line in its output is a FAIL, and so is `Can't run project: no main scene defined` — the game cannot launch; Unity `smoke` exits 1 on a compile error; any other non-zero exit is a FAIL. Record `Build check: PASS — [command] exit 0`, `FAIL — [reason]`, or `NOT RUN — [commands.smoke and commands.build unset | executable not found]`. A build check that did not run is named in the report and leaves the launch in Batch 1 to carry the verdict; it never passes on its own. ---