Claude-Code-Game-Studios/CCGS Skill Testing Framework/skills/analysis/consistency-check.md

Skill Test Spec: /consistency-check

Skill Summary

/consistency-check [full | since-last-review | entity:<name> | item:<name>] compares every in-scope GDD in design/gdd/ against the entity registry (design/registry/entities.yaml). It reads the registry once, builds entity, item, formula and constant lookups, then greps each GDD for each registered name (-C 3) instead of reading whole documents; only conflicts get a targeted section read. The registry entry's source: GDD is the authoritative owner. In a GDD whose effective tier is standard — the project tier, or that system's system_overrides row — only the required GDD sections are compared (plus Tuning Knobs when workflow_overrides.tuning_knobs is true), and the report says so. Findings are classified 🔴 CONFLICT, ⚠️ STALE REGISTRY or ℹ️ UNVERIFIABLE, and the report verdict is PASS or CONFLICTS FOUND — or NOT ASSESSED when the registry is empty, a named entity is not in it, or no GDD is in scope, since a scan of nothing is not a PASS. Registry corrections are written only after a "May I update / May I add" ask; afterwards the verdict is COMPLETE, or BLOCKED while conflicts remain. Every 🔴 CONFLICT is appended to docs/consistency-failures.md after a "May I append" ask, a breadcrumb is appended to production/session-state/active.md, and the skill closes with an AskUserQuestion widget (collaborative and guided modes). No director gates.

---

Static Assertions (Structural)

Checked against the SKILL.md by /skill-test spec — no fixture needed.

• Has required frontmatter fields: name, description, argument-hint, user-invocable, allowed-tools
• Has ≥2 phase headings
• Contains verdict keywords: PASS, CONFLICTS FOUND, NOT ASSESSED, COMPLETE, BLOCKED
• The analysis phases (1–5) are read-only
• Has a next-step handoff at the end
• Registry writes are preceded by "May I update" / "May I add" language naming design/registry/entities.yaml
• The conflict log write is preceded by "May I append" language naming docs/consistency-failures.md

---

Director Gate Checks

No director gates — this skill spawns no director gate agents. Consistency checking is a mechanical scan; no creative or technical director review is required as part of the scan itself.

---

Test Cases

Case 1: Happy Path — Registry and 4 GDDs agree

Fixture:

• modes.workflow resolves to full; modes.automation resolves to collaborative
• design/registry/entities.yaml has 3 entities and 2 constants, each with a source: GDD
• design/gdd/ contains 4 system GDDs plus game-concept.md, systems-index.md and a gdd-cross-review-[date].md
• Every value the GDDs state for a registered name matches the registry

Input: /consistency-check

Expected behavior:

1. Skill reads the registry and reports Registry loaded: 3 entities, 0 items, 0 formulas, 2 constants and Scope: full
2. Skill globs design/gdd/*.md, excludes game-concept.md, systems-index.md and the cross-review report, and reports the 4 in-scope GDDs before scanning
3. Each registered name is grepped across the in-scope GDDs with 3 lines of context; no GDD is read in full
4. No conflicts or stale entries; the report's Clean Entries line reads ✅ 5 registry entries verified across all GDDs with no conflicts.
5. Verdict: PASS
6. Nothing is appended to docs/consistency-failures.md; the breadcrumb is appended to production/session-state/active.md
7. Closes with the AskUserQuestion widget (Fix / Run /design-review / Stop)

Assertions:

• The in-scope GDD list (4 GDDs, non-system docs excluded) is reported before the scan
• The scan greps for registered names rather than fully reading each GDD
• Verdict is PASS when registry and GDDs agree
• design/registry/entities.yaml and docs/consistency-failures.md are not written; the only write is the active.md breadcrumb
• The breadcrumb names docs/consistency-failures.md, not an invented report file
• The skill closes with the AskUserQuestion widget, not plain text

---

Case 2: Failure Path — A GDD contradicts a registered constant

Fixture:

• modes.workflow resolves to full
• Registry constants: has crit_multiplier value 1.5, source: design/gdd/combat.md, referenced_by: [combat.md, loot.md]
• design/gdd/combat.md states crit_multiplier = 1.5
• design/gdd/loot.md states crit_multiplier = 2.0
• The registry entry is newer than the last commit to combat.md (not stale)

Input: /consistency-check

Expected behavior:

1. The constant scan (3d) greps crit_multiplier and finds 2.0 in loot.md
2. Phase 4 does a targeted read of the conflicting section and uses the registry source: to decide combat.md is authoritative
3. Report lists 🔴 crit_multiplier: Registry (source: combat.md) = 1.5; Conflict in loot.md = 2.0; Resolution: change loot.md to 1.5
4. Verdict: CONFLICTS FOUND
5. Skill asks "May I append 1 entries to docs/consistency-failures.md (creating it if absent)?"; on approval an entry for the conflict is appended (the file is created with its header first if absent), naming combat.md vs loot.md
6. No GDD is edited automatically; with the conflict unresolved the final verdict is BLOCKED

Assertions:

• Verdict is CONFLICTS FOUND (not PASS)
• The conflict entry names both GDDs and shows both values (1.5 and 2.0)
• The conflict is classified 🔴 CONFLICT, and the resolution targets the non-source GDD (loot.md)
• A reflexion-log entry is appended to docs/consistency-failures.md, only after the "May I append" ask naming it is approved
• Skill does NOT auto-resolve the conflict — neither GDD is edited without the user choosing to fix it
• Final verdict is BLOCKED while the conflict is unresolved

---

Case 3: Stale Registry — The source GDD changed after the registry entry

Fixture:

• modes.workflow resolves to full
• Registry entities: has goblin with health: 40, source: design/gdd/combat.md
• design/gdd/combat.md now says goblin health is 50; git log shows it changed after the registry entry was written
• No other GDD states a goblin health value

Input: /consistency-check entity:goblin

Expected behavior:

1. Scope is limited to the goblin entity across all GDDs; the report shows Scope: entity:goblin
2. The mismatch is found in the source GDD itself; Phase 4 checks git history and classifies it ⚠️ STALE REGISTRY, not 🔴 CONFLICT
3. Report lists it under Stale Registry Entries: registry says 40, source GDD now says 50
4. Skill asks: "May I update design/registry/entities.yaml to fix the 1 stale entries?"
5. On approval: health becomes 50, revised: is set to today, and a # was: 40 before [date] comment is added; nothing is deleted
6. After writing, verdict is COMPLETE
7. No entry is appended to docs/consistency-failures.md (only 🔴 CONFLICT entries are logged)

Assertions:

• The finding is classified ⚠️ STALE REGISTRY, distinct from 🔴 CONFLICT
• The registry is not written until the "May I update design/registry/entities.yaml" ask is approved
• The approved update sets revised: and keeps the old value in a # was: comment
• Verdict after the write is COMPLETE
• docs/consistency-failures.md is not written for a stale-registry finding

---

Case 4: Edge Case — Empty registry, no GDDs

Fixture:

• modes.workflow resolves to full
• design/registry/entities.yaml is the shipped stub with no entries
• design/gdd/ is empty

Input: /consistency-check

Expected behavior:

1. Skill reads the registry and finds no entries
2. Skill outputs: "Entity registry is empty. Run /design-system to write GDDs — the registry is populated automatically after each GDD is completed. Nothing to check yet."
3. Verdict: NOT ASSESSED — entity registry empty; the skill stops without scanning
4. No file is written — not the registry, not the reflexion log, not the session-state breadcrumb

Assertions:

• Skill outputs the empty-registry message naming /design-system
• Verdict is NOT ASSESSED, naming the empty registry as the reason — never PASS, CONFLICTS FOUND, COMPLETE or BLOCKED, and never no verdict at all (.claude/rules/skill-authoring.md obligation 1)
• No file is written, including production/session-state/active.md
• Skill does NOT crash or produce a partial report

---

Case 4b: Edge Case — Registry populated, but no GDD in scope

Fixture:

• modes.workflow resolves to full
• design/registry/entities.yaml has 3 entities, each with a source: GDD
• design/gdd/gdd-cross-review-[date].md exists, and no system GDD has changed since it

Input: /consistency-check since-last-review

Expected behavior:

1. Skill reads the registry and reports what it loaded
2. The since-last-review filter finds zero changed GDDs; the in-scope list is empty
3. Verdict: NOT ASSESSED — no GDDs in scope, naming since-last-review as the filter that produced zero; the skill stops without scanning

Assertions:

• Verdict is NOT ASSESSED, never PASS — nothing was compared
• The output names the filter that produced the empty scope
• No GDD is scanned and no conflict log entry is written

---

Case 4c: Edge Case — Named entity is not in the registry

Fixture:

• modes.workflow resolves to full
• design/registry/entities.yaml has entities goblin and orc_warlord
• design/gdd/ contains 3 system GDDs; one mentions a goblin_shaman

Input: /consistency-check entity:goblin_shaman

Expected behavior:

1. Skill reads the registry and reports what it loaded, Scope: entity:goblin_shaman
2. goblin_shaman has no registry entry
3. Verdict: NOT ASSESSED — goblin_shaman is not in the registry, listing the closest registered names (goblin); the skill stops without scanning

Assertions:

• Verdict is NOT ASSESSED naming the unregistered entity — never PASS
• The closest registered names are listed
• No GDD is scanned and nothing is written

---

Case 5: Director Gate — No gate spawned in full review mode

Fixture:

• project.yaml: modes.review_mode: full, modes.workflow: full
• design/registry/entities.yaml has entries
• design/gdd/ contains ≥2 GDDs

Input: /consistency-check

Expected behavior:

1. Skill loads the registry and runs the consistency scan
2. The review mode plays no part: the skill's config block resolves only automation, workflow and system_overrides
3. No director gate agents are spawned at any point
4. Report and verdict are produced normally

Assertions:

• No director gate agents are spawned (no CD-, TD-, PR-, AD- prefixed gates)
• The skill does not resolve or read the review mode — its config block asks only for automation, workflow and system_overrides
• Output contains no "Gate: [GATE-ID]" or gate-skipped entries
• Review mode full has no effect on this skill's behavior

---

Case 6: Standard Workflow — Only the required GDD sections are compared

Fixture:

• modes.workflow resolves to standard
• Registry constants: has crit_multiplier value 1.5, source: design/gdd/combat.md
• design/gdd/combat.md ## Detailed Design states crit_multiplier = 1.5
• design/gdd/loot.md states crit_multiplier = 2.0 only in its ## Tuning Knobs section, which standard does not require; no required section of loot.md states a value for it

Input: /consistency-check

Expected behavior:

1. Only hits inside the sections standard requires are compared
2. The loot.md value in ## Tuning Knobs is not compared: no 🔴 CONFLICT is raised for it and nothing is appended to docs/consistency-failures.md
3. The report's Scope line says the check covered the required sections only, and lists the loot.md Tuning Knobs mention as not checked at this tier
4. Verdict: PASS — the required sections agree with the registry

Assertions:

• The Tuning Knobs value is not reported as a 🔴 CONFLICT at standard
• The report states the required-sections-only scope and names the mention it did not check — the narrower scan is not silent
• Verdict is PASS

---

Case 6b: Standard Workflow — A per-system override makes one GDD full

Fixture:

• modes.workflow resolves to standard; project.yaml has workflow_overrides.system_overrides.loot: full
• Registry constants: has crit_multiplier value 1.5, source: design/gdd/combat.md
• design/gdd/combat.md ## Detailed Design states crit_multiplier = 1.5
• design/gdd/loot.md states crit_multiplier = 2.0 only in its ## Tuning Knobs section

Input: /consistency-check

Expected behavior:

1. The config block lists system_overrides: loot=full, so loot.md's effective tier is full and every one of its sections is compared; combat.md stays at standard
2. The loot.md Tuning Knobs value is compared and raised as 🔴 crit_multiplier (registry 1.5, loot.md 2.0)
3. The report's Scope line names loot.md as checked at full
4. Verdict: CONFLICTS FOUND

Assertions:

• The Tuning Knobs value in loot.md is compared, because its effective tier is full — the project tier alone would have skipped it (Case 6)
• The report names the GDD whose effective tier differs from the project's
• Verdict is CONFLICTS FOUND, not PASS

---

Protocol Compliance

• Reads the registry once, then greps GDDs; full-section reads happen only for conflicts
• The report is shown in full before any registry write ask
• Report verdict is PASS, CONFLICTS FOUND or NOT ASSESSED (empty registry, unregistered name, or no GDD in scope — never PASS over nothing compared); post-correction verdict is COMPLETE or BLOCKED
• Applies each GDD's effective tier (the resolved modes.workflow, or that system's system_overrides row): at standard, compares the required GDD sections only (plus Tuning Knobs when workflow_overrides.tuning_knobs is true) and says so in the report
• No director gates — the review mode is not an input
• Registry writes are gated by "May I update" / "May I add"; entries are never deleted (status: deprecated instead)
• Every 🔴 CONFLICT is appended to docs/consistency-failures.md after a "May I append" ask that names it
• Closes with AskUserQuestion in collaborative and guided modes; in autonomous mode prints findings and records via log_decision

---

Coverage Notes

• This skill checks registered facts across GDDs. Deep design theory analysis (pillar drift, dominant strategies) is handled by /review-all-gdds.
• Facts that are not in the registry are not compared; two GDDs that disagree on an unregistered value are not detected. Phase 6 offers to register names that appear in more than one GDD.
• since-last-review is tested only on its empty-scope path (Case 4b), not on a run that finds changed GDDs.
• modes.workflow: minimal is not tested separately: a minimal project normally has no GDDs, and an empty scope ends NOT ASSESSED as in Cases 4 and 4b.