McRogueFace/tests
John McCardle a46667df6f test(snippets): the docs site's 130 code samples are now part of the test suite
The samples published on mcrogueface.github.io were executed by nothing at all.
Each carried a machine-readable header -- objects=[...] verified=0.2.8@b6720f6
status=ok -- and every one of those fields was hand-typed. 130 snippets asserted
"status=ok" while no run had ever confirmed it. This is the same failure mode as
the 82 rotted tests in 112f357, at a larger scale and with no gate whatsoever.

The samples now live here, in the engine repo, so that breaking the API breaks the
build. The site pulls them from tests/snippets/ rather than keeping a copy to rot
in parallel (build_library.py, site-side).

They are display scripts: each builds a scene and stops. They deliberately do NOT
call sys.exit(), because they must stay copy-pasteable -- a reader who pastes one
into their game should get a running scene, not an interpreter that quits. Since
#350 that also makes them un-runnable on their own, so _harness.py is chained as a
second --exec to supply the ending the snippet must not have. Chaining rather than
exec()-ing the source is deliberate: the snippet travels the same path a real
user's script travels, and nothing has to read the file -- which is how the first
draft of this harness tripped over #378 and mistook three healthy snippets for
broken ones.

A snippet passes only if it runs clean AND leaves a scene with something in it.
Not raising is not enough: a sample whose body silently no-ops still renders as
code on the docs site, and would still be wrong.

tools/stamp_snippets.py makes the header a measurement instead of a claim: status
comes from an actual run, verified from the engine that ran it, and objects is
derived from the source intersected with what the engine exports -- so a snippet
that starts using a new type gets tagged for it without anyone remembering to. It
immediately found tags the humans had missed (062 uses Transition, Key and
InputState; the hand-written header listed none of them). --check is the CI gate.

The gate earned its keep on the first run, catching #379 (fixed here): configuring
default_transition before the first scene activation made the engine transition
FROM the internal "uitest" bootstrap scene, which is not a PyScene and has no
Python wrapper -- so mcrfpy.current_scene reported None for the transition's full
duration, immediately after the user had assigned it. A transition with nothing
meaningful to transition from now changes scene immediately. Scene-to-scene
transitions still report the outgoing scene until they complete.

Suite: 468/468 (336 + 130 snippets + 2 regressions).

closes #379
2026-07-14 18:29:37 -04:00
..
api Add Game-to-API Bridge for external client integration 2026-01-29 23:08:26 -05:00
benchmarks feat(bench): Gauntlet OOM safety guards + The Crucible wall-clock benchmark; closes #353 closes #354 2026-07-11 10:28:13 -04:00
cookbook Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00
demo fix(demos): unrot tests/demo and gate it in CI 2026-07-14 06:46:03 -04:00
docs Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00
fixtures Test suite modernization 2026-02-09 08:15:18 -05:00
fuzz Fold Tier C surface into existing fuzz targets; closes #312 2026-06-21 16:45:03 -04:00
geometry_demo Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00
integration test(suite): unrot 82 tests that were passing without ever running 2026-07-14 07:29:23 -04:00
notes Timer overhaul: update tests 2026-01-03 22:44:53 -05:00
procgen_interactive Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00
regression test(snippets): the docs site's 130 code samples are now part of the test suite 2026-07-14 18:29:37 -04:00
snapshots fix(bindings): make dynamic module attributes dir()-discoverable 2026-07-13 23:47:04 -04:00
snippets test(snippets): the docs site's 130 code samples are now part of the test suite 2026-07-14 18:29:37 -04:00
unit test(suite): unrot 82 tests that were passing without ever running 2026-07-14 07:29:23 -04:00
vllm_demo update tests: new scene API 2026-01-03 10:59:52 -05:00
all_inputs.py Test suite modernization 2026-02-09 08:15:18 -05:00
conftest.py Add API verification test suite and documentation 2026-01-15 04:05:32 +00:00
debug_viewport.py Test suite modernization 2026-02-09 08:15:18 -05:00
gui.py Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00
KNOWN_ISSUES.md Refactor 11 more tests to mcrfpy.step() pattern 2026-01-14 03:09:47 +00:00
procgen_cave2_visualization.py Test suite modernization 2026-02-09 08:15:18 -05:00
procgen_cave_visualization.py Test suite modernization 2026-02-09 08:15:18 -05:00
pytest.ini Test suite modernization: pytest wrapper and runner fixes 2026-01-14 01:54:31 +00:00
README.md Organize test suite: add README, move loose tests to proper directories 2026-01-21 21:34:22 -05:00
run_procgen_interactive.py Test suite modernization 2026-02-09 08:15:18 -05:00
run_tests.py test(snippets): the docs site's 130 code samples are now part of the test suite 2026-07-14 18:29:37 -04:00
shader_poc_test.py Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00
shader_toggle_test.py Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00
test_mcrogueface.py Test suite modernization: pytest wrapper and runner fixes 2026-01-14 01:54:31 +00:00
wiki_api_verify.py Test suite modernization 2026-02-09 08:15:18 -05:00
wiki_phase_d2_verify.py Test suite modernization 2026-02-09 08:15:18 -05:00
wiki_phase_d3_verify.py Test suite modernization 2026-02-09 08:15:18 -05:00
wiki_phase_d_verify.py Test suite modernization 2026-02-09 08:15:18 -05:00
wiki_snippets_verify.py Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00

McRogueFace Test Suite

Automated tests for the McRogueFace game engine.

Directory Structure

tests/
├── unit/           # Unit tests for individual components (155+ tests)
├── integration/    # Integration tests for system interactions
├── regression/     # Bug regression tests (issue_XX_*.py)
├── benchmarks/     # Performance benchmarks
├── demo/           # Interactive demo system
│   ├── demo_main.py    # Demo runner
│   └── screens/        # Per-feature demo screens
├── conftest.py     # Pytest configuration and fixtures
├── pytest.ini      # Pytest settings
├── run_tests.py    # Standalone test runner
└── KNOWN_ISSUES.md # Test status and mcrfpy.step() documentation

Running Tests

# Run all tests
pytest tests/ -v

# Run specific directory
pytest tests/unit/ -v

# Run tests matching a pattern
pytest tests/ -k "bsp" -v

# Quick run with short timeout (some timeouts expected)
pytest tests/ -q --mcrf-timeout=5

# Full run with longer timeout
pytest tests/ -q --mcrf-timeout=30

# Stop on first failure
pytest tests/ -x

With run_tests.py

# Run all tests
python3 tests/run_tests.py

# Run specific category
python3 tests/run_tests.py unit
python3 tests/run_tests.py regression

# Verbose output
python3 tests/run_tests.py -v

# Quiet (no checksums)
python3 tests/run_tests.py -q

# Custom timeout
python3 tests/run_tests.py --timeout=30

Running individual tests

cd build
./mcrogueface --headless --exec ../tests/unit/some_test.py

Writing Tests

Test Pattern: Synchronous with mcrfpy.step()

Recommended: Use mcrfpy.step(t) to advance simulation time synchronously.

import mcrfpy
import sys

# Setup
scene = mcrfpy.Scene("test")
scene.activate()

# Create UI elements
frame = mcrfpy.Frame(pos=(100, 100), size=(50, 50))
scene.children.append(frame)

# Start animation
frame.animate("x", 500.0, 1.0, mcrfpy.Easing.LINEAR)

# Advance simulation synchronously
mcrfpy.step(1.5)

# Verify results
if abs(frame.x - 500.0) < 0.1:
    print("PASS")
    sys.exit(0)
else:
    print(f"FAIL: frame.x = {frame.x}, expected 500.0")
    sys.exit(1)

Test Pattern: Timer-based (legacy)

For tests that need multiple timer callbacks or complex timing:

import mcrfpy
import sys

def run_test(runtime):
    # Test code here
    print("PASS")
    sys.exit(0)

scene = mcrfpy.Scene("test")
scene.activate()

# Timer fires after 100ms
timer = mcrfpy.Timer("test", run_test, 100)
# Script ends, game loop runs timer

Test Output

  • Print PASS and sys.exit(0) for success
  • Print FAIL with details and sys.exit(1) for failure
  • Tests that timeout are marked as failures

Test Categories

Unit Tests (unit/)

Test individual components in isolation:

  • *_test.py - Standard component tests
  • api_*_test.py - Python API tests
  • automation_*_test.py - Automation module tests

Regression Tests (regression/)

Tests for specific bug fixes, named by issue number:

  • issue_XX_description_test.py

Integration Tests (integration/)

Tests for system interactions and complex scenarios.

Benchmarks (benchmarks/)

Performance measurement tests.

Demo (demo/)

Interactive demonstrations of features. Run with:

cd build
./mcrogueface ../tests/demo/demo_main.py

Tips

  • Read tests as examples: Tests show correct API usage
  • Use mcrfpy.step(): Avoids timeout issues, makes tests deterministic
  • Check KNOWN_ISSUES.md: Documents expected failures and workarounds
  • Screenshots: Use mcrfpy.automation.screenshot("name.png") after mcrfpy.step()

See Also

  • KNOWN_ISSUES.md - Current test status and the mcrfpy.step() pattern
  • conftest.py - Pytest fixtures and custom collector
  • demo/screens/ - Feature demonstrations (good API examples)