McRogueFace/tests
John McCardle 30abb0b7ec fix(engine): headless step() is a full simulation frame; --exec can no longer hang
step() advanced animations and timers only. It never called updatePythonScenes(),
which lives solely in doFrame() -- so headless Scene.update(dt) overrides never
fired. That was the filed bug, but it was one of twelve omissions: the C++ scene
update never ran, scene transitions never progressed or COMPLETED, and
current_frame never advanced. Headless behaved measurably differently from a real
frame, and any game with per-frame logic in update() was untestable.

step() is now a full simulation frame: scene update, timers, Python scene update,
animations, transitions (including completion), frame-time metrics, currentFrame++.
Render and input stay out, deliberately -- rendering is orthogonal to the clock and
costs zero simulation time (see renderScene(); a screenshot draws arbitrary state
without time passing). Timers remain on simulation_time: explicit, deterministic
time is the entire point of driving a headless test with step().

This exposed the deeper bug. testTimers() reads simulation_time when headless, and
simulation_time is advanced ONLY by step() -- the doFrame run loop never touched it.
So the headless run loop spun forever on a frozen clock: any headless script that
registered a timer and did not call step() hung until killed. And a script that died
during setup registered no timers, hit the auto-exit-when-no-timers path, exited 0,
and was scored as a PASSING test.

So headless --exec now exits when its scripts finish, nonzero if a script fell off
the end without calling sys.exit(). step() is the clock; the engine cannot advance
time on its own, and pretending otherwise is what produced the hang. A genuinely
long-lived headless process (server, REPL host) opts in with the new --run-forever.

run_tests.py additionally refuses to pass any test whose output contains a
Traceback. Between the two, 82 tests that were being scored green while asserting
nothing are now correctly red; they are repaired in the following commits.

closes #350
2026-07-14 06:37:52 -04:00
..
api Add Game-to-API Bridge for external client integration 2026-01-29 23:08:26 -05:00
benchmarks feat(bench): Gauntlet OOM safety guards + The Crucible wall-clock benchmark; closes #353 closes #354 2026-07-11 10:28:13 -04:00
cookbook Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00
demo Polish #315 pathfinding demo: panel backdrops, base layer, fixed layout 2026-06-26 20:57:23 -04:00
docs Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00
fixtures Test suite modernization 2026-02-09 08:15:18 -05:00
fuzz Fold Tier C surface into existing fuzz targets; closes #312 2026-06-21 16:45:03 -04:00
geometry_demo Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00
integration Add Phase 5.1 end-to-end scenario test for Grid entity behaviors 2026-04-18 05:44:06 -04:00
notes Timer overhaul: update tests 2026-01-03 22:44:53 -05:00
procgen_interactive Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00
regression fix(engine): headless step() is a full simulation frame; --exec can no longer hang 2026-07-14 06:37:52 -04:00
snapshots fix(bindings): make dynamic module attributes dir()-discoverable 2026-07-13 23:47:04 -04:00
unit fix(bindings): make dynamic module attributes dir()-discoverable 2026-07-13 23:47:04 -04:00
vllm_demo update tests: new scene API 2026-01-03 10:59:52 -05:00
all_inputs.py Test suite modernization 2026-02-09 08:15:18 -05:00
conftest.py Add API verification test suite and documentation 2026-01-15 04:05:32 +00:00
debug_viewport.py Test suite modernization 2026-02-09 08:15:18 -05:00
gui.py Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00
KNOWN_ISSUES.md Refactor 11 more tests to mcrfpy.step() pattern 2026-01-14 03:09:47 +00:00
procgen_cave2_visualization.py Test suite modernization 2026-02-09 08:15:18 -05:00
procgen_cave_visualization.py Test suite modernization 2026-02-09 08:15:18 -05:00
pytest.ini Test suite modernization: pytest wrapper and runner fixes 2026-01-14 01:54:31 +00:00
README.md Organize test suite: add README, move loose tests to proper directories 2026-01-21 21:34:22 -05:00
run_procgen_interactive.py Test suite modernization 2026-02-09 08:15:18 -05:00
run_tests.py fix(engine): headless step() is a full simulation frame; --exec can no longer hang 2026-07-14 06:37:52 -04:00
shader_poc_test.py Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00
shader_toggle_test.py Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00
test_mcrogueface.py Test suite modernization: pytest wrapper and runner fixes 2026-01-14 01:54:31 +00:00
wiki_api_verify.py Test suite modernization 2026-02-09 08:15:18 -05:00
wiki_phase_d2_verify.py Test suite modernization 2026-02-09 08:15:18 -05:00
wiki_phase_d3_verify.py Test suite modernization 2026-02-09 08:15:18 -05:00
wiki_phase_d_verify.py Test suite modernization 2026-02-09 08:15:18 -05:00
wiki_snippets_verify.py Remove legacy string enum comparisons from InputState/Key/MouseButton, closes #306 2026-04-09 22:19:02 -04:00

McRogueFace Test Suite

Automated tests for the McRogueFace game engine.

Directory Structure

tests/
├── unit/           # Unit tests for individual components (155+ tests)
├── integration/    # Integration tests for system interactions
├── regression/     # Bug regression tests (issue_XX_*.py)
├── benchmarks/     # Performance benchmarks
├── demo/           # Interactive demo system
│   ├── demo_main.py    # Demo runner
│   └── screens/        # Per-feature demo screens
├── conftest.py     # Pytest configuration and fixtures
├── pytest.ini      # Pytest settings
├── run_tests.py    # Standalone test runner
└── KNOWN_ISSUES.md # Test status and mcrfpy.step() documentation

Running Tests

# Run all tests
pytest tests/ -v

# Run specific directory
pytest tests/unit/ -v

# Run tests matching a pattern
pytest tests/ -k "bsp" -v

# Quick run with short timeout (some timeouts expected)
pytest tests/ -q --mcrf-timeout=5

# Full run with longer timeout
pytest tests/ -q --mcrf-timeout=30

# Stop on first failure
pytest tests/ -x

With run_tests.py

# Run all tests
python3 tests/run_tests.py

# Run specific category
python3 tests/run_tests.py unit
python3 tests/run_tests.py regression

# Verbose output
python3 tests/run_tests.py -v

# Quiet (no checksums)
python3 tests/run_tests.py -q

# Custom timeout
python3 tests/run_tests.py --timeout=30

Running individual tests

cd build
./mcrogueface --headless --exec ../tests/unit/some_test.py

Writing Tests

Test Pattern: Synchronous with mcrfpy.step()

Recommended: Use mcrfpy.step(t) to advance simulation time synchronously.

import mcrfpy
import sys

# Setup
scene = mcrfpy.Scene("test")
scene.activate()

# Create UI elements
frame = mcrfpy.Frame(pos=(100, 100), size=(50, 50))
scene.children.append(frame)

# Start animation
frame.animate("x", 500.0, 1.0, mcrfpy.Easing.LINEAR)

# Advance simulation synchronously
mcrfpy.step(1.5)

# Verify results
if abs(frame.x - 500.0) < 0.1:
    print("PASS")
    sys.exit(0)
else:
    print(f"FAIL: frame.x = {frame.x}, expected 500.0")
    sys.exit(1)

Test Pattern: Timer-based (legacy)

For tests that need multiple timer callbacks or complex timing:

import mcrfpy
import sys

def run_test(runtime):
    # Test code here
    print("PASS")
    sys.exit(0)

scene = mcrfpy.Scene("test")
scene.activate()

# Timer fires after 100ms
timer = mcrfpy.Timer("test", run_test, 100)
# Script ends, game loop runs timer

Test Output

  • Print PASS and sys.exit(0) for success
  • Print FAIL with details and sys.exit(1) for failure
  • Tests that timeout are marked as failures

Test Categories

Unit Tests (unit/)

Test individual components in isolation:

  • *_test.py - Standard component tests
  • api_*_test.py - Python API tests
  • automation_*_test.py - Automation module tests

Regression Tests (regression/)

Tests for specific bug fixes, named by issue number:

  • issue_XX_description_test.py

Integration Tests (integration/)

Tests for system interactions and complex scenarios.

Benchmarks (benchmarks/)

Performance measurement tests.

Demo (demo/)

Interactive demonstrations of features. Run with:

cd build
./mcrogueface ../tests/demo/demo_main.py

Tips

  • Read tests as examples: Tests show correct API usage
  • Use mcrfpy.step(): Avoids timeout issues, makes tests deterministic
  • Check KNOWN_ISSUES.md: Documents expected failures and workarounds
  • Screenshots: Use mcrfpy.automation.screenshot("name.png") after mcrfpy.step()

See Also

  • KNOWN_ISSUES.md - Current test status and the mcrfpy.step() pattern
  • conftest.py - Pytest fixtures and custom collector
  • demo/screens/ - Feature demonstrations (good API examples)