john/McRogueFace
1
0
Fork 0
Code Issues 43 Pull requests Projects Releases Wiki Activity

Untrack generated docs; add API manifest infra and pre-commit hook

Browse source 
Rendered/generated documentation no longer lives in git -- it's regenerated
by a pre-commit hook (tools/hooks/pre-commit, installed via `make
install-hooks`) and shipped inside release artifacts (tools/package.sh) instead.
A compact api/manifest.json (signatures + docstring hashes + carried-forward
lifecycle metadata: since/modified per object and member) is the one exception
and stays committed, refreshed by the same hook, so API changes remain
trackable by commit/tag via `git show <ref>:api/manifest.json` and
`tools/api_delta.py REF1 REF2` without needing a full rebuild at old refs.

- tools/generate_api_manifest.py: engine-side introspection, run headless;
  emits api/manifest.json (committed) and docs/generated/api_full.json
  (gitignored, full docstrings -- the docs-site generator's data source).
- tools/api_delta.py: stdlib-only diff CLI (md/json/gitea-checklist formats),
  with --site-dir to cross-reference affected docs-site pages via their mcrf
  frontmatter.
- tools/hooks/pre-commit: incremental build + frozen-docstring gate + doc
  regen + manifest refresh, gated on staged src/ changes.
- .gitignore covers the rendered docs/stubs plus personal working-notes docs
  (plans, audits, research write-ups) that were never McRogueFace-user-facing.
- docs/api-stability.md and docs/threading-model.md removed: their content is
  now the wiki's "API Stability Contract" and "Threading Model" system pages
  (ratified 2026-07-02, migration verified 2026-07-11); the repo continues to
  enforce them via tests/unit/api_surface_snapshot_test.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
John McCardle 2026-07-11 20:57:31 -04:00
commit 54624b353d
22 changed files with 1194 additions and 25884 deletions
Download patch file Download diff file Expand all files Collapse all files

26
.gitignore vendored
View file

@ -56,3 +56,29 @@ dist/
# Fuzzing build artifacts and runtime data (build-fuzz matched by build* above)
tests/fuzz/corpora/
tests/fuzz/crashes/
# >>> generated-docs (untrack via tools/untrack_generated_docs.sh) >>>
# Rendered/generated documentation is regenerated by the pre-commit hook and
# shipped inside release artifacts -- it no longer lives in git. The compact
# api/manifest.json is the exception: it STAYS committed (refreshed by the hook).
# Personal-notes docs (plans, triage, research) are also untracked here.
# tools/untrack_generated_docs.sh reads exactly this block to migrate the
# currently-tracked copies out of the index.
docs/API_REFERENCE_DYNAMIC.md
docs/api_reference_dynamic.html
docs/mcrfpy.3
stubs/mcrfpy.pyi
docs/generated/
docs/API_REFERENCE_COMPLETE.md
docs/api_reference_complete.html
docs/VISION.md
docs/GAMES.md
docs/plan-*.md
docs/api-audit-*.md
docs/ISSUE_TRIAGE_*.md
docs/GRID_ENTITY_OVERHAUL_ROADMAP.md
docs/sprint-*.md
docs/EMSCRIPTEN_RESEARCH.md
docs/WASM_TROUBLESHOOTING.md
docs/MCROGUFACE_LITE_PICOCALC_RESEARCH.md
# <<< generated-docs <<<

12
Makefile
View file

@ -36,6 +36,7 @@
.PHONY: version-bump
.PHONY: debug debug-test asan asan-test tsan tsan-test valgrind-test massif-test analyze clean-debug
.PHONY: profile callgrind clean-profile
.PHONY: install-hooks
# Number of parallel jobs for compilation
JOBS := $(shell nproc 2>/dev/null || echo 4)
@ -84,6 +85,17 @@ clean-all: clean clean-windows clean-wasm clean-debug clean-profile clean-dist
run: linux
@cd build && ./mcrogueface
# Install git hooks by symlinking tools/hooks/* into .git/hooks/ (idempotent).
install-hooks:
@hooks_dir="$$(git rev-parse --git-path hooks)"; \
mkdir -p "$$hooks_dir"; \
for hook in tools/hooks/*; do \
[ -e "$$hook" ] || continue; \
name="$$(basename "$$hook")"; \
ln -sf "$$(pwd)/$$hook" "$$hooks_dir/$$name"; \
echo "Linked $$hooks_dir/$$name -> $$hook"; \
done
# Debug and sanitizer targets
debug:
@echo "Building McRogueFace with debug Python (pydebug assertions)..."

1
api/manifest.json Normal file
View file

File diff suppressed because one or more lines are too long

1158
docs/API_REFERENCE_COMPLETE.md
View file

File diff suppressed because it is too large Load diff

5867
docs/API_REFERENCE_DYNAMIC.md
View file

File diff suppressed because it is too large Load diff

860
docs/EMSCRIPTEN_RESEARCH.md
View file

@ -1,860 +0,0 @@
# McRogueFace Emscripten & Renderer Abstraction Research
**Date**: 2026-01-30
**Branch**: `emscripten-mcrogueface`
**Related Issues**: #157 (True Headless), #158 (Emscripten/WASM)
## Executive Summary
This document analyzes the technical requirements for:
1. **SFML 2.6 → 3.0 migration** (modernization)
2. **Emscripten/WebAssembly compilation** (browser deployment)
Both goals share a common prerequisite: **renderer abstraction**. The codebase already has a partial abstraction via `sf::RenderTarget*` pointer, but SFML types are pervasive (1276 occurrences across 78 files).
**Key Insight**: This is a **build-time configuration**, not runtime switching. The standard McRogueFace binary remains a dynamic environment; Emscripten builds bundle assets and scripts at compile time.
---
## Current Architecture Analysis
### Existing Abstraction Strengths
1. **RenderTarget Pointer Pattern** (`GameEngine.h:156`)
```cpp
sf::RenderTarget* render_target;
// Points to either window.get() or headless_renderer->getRenderTarget()
```
This already decouples rendering logic from the specific backend.
2. **HeadlessRenderer** (`src/HeadlessRenderer.h`)
- Uses `sf::RenderTexture` internally
- Provides unified interface: `getRenderTarget()`, `display()`, `saveScreenshot()`
- Demonstrates the pattern for additional backends
3. **UIDrawable Hierarchy**
- Virtual `render(sf::Vector2f, sf::RenderTarget&)` method
- 7 drawable types: Frame, Caption, Sprite, Entity, Grid, Line, Circle, Arc
- Each manages its own SFML primitives internally
4. **Asset Wrappers**
- `PyTexture`, `PyFont`, `PyShader` wrap SFML types
- Python reference counting integrated
- Single point of change for asset loading APIs
### Current SFML Coupling Points
| Area | Count | Difficulty | Notes |
|------|-------|------------|-------|
| `sf::Vector2f` | ~200+ | Medium | Used everywhere for positions, sizes |
| `sf::Color` | ~100+ | Easy | Simple 4-byte struct replacement |
| `sf::FloatRect` | ~50+ | Medium | Bounds, intersection testing |
| `sf::RenderTexture` | ~20 | Hard | Shader effects, caching |
| `sf::Sprite/Text` | ~30 | Hard | Core rendering primitives |
| `sf::Event` | ~15 | Medium | Input system coupling |
| `sf::Keyboard/Mouse` | ~50+ | Easy | Enum mappings |
Total: **1276 occurrences across 78 files**
---
## SFML 3.0 Migration Analysis
### Breaking Changes Requiring Code Updates
#### 1. Vector Parameters (High Impact)
```cpp
// SFML 2.6
setPosition(10, 20);
sf::VideoMode(1024, 768, 32);
sf::FloatRect(x, y, w, h);
// SFML 3.0
setPosition({10, 20});
sf::VideoMode({1024, 768}, 32);
sf::FloatRect({x, y}, {w, h});
```
**Strategy**: Regex-based search/replace with manual verification.
#### 2. Rect Member Changes (Medium Impact)
```cpp
// SFML 2.6
rect.left, rect.top, rect.width, rect.height
rect.getPosition(), rect.getSize()
// SFML 3.0
rect.position.x, rect.position.y, rect.size.x, rect.size.y
rect.position, rect.size // direct access
rect.findIntersection() -> std::optional<Rect<T>>
```
#### 3. Resource Constructors (Low Impact)
```cpp
// SFML 2.6
sf::Sound sound; // default constructible
sound.setBuffer(buffer);
// SFML 3.0
sf::Sound sound(buffer); // requires buffer at construction
```
#### 4. Keyboard/Mouse Enum Scoping (Medium Impact)
```cpp
// SFML 2.6
sf::Keyboard::A
sf::Mouse::Left
// SFML 3.0
sf::Keyboard::Key::A
sf::Mouse::Button::Left
```
#### 5. Event Handling (Medium Impact)
```cpp
// SFML 2.6
sf::Event event;
while (window.pollEvent(event)) {
if (event.type == sf::Event::Closed) ...
}
// SFML 3.0
while (auto event = window.pollEvent()) {
if (event->is<sf::Event::Closed>()) ...
}
```
#### 6. CMake Target Changes
```cmake
# SFML 2.6
find_package(SFML 2 REQUIRED COMPONENTS graphics audio)
target_link_libraries(app sfml-graphics sfml-audio)
# SFML 3.0
find_package(SFML 3 REQUIRED COMPONENTS Graphics Audio)
target_link_libraries(app SFML::Graphics SFML::Audio)
```
### Migration Effort Estimate
| Phase | Files | Changes | Effort |
|-------|-------|---------|--------|
| CMakeLists.txt | 1 | Target names | 1 hour |
| Vector parameters | 30+ | ~200 calls | 4-8 hours |
| Rect refactoring | 20+ | ~50 usages | 2-4 hours |
| Event handling | 5 | ~15 sites | 2 hours |
| Keyboard/Mouse | 10 | ~50 enums | 2 hours |
| Resource constructors | 10 | ~30 sites | 2 hours |
| **Total** | - | - | **~15-25 hours** |
---
## Emscripten/VRSFML Analysis
### Why VRSFML Over Waiting for SFML 4.x?
1. **Available Now**: VRSFML is working today with browser demos
2. **Modern OpenGL**: Removes legacy calls, targets OpenGL ES 3.0+ (WebGL 2)
3. **SFML_GAME_LOOP Macro**: Handles blocking vs callback loop abstraction
4. **Performance**: 500k sprites @ 60FPS vs 3 FPS upstream (batching)
5. **SFML 4.x Timeline**: Unknown, potentially years away
### VRSFML API Differences from SFML
| Feature | SFML 2.6/3.0 | VRSFML |
|---------|--------------|--------|
| Default constructors | Allowed | Not allowed for resources |
| Texture ownership | Pointer in Sprite | Passed at draw time |
| Context management | Hidden global | Explicit `GraphicsContext` |
| Drawable base class | Polymorphic | Removed |
| Loading methods | `loadFromFile()` returns bool | Returns `std::optional` |
| Main loop | `while(running)` | `SFML_GAME_LOOP { }` |
### Main Loop Refactoring
Current blocking loop:
```cpp
void GameEngine::run() {
while (running) {
processEvents();
update();
render();
display();
}
}
```
Emscripten-compatible pattern:
```cpp
// Option A: VRSFML macro
SFML_GAME_LOOP {
processEvents();
update();
render();
display();
}
// Option B: Manual Emscripten integration
#ifdef __EMSCRIPTEN__
void mainLoopCallback() {
if (!game.running) {
emscripten_cancel_main_loop();
return;
}
game.doFrame();
}
emscripten_set_main_loop(mainLoopCallback, 0, 1);
#else
while (running) { doFrame(); }
#endif
```
**Recommendation**: Use preprocessor-based approach with `doFrame()` extraction for cleaner separation.
---
## Build-Time Configuration Strategy
### Normal Build (Desktop)
- Dynamic loading of assets from `assets/` directory
- Python scripts loaded from `scripts/` directory at runtime
- Full McRogueFace environment with dynamic game loading
### Emscripten Build (Web)
- Assets bundled via `--preload-file assets`
- Scripts bundled via `--preload-file scripts`
- Virtual filesystem (MEMFS/IDBFS)
- Optional: Script linting with Pyodide before bundling
- Single-purpose deployment (one game per build)
### CMake Configuration
```cmake
option(MCRF_BUILD_EMSCRIPTEN "Build for Emscripten/WebAssembly" OFF)
if(MCRF_BUILD_EMSCRIPTEN)
set(CMAKE_TOOLCHAIN_FILE ${CMAKE_SOURCE_DIR}/cmake/toolchains/emscripten.cmake)
add_definitions(-DMCRF_EMSCRIPTEN)
# Bundle assets
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} \
--preload-file ${CMAKE_SOURCE_DIR}/assets@/assets \
--preload-file ${CMAKE_SOURCE_DIR}/scripts@/scripts")
endif()
```
---
## Phased Implementation Plan
### Phase 0: Preparation (This PR)
- [ ] Create `docs/EMSCRIPTEN_RESEARCH.md` (this document)
- [ ] Update Gitea issues #157, #158 with findings
- [ ] Identify specific files requiring changes
- [ ] Create test matrix for rendering features
### Phase 1: Type Abstraction Layer
**Goal**: Isolate SFML types behind McRogueFace wrappers
```cpp
// src/types/McrfTypes.h
namespace mcrf {
using Vector2f = sf::Vector2f; // Alias initially, replace later
using Color = sf::Color;
using FloatRect = sf::FloatRect;
}
```
Changes:
- [ ] Create `src/types/` directory with wrapper types
- [ ] Gradually replace `sf::` with `mcrf::` namespace
- [ ] Update Common.h to provide both namespaces during transition
### Phase 2: Main Loop Extraction
**Goal**: Make game loop callback-compatible
- [ ] Extract `GameEngine::doFrame()` from `run()`
- [ ] Add `#ifdef __EMSCRIPTEN__` conditional in `run()`
- [ ] Test that desktop behavior is unchanged
### Phase 3: Render Backend Interface
**Goal**: Abstract RenderTarget operations
```cpp
class RenderBackend {
public:
virtual ~RenderBackend() = default;
virtual void clear(const Color& color) = 0;
virtual void draw(const Sprite& sprite) = 0;
virtual void draw(const Text& text) = 0;
virtual void display() = 0;
virtual bool isOpen() const = 0;
virtual Vector2u getSize() const = 0;
};
class SFMLBackend : public RenderBackend { ... };
class VRSFMLBackend : public RenderBackend { ... }; // Future
```
### Phase 4: SFML 3.0 Migration
**Goal**: Update to SFML 3.0 API
- [ ] Update CMakeLists.txt targets
- [ ] Fix vector parameter calls
- [ ] Fix rect member access
- [ ] Fix event handling
- [ ] Fix keyboard/mouse enums
- [ ] Test thoroughly
### Phase 5: VRSFML Integration (Experimental)
**Goal**: Add VRSFML as alternative backend
- [ ] Add VRSFML as submodule/dependency
- [ ] Implement VRSFMLBackend
- [ ] Add Emscripten CMake configuration
- [ ] Test in browser
### Phase 6: Python-in-WASM
**Goal**: Get Python scripting working in browser
**High Risk** - This is the major unknown:
- [ ] Build CPython for Emscripten
- [ ] Test `McRFPy_API` binding compatibility
- [ ] Evaluate Pyodide vs raw CPython
- [ ] Handle filesystem virtualization
- [ ] Test threading limitations
---
## Risk Assessment
| Risk | Probability | Impact | Mitigation |
|------|-------------|--------|------------|
| SFML 3.0 breaks unexpected code | Medium | Medium | Comprehensive test suite |
| VRSFML API too different | Low | High | Can fork/patch VRSFML |
| Python-in-WASM fails | Medium | Critical | Evaluate Pyodide early |
| Performance regression | Low | Medium | Benchmark before/after |
| Binary size too large | Medium | Medium | Lazy loading, stdlib trimming |
---
## References
### SFML 3.0
- [Migration Guide](https://www.sfml-dev.org/tutorials/3.0/getting-started/migrate/)
- [Changelog](https://www.sfml-dev.org/development/changelog/)
- [Release Notes](https://github.com/SFML/SFML/releases/tag/3.0.0)
### VRSFML/Emscripten
- [VRSFML Blog Post](https://vittorioromeo.com/index/blog/vrsfml.html)
- [VRSFML GitHub](https://github.com/vittorioromeo/VRSFML)
- [Browser Demos](https://vittorioromeo.github.io/VRSFML_HTML5_Examples/)
### Python WASM
- [PEP 776 - Python Emscripten Support](https://peps.python.org/pep-0776/)
- [CPython WASM Build Guide](https://github.com/python/cpython/blob/main/Tools/wasm/README.md)
- [Pyodide](https://github.com/pyodide/pyodide)
### Related Issues
- [SFML Emscripten Discussion #1494](https://github.com/SFML/SFML/issues/1494)
- [libtcod Emscripten #41](https://github.com/libtcod/libtcod/issues/41)
---
## Appendix A: File-by-File SFML Usage Inventory
### Critical Files (Must Abstract for Emscripten)
| File | SFML Types Used | Role | Abstraction Difficulty |
|------|-----------------|------|------------------------|
| `GameEngine.h/cpp` | RenderWindow, Clock, Font, Event | Main loop, window | **CRITICAL** |
| `HeadlessRenderer.h/cpp` | RenderTexture | Headless backend | **CRITICAL** |
| `UIDrawable.h/cpp` | Vector2f, RenderTarget, FloatRect | Base render interface | **HARD** |
| `UIFrame.h/cpp` | RectangleShape, Vector2f, Color | Container rendering | **HARD** |
| `UISprite.h/cpp` | Sprite, Texture, Vector2f | Texture display | **HARD** |
| `UICaption.h/cpp` | Text, Font, Vector2f, Color | Text rendering | **HARD** |
| `UIGrid.h/cpp` | RenderTexture, Sprite, Vector2f | Tile grid system | **HARD** |
| `UIEntity.h/cpp` | Sprite, Vector2f | Game entities | **HARD** |
| `UICircle.h/cpp` | CircleShape, Vector2f, Color | Circle shape | **MEDIUM** |
| `UILine.h/cpp` | VertexArray, Vector2f, Color | Line rendering | **MEDIUM** |
| `UIArc.h/cpp` | CircleShape segments, Vector2f | Arc shape | **MEDIUM** |
| `Scene.h/cpp` | Vector2f, RenderTarget | Scene management | **MEDIUM** |
| `SceneTransition.h/cpp` | RenderTexture, Sprite | Transitions | **MEDIUM** |
### Wrapper Files (Already Partially Abstracted)
| File | SFML Types Wrapped | Python API | Notes |
|------|-------------------|------------|-------|
| `PyVector.h/cpp` | sf::Vector2f | Vector | Ready for backend swap |
| `PyColor.h/cpp` | sf::Color | Color | Ready for backend swap |
| `PyTexture.h/cpp` | sf::Texture | Texture | Asset loading needs work |
| `PyFont.h/cpp` | sf::Font | Font | Asset loading needs work |
| `PyShader.h/cpp` | sf::Shader | Shader | Optional feature |
### Input System Files
| File | SFML Types Used | Notes |
|------|-----------------|-------|
| `ActionCode.h` | Keyboard::Key, Mouse::Button | Enum encoding only |
| `PyKey.h/cpp` | Keyboard::Key enum | 140+ key mappings |
| `PyMouseButton.h/cpp` | Mouse::Button enum | Simple enum |
| `PyKeyboard.h/cpp` | Keyboard::isKeyPressed | State queries |
| `PyMouse.h/cpp` | Mouse::getPosition | Position queries |
| `PyInputState.h/cpp` | None (pure enum) | No SFML dependency |
### Support Files (Low Priority)
| File | SFML Types Used | Notes |
|------|-----------------|-------|
| `Animation.h/cpp` | Vector2f, Color (as values) | Pure data animation |
| `GridLayers.h/cpp` | RenderTexture, Color | Layer caching |
| `IndexTexture.h/cpp` | Texture, IntRect | Legacy texture format |
| `Resources.h/cpp` | Font | Global font storage |
| `ProfilerOverlay.cpp` | Text, RectangleShape | Debug overlay |
| `McRFPy_Automation.h/cpp` | Various | Testing only |
---
## Appendix B: Recommended First Steps
### Immediate (Non-Breaking Changes)
1. **Extract `GameEngine::doFrame()`**
- Move loop body to separate method
- No API changes, just internal refactoring
- Enables future Emscripten callback integration
2. **Create type aliases in Common.h**
```cpp
namespace mcrf {
using Vector2f = sf::Vector2f;
using Vector2i = sf::Vector2i;
using Color = sf::Color;
using FloatRect = sf::FloatRect;
}
```
- Allows gradual migration from `sf::` to `mcrf::`
- No functional changes
3. **Document current render path**
- Add comments to key rendering functions
- Identify all `target.draw()` call sites
- Create rendering flow diagram
### Short-Term (Preparation for SFML 3.0)
1. **Audit vector parameter calls**
- Find all `setPosition(x, y)` style calls
- Prepare regex patterns for migration
2. **Audit rect member access**
- Find all `.left`, `.top`, `.width`, `.height` uses
- Prepare for `.position.x`, `.size.x` style
3. **Test suite expansion**
- Add rendering validation tests
- Screenshot comparison tests
- Animation correctness tests
---
## Appendix C: libtcod Architecture Analysis
**Key Finding**: libtcod uses a much simpler abstraction pattern than initially proposed.
### libtcod's Context Vtable Pattern
libtcod doesn't wrap every SDL type. Instead, it abstracts at the **context level** using a C-style vtable:
```c
struct TCOD_Context {
int type;
void* contextdata_; // Backend-specific data (opaque pointer)
// Function pointers - the "vtable"
void (*c_destructor_)(struct TCOD_Context* self);
TCOD_Error (*c_present_)(struct TCOD_Context* self,
const TCOD_Console* console,
const TCOD_ViewportOptions* viewport);
void (*c_pixel_to_tile_)(struct TCOD_Context* self, double* x, double* y);
TCOD_Error (*c_save_screenshot_)(struct TCOD_Context* self, const char* filename);
struct SDL_Window* (*c_get_sdl_window_)(struct TCOD_Context* self);
TCOD_Error (*c_set_tileset_)(struct TCOD_Context* self, TCOD_Tileset* tileset);
TCOD_Error (*c_screen_capture_)(struct TCOD_Context* self, ...);
// ... more operations
};
```
### How Backends Implement It
Each renderer fills in the function pointers:
```c
// In renderer_sdl2.c
context->c_destructor_ = sdl2_destructor;
context->c_present_ = sdl2_present;
context->c_get_sdl_window_ = sdl2_get_window;
// ...
// In renderer_xterm.c
context->c_destructor_ = xterm_destructor;
context->c_present_ = xterm_present;
// ...
```
### Conditional Compilation with NO_SDL
libtcod uses simple preprocessor guards:
```c
// In CMakeLists.txt
if(LIBTCOD_SDL3)
target_link_libraries(${PROJECT_NAME} PUBLIC SDL3::SDL3)
else()
target_compile_definitions(${PROJECT_NAME} PUBLIC NO_SDL)
endif()
// In source files
#ifndef NO_SDL
#include <SDL3/SDL.h>
// ... SDL-dependent code ...
#endif
```
**47 files** use this pattern. When building headless, SDL code is simply excluded.
### Why This Pattern Works
1. **Core functionality is SDL-independent**: Console manipulation, pathfinding, FOV, noise, BSP, etc. don't need SDL
2. **Only rendering needs abstraction**: The `TCOD_Context` is the single point of abstraction
3. **Minimal API surface**: Just ~10 function pointers instead of wrapping every primitive
4. **Backend-specific data is opaque**: `contextdata_` holds renderer-specific state
### Implications for McRogueFace
**libtcod's approach suggests we should NOT try to abstract every `sf::` type.**
Instead, consider:
1. **Keep SFML types internally** - `sf::Vector2f`, `sf::Color`, `sf::FloatRect` are fine
2. **Abstract at the RenderContext level** - One vtable for window/rendering operations
3. **Use `#ifndef NO_SFML` guards** - Compile-time backend selection
4. **Create alternative backend for Emscripten** - WebGL + canvas implementation
### Proposed McRogueFace Context Pattern
```cpp
struct McRF_RenderContext {
void* backend_data; // SFML or WebGL specific data
// Function pointers
void (*destroy)(McRF_RenderContext* self);
void (*clear)(McRF_RenderContext* self, uint32_t color);
void (*present)(McRF_RenderContext* self);
void (*draw_sprite)(McRF_RenderContext* self, const Sprite* sprite);
void (*draw_text)(McRF_RenderContext* self, const Text* text);
void (*draw_rect)(McRF_RenderContext* self, const Rect* rect);
bool (*poll_event)(McRF_RenderContext* self, Event* event);
void (*screenshot)(McRF_RenderContext* self, const char* path);
// ...
};
// SFML backend
McRF_RenderContext* mcrf_sfml_context_new(int width, int height, const char* title);
// Emscripten backend (future)
McRF_RenderContext* mcrf_webgl_context_new(const char* canvas_id);
```
### Comparison: Original Plan vs libtcod-Inspired Plan
| Aspect | Original Plan | libtcod-Inspired Plan |
|--------|---------------|----------------------|
| Type abstraction | Replace all `sf::*` with `mcrf::*` | Keep `sf::*` internally |
| Abstraction point | Every primitive type | Single Context object |
| Files affected | 78+ files | ~10 core files |
| Compile-time switching | Complex namespace aliasing | Simple `#ifndef NO_SFML` |
| Backend complexity | Full reimplementation | Focused vtable |
**Recommendation**: Adopt libtcod's simpler pattern. Focus abstraction on the rendering context, not on data types.
---
## Appendix D: Headless Build Experiment Results
**Experiment Date**: 2026-01-30
**Branch**: `emscripten-mcrogueface`
### Objective
Attempt to compile McRogueFace without SFML dependencies to identify true coupling points.
### What We Created
1. **`src/platform/HeadlessTypes.h`** - Complete SFML type stubs (~600 lines):
- Vector2f, Vector2i, Vector2u
- Color with standard color constants
- FloatRect, IntRect
- Time, Clock (with chrono-based implementation)
- Transform, Vertex, View
- Shape hierarchy (RectangleShape, CircleShape, etc.)
- Texture, Sprite, Font, Text stubs
- RenderTarget, RenderTexture, RenderWindow stubs
- Audio stubs (Sound, Music, SoundBuffer)
- Input stubs (Keyboard, Mouse, Event)
- Shader stub
2. **Modified `src/Common.h`** - Conditional include:
```cpp
#ifdef MCRF_HEADLESS
#include "platform/HeadlessTypes.h"
#else
#include <SFML/Graphics.hpp>
#include <SFML/Audio.hpp>
#endif
```
### Build Attempt Result
**SUCCESS** - Headless build compiles after consolidating includes and adding stubs.
### Work Completed
#### 1. Consolidated SFML Includes
**15 files** had direct SFML includes that bypassed Common.h. All were modified to use `#include "Common.h"` instead:
| File | Original Include | Fixed |
|------|------------------|-------|
| `main.cpp` | `<SFML/Graphics.hpp>` | ✓ |
| `Animation.h` | `<SFML/Graphics.hpp>` | ✓ |
| `GridChunk.h` | `<SFML/Graphics.hpp>` | ✓ |
| `GridLayers.h` | `<SFML/Graphics.hpp>` | ✓ |
| `HeadlessRenderer.h` | `<SFML/Graphics.hpp>` | ✓ |
| `SceneTransition.h` | `<SFML/Graphics.hpp>` | ✓ |
| `McRFPy_Automation.h` | `<SFML/Graphics.hpp>`, `<SFML/Window.hpp>` | ✓ |
| `PyWindow.cpp` | `<SFML/Graphics.hpp>` | ✓ |
| `ActionCode.h` | `<SFML/Window/Keyboard.hpp>` | ✓ |
| `PyKey.h` | `<SFML/Window/Keyboard.hpp>` | ✓ |
| `PyMouseButton.h` | `<SFML/Window/Mouse.hpp>` | ✓ |
| `PyBSP.h` | `<SFML/System/Vector2.hpp>` | ✓ |
| `UIGridPathfinding.h` | `<SFML/System/Vector2.hpp>` | ✓ |
#### 2. Wrapped ImGui-SFML with Guards
ImGui-SFML is disabled entirely in headless builds since debug tools can't be accessed through the API:
| File | Changes |
|------|---------|
| `GameEngine.h` | Guarded includes and member variables |
| `GameEngine.cpp` | Guarded all ImGui::SFML calls |
| `ImGuiConsole.h/cpp` | Entire file wrapped with `#ifndef MCRF_HEADLESS` |
| `ImGuiSceneExplorer.h/cpp` | Entire file wrapped with `#ifndef MCRF_HEADLESS` |
| `McRFPy_API.cpp` | Guarded ImGuiConsole include and setEnabled call |
#### 3. Extended HeadlessTypes.h
The stub file grew from ~700 lines to ~900 lines with additional types and methods:
**Types Added:**
- `sf::Image` - For screenshot functionality
- `sf::Glsl::Vec3`, `sf::Glsl::Vec4` - For shader uniforms
- `sf::BlendMode` - For rendering states
- `sf::CurrentTextureType` - For shader texture binding
**Methods Added:**
- `Font::Info` struct and `Font::getInfo()`
- `Texture::update()` overloads
- `Texture::copyToImage()`
- `Transform::getInverse()`
- `RenderStates` constructors from Transform, BlendMode, Shader*
- `Music::getDuration()`, `getPlayingOffset()`, `setPlayingOffset()`
- `SoundBuffer::getDuration()`
- `RenderWindow::setMouseCursorGrabbed()`
- `sf::err()` stream function
- Keyboard aliases: `BackSpace`, `BackSlash`, `SemiColon`, `Dash`
### Build Commands
```bash
# Normal SFML build (default)
make
# Headless build (no SFML/ImGui dependencies)
mkdir build-headless && cd build-headless
cmake .. -DMCRF_HEADLESS=ON -DCMAKE_BUILD_TYPE=Release
make
```
### Key Insight
The libtcod approach of `#ifndef NO_SDL` guards works when **all platform includes go through a single point**. The consolidation of 15+ bypass points into Common.h was the prerequisite that made this work.
### Actual Effort
| Task | Files | Time |
|------|-------|------|
| Replace direct SFML includes with Common.h | 15 | ~30 min |
| Wrap ImGui-SFML in guards | 5 | ~20 min |
| Extend HeadlessTypes.h with missing stubs | 1 | ~1 hour |
| Fix compilation errors iteratively | - | ~1 hour |
**Total**: ~3 hours for clean headless compilation
### Completed Milestones
1. ✅ **Test Python bindings** - mcrfpy module loads and works in headless mode
- Vector, Color, Scene, Frame, Grid all functional
- libtcod integrations (BSP, pathfinding) available
2. ✅ **Add CMake option** - `option(MCRF_HEADLESS "Build without graphics" OFF)`
- Proper conditional compilation and linking
- No SFML symbols in headless binary
3. ✅ **Link-time validation** - `ldd` confirms zero SFML/OpenGL dependencies
4. ✅ **Binary size reduction** - Headless is 1.6 MB vs 2.5 MB normal build (36% smaller)
### Python Test Results (Headless Mode)
```python
# All these work in headless build:
import mcrfpy
v = mcrfpy.Vector(10, 20) # ✅
c = mcrfpy.Color(255, 128, 64) # ✅
scene = mcrfpy.Scene('test') # ✅
frame = mcrfpy.Frame(pos=(0,0)) # ✅
grid = mcrfpy.Grid(grid_size=(10,10)) # ✅
```
### Remaining Steps for Emscripten
1. ✅ **Main loop extraction** - `GameEngine::doFrame()` extracted with Emscripten callback support
- `run()` now uses `#ifdef __EMSCRIPTEN__` to choose between callback and blocking loop
- `emscripten_set_main_loop_arg()` integration ready
2. ✅ **Emscripten toolchain** - `emcmake cmake` works with headless mode
3. ✅ **Python-in-WASM** - Built CPython 3.14.2 for wasm32-emscripten target
- Uses official `Tools/wasm/emscripten build` script from CPython repo
- Produced libpython3.14.a (47MB static library)
- Also builds: libmpdec, libffi, libexpat for WASM
4. ✅ **libtcod-in-WASM** - Built libtcod-headless for Emscripten
- Uses `LIBTCOD_SDL3=OFF` to avoid SDL dependency
- Includes lodepng and utf8proc dependencies
5. ✅ **First successful WASM build** - mcrogueface.wasm (8.9MB) + mcrogueface.js (126KB)
- All 68 C++ source files compile with emcc
- Links: Python, libtcod, HACL crypto, expat, mpdec, ffi, zlib, bzip2, sqlite3
6. 🔲 **Python stdlib bundling** - Need to package Python stdlib for WASM filesystem
7. 🔲 **VRSFML integration** - Replace stubs with actual WebGL rendering
### First Emscripten Build Attempt (2026-01-31)
**Command:**
```bash
source ~/emsdk/emsdk_env.sh
emcmake cmake .. -DMCRF_HEADLESS=ON -DCMAKE_BUILD_TYPE=Release
emmake make -j8
```
**Result:** Build failed on Python headers.
**Key Errors:**
```
deps/Python/pyport.h:429:2: error: "LONG_BIT definition appears wrong for platform"
```
```
warning: shift count >= width of type [-Wshift-count-overflow]
_Py_STATIC_FLAG_BITS << 48 // 48-bit shift on 32-bit WASM!
```
**Root Cause:**
1. Desktop Python 3.14 headers assume 64-bit Linux with glibc
2. Emscripten targets 32-bit WASM with musl-based libc
3. Python's immortal reference counting uses `<< 48` shifts that overflow on 32-bit
4. `LONG_BIT` check fails because WASM's `long` is 32 bits
**Analysis:**
The HeadlessTypes.h stubs and game engine code compile fine. The blocker is exclusively the Python C API integration.
### Python-in-WASM Options
| Option | Complexity | Description |
|--------|------------|-------------|
| **Pyodide** | Medium | Pre-built Python WASM with package ecosystem |
| **CPython WASM** | High | Build CPython ourselves with Emscripten |
| **No-Python mode** | Low | New CMake option to exclude Python entirely |
**Pyodide Approach (Recommended):**
- Pyodide provides Python 3.12 compiled for WASM
- Would need to replace `deps/Python` with Pyodide headers
- `McRFPy_API` binding layer needs adaptation
- Pyodide handles asyncio, file system virtualization
- Active project with good documentation
### CPython WASM Build (Successful!)
**Date**: 2026-01-31
Used the official CPython WASM build process:
```bash
# From deps/cpython directory
./Tools/wasm/emscripten build
# This produces:
# - cross-build/wasm32-emscripten/build/python/libpython3.14.a
# - cross-build/wasm32-emscripten/prefix/lib/libmpdec.a
# - cross-build/wasm32-emscripten/prefix/lib/libffi.a
# - cross-build/wasm32-emscripten/build/python/Modules/expat/libexpat.a
```
**CMake Integration:**
```cmake
if(EMSCRIPTEN)
set(PYTHON_WASM_BUILD "${CMAKE_SOURCE_DIR}/deps/cpython/cross-build/wasm32-emscripten/build/python")
set(PYTHON_WASM_PREFIX "${CMAKE_SOURCE_DIR}/deps/cpython/cross-build/wasm32-emscripten/prefix")
# Force WASM-compatible pyconfig.h
add_compile_options(-include ${PYTHON_WASM_BUILD}/pyconfig.h)
# Link all Python dependencies
set(LINK_LIBS
${PYTHON_WASM_BUILD}/libpython3.14.a
${PYTHON_WASM_BUILD}/Modules/_hacl/*.o # HACL crypto not in libpython
${PYTHON_WASM_BUILD}/Modules/expat/libexpat.a
${PYTHON_WASM_PREFIX}/lib/libmpdec.a
${PYTHON_WASM_PREFIX}/lib/libffi.a
)
# Emscripten ports for common libraries
target_link_options(mcrogueface PRIVATE
-sUSE_ZLIB=1
-sUSE_BZIP2=1
-sUSE_SQLITE3=1
)
endif()
```
**No-Python Mode (For Testing):**
- Add `MCRF_NO_PYTHON` CMake option
- Allows testing WASM build without Python complexity
- Game engine would be pure C++ (no scripting)
- Useful for validating rendering, input, timing first
### Main Loop Architecture
The game loop now supports both desktop (blocking) and browser (callback) modes:
```cpp
// GameEngine::run() - build-time conditional
#ifdef __EMSCRIPTEN__
emscripten_set_main_loop_arg(emscriptenMainLoopCallback, this, 0, 1);
#else
while (running) { doFrame(); }
#endif
// GameEngine::doFrame() - same code runs in both modes
void GameEngine::doFrame() {
metrics.resetPerFrame();
currentScene()->update();
testTimers();
// ... animations, input, rendering ...
currentFrame++;
frameTime = clock.restart().asSeconds();
}
```

273
docs/ISSUE_TRIAGE_2026-04.md
View file

@ -1,273 +0,0 @@
# McRogueFace Issue Triage — April 2026
**46 open issues** across #53–#304. Grouped by system, ordered by impact.
---
## Group 1: Render Cache Dirty Flags (Bugfix Cluster)
**4 issues — all quick-to-moderate fixes, high user-visible impact**
These are systemic bugs where Python property setters bypass the render cache invalidation system (#144). They cause stale frames when using `clip_children` or `cache_subtree`. Issue #291 is the umbrella audit; the other three are specific bugs it identified.
| Issue | Title | Difficulty |
|-------|-------|------------|
| #291 | Audit all Python property setters for missing markDirty() calls | Medium — systematic sweep of all tp_getset setters |
| #290 | UIDrawable base x/y/pos setters don't propagate dirty flags to parent | Quick — add markCompositeDirty() call in set_float_member() |
| #289 | Caption Python property setters don't call markDirty() | Quick — add markDirty() to text/font_size/fill_color setters |
| #288 | UICollection mutations don't invalidate parent Frame's render cache | Quick — add markCompositeDirty() in append/remove/etc |
**Dependencies:** None external. #291 depends on #288–#290 being fixed first (or done together).
**Recommendation: Tackle first.** These are correctness bugs affecting every user of the caching system. The fixes are mechanical (add missing dirty-flag calls), low risk, and testable. One focused session can close all four.
---
## Group 2: Grid Dangling Pointer Bugs
**3 issues — moderate fixes, memory safety impact**
All three are the same class of bug: raw `UIGrid*` pointers in child objects that dangle when the parent grid is destroyed. Part of the broader memory safety audit (#279).
| Issue | Title | Difficulty |
|-------|-------|------------|
| #270 | GridLayer::parent_grid dangling raw pointer | Moderate — convert to weak_ptr or add invalidation |
| #271 | UIGridPoint::parent_grid dangling raw pointer | Moderate — same pattern as #270 |
| #277 | GridChunk::parent_grid dangling raw pointer | Moderate — same pattern as #270 |
**Dependencies:** These are the last 3 unfixed bugs from the memory safety audit (#279). Fixing all three would effectively close #279.
**Recommendation: Tackle second.** Same fix pattern applied three times. Closes the memory safety audit chapter.
---
## Group 3: Animation System Fixes
**2 issues — one bugfix, one feature**
| Issue | Title | Difficulty |
|-------|-------|------------|
| #256 | Animation system bypasses spatial hash updates for entity position | Moderate — hook animation property changes into spatial hash |
| #218 | Color and Vector animation targets | Minor feature — compound property animation support |
**Dependencies:** #256 is independent. #218 is a nice-to-have that improves DX.
---
## Group 4: Grid Layer & Rendering Fixes
**2 issues — quick fixes**
| Issue | Title | Difficulty |
|-------|-------|------------|
| #257 | Grid layers with z_index of zero are on top of entities | Quick — change `>=0` to `>0` or similar in draw order |
| #152 | Sparse Grid Layers | Major feature — default values + sub-grid chunk optimization |
**Dependencies:** #257 is standalone. #152 builds on the existing layer system.
---
## Group 5: API Cleanup & Consistency
**1 issue — quick, blocks v1.0**
| Issue | Title | Difficulty |
|-------|-------|------------|
| #304 | Remove camelCase module functions before 1.0 | Quick — delete 4 method entries from mcrfpyMethods[], update tests |
**Dependencies:** Snake_case aliases already added. This is a breaking change gated on the 1.0 release.
---
## Group 6: Multi-Tile Entity Rendering
**5 issues — parent + 4 children, all tier3-future**
Umbrella issue #233 with four sub-issues for different approaches to entities larger than one grid cell.
| Issue | Title | Difficulty |
|-------|-------|------------|
| #233 | Enhance Entity rendering and positioning capabilities (parent) | Meta/tracking |
| #234 | Entity origin offset for oversized sprites | Minor — add pixel offset to entity draw position |
| #235 | Texture display bounds for non-uniform sprite content | Minor — support non-cell-aligned sprite regions |
| #236 | Multi-tile entities using oversized sprites | Minor — render single large sprite across cells |
| #237 | Multi-tile entities using composite sprites | Major — multiple sprite indices per entity |
**Dependencies:** #234 is the simplest starting point. #236 and #237 build on #234/#235.
---
## Group 7: Memory Safety Audit Tail
**9 issues — testing/tooling infrastructure for the #279 audit**
These are the remaining items from the 7DRL 2026 post-mortem. The actual bugs are mostly fixed; these are about preventing regressions and improving the safety toolchain.
| Issue | Title | Difficulty |
|-------|-------|------------|
| #279 | Engine memory safety audit — meta/tracking | Meta — close when #270/#271/#277 done |
| #287 | Regression tests for each bug from #258–#278 | Medium — write targeted test scripts |
| #285 | CI pipeline for debug-test and asan-test | Medium — CI/CD configuration |
| #286 | Re-enable ASan leak detection | Tiny — remove detect_leaks=0 suppression |
| #284 | Valgrind Massif heap profiling target | Tiny — add Makefile target |
| #283 | Atheris fuzzing harness for Python API | Major — significant new infrastructure |
| #282 | Install modern Clang for TSan/fuzzing | Minor — toolchain upgrade |
| #281 | Free-threaded CPython + TSan Makefile targets | Minor — Makefile additions |
| #280 | Instrumented libtcod debug build | Minor — rebuild libtcod with sanitizers |
**Dependencies:** #286 depends on #266/#275 (both closed). #287 depends on the actual bugs being fixed. #283 depends on #282.
---
## Group 8: Grid Data Model Enhancements
**3 issues — foundation work for game data**
| Issue | Title | Difficulty |
|-------|-------|------------|
| #293 | DiscreteMap serialization via bytes | Minor — add bytes() and from_bytes() to DiscreteMap |
| #294 | Entity.gridstate as DiscreteMap reference | Minor — refactor internal representation |
| #149 | Reduce the size of UIGrid.cpp | Refactoring — break 1400+ line file into logical units |
**Dependencies:** #294 depends on #293. #149 is independent refactoring.
---
## Group 9: Performance Optimization
**4 issues — significant effort, needs benchmarks first**
| Issue | Title | Difficulty |
|-------|-------|------------|
| #255 | Tracking down performance improvement opportunities | Investigation — profiling session |
| #117 | Memory Pool for Entities | Major — custom allocator |
| #145 | TexturePool with power-of-2 RenderTexture reuse | Major — deferred from #144 |
| #124 | Grid Point Animation | Major — per-tile animation system, needs design |
**Dependencies:** #255 should be done first to identify where optimization matters. #145 builds on the dirty-flag system (#144, closed). #124 is a large standalone feature.
---
## Group 10: WASM / Playground Tooling
**3 issues — all tier3-future**
| Issue | Title | Difficulty |
|-------|-------|------------|
| #238 | Emscripten debugging infrastructure (DWARF, source maps) | Minor — build config additions |
| #239 | Automated WASM testing with headless browser | Major — new test infrastructure |
| #240 | Developer troubleshooting docs for WASM deployments | Documentation — write guide |
**Dependencies:** #238 supports #239. #240 is standalone documentation.
---
## Group 11: LLM Agent Testbed
**3 issues — research/demo infrastructure**
| Issue | Title | Difficulty |
|-------|-------|------------|
| #55 | McRogueFace as Agent Simulation Environment | Major — umbrella/vision issue |
| #154 | Grounded Multi-Agent Testbed | Major — research infrastructure |
| #156 | Turn-based LLM Agent Orchestration | Major — orchestration layer |
**Dependencies:** #156 depends on #154. Both depend on #55 conceptually. These also depend on #53 (alternative input methods) and mature API stability.
---
## Group 12: Demo Games & Tutorials
**2 issues — showcase/marketing**
| Issue | Title | Difficulty |
|-------|-------|------------|
| #248 | Crypt of Sokoban Remaster (7DRL prep) | Major — full game remaster |
| #167 | r/roguelikedev Tutorial Series Demo Game | Major — tutorial content + demo game |
**Dependencies:** Both benefit from a stable, well-documented API. #167 specifically needs the API to be settled.
---
## Group 13: Platform & Architecture (Far Future)
**5 issues — large features, mostly deferred**
| Issue | Title | Difficulty |
|-------|-------|------------|
| #70 | Package mcrfpy without embedded interpreter (wheels) | Major — significant build system rework |
| #62 | Multiple Windows | Major — architectural change |
| #67 | Grid Stitching / infinite world prototype | Major — new rendering/data infrastructure |
| #54 | Jupyter Notebook Interface | Major — alternative rendering target |
| #53 | Alternative Input Methods | Major — depends on #220 |
---
## Group 14: Concurrency
**1 issue — deferred**
| Issue | Title | Difficulty |
|-------|-------|------------|
| #220 | Secondary Concurrency Model: Subinterpreter Support | Major — Python 3.12+ subinterpreters |
**Dependencies:** Depends on free-threaded CPython work (#281).
---
## Summary by Priority
| Priority | Groups | Issue Count | Session Estimate |
|----------|--------|-------------|-----------------|
| **Do now** | G1 (dirty flags), G2 (dangling ptrs) | 7 | 1 session |
| **Do soon** | G3 (animation), G4 (grid fixes), G5 (API cleanup) | 5 | 1 session |
| **Foundation** | G7 (safety tests), G8 (grid data) | 12 | 2-3 sessions |
| **When ready** | G6 (multi-tile), G9 (perf), G10 (WASM) | 12 | 3-4 sessions |
| **Future** | G11 (LLM), G12 (demos), G13 (platform), G14 (concurrency) | 10 | unbounded |
## Recommended First Session
**Groups 1 + 2: Dirty flags + dangling pointers (7 issues)**
Rationale:
- All are correctness/safety bugs, not features — fixes don't need design decisions
- Dirty flag fixes (#288-#291) share the same mechanical pattern: add missing `markDirty()` or `markCompositeDirty()` calls
- Dangling pointer fixes (#270, #271, #277) share the same pattern: convert `UIGrid*` to `weak_ptr<UIGrid>` or add invalidation on grid destruction
- Closing these also effectively closes the meta issue #279
- High confidence of completing all 7 in one session
- Clears the way for performance work (Group 9) which depends on correct caching
---
## Triage Completion Status (2026-04-20)
### Groups 1–5: COMPLETE (overnight sessions, prior to this entry)
All issues fixed or labeled. See commit history for details.
### Groups 6–14: COMPLETE (2026-04-20)
| Group | Issues | Status |
|-------|--------|--------|
| G6 Multi-tile entities | #233–#237 | All **closed** |
| G7 Memory safety tooling | #279–#287 | All **closed** except #282 (open, labeled) |
| G8 Grid data model | #149, #293, #294 | All **closed** |
| G9 Performance | #117, #124, #145, #255 | Open, all labeled |
| G10 WASM/Playground | #238, #239, #240 | #238/#240 closed; #239 open, labeled |
| G11 LLM agent testbed | #55, #154, #156 | Open, all labeled |
| G12 Demo games | #167, #248 | Open, all labeled |
| G13 Platform/architecture | #53, #54, #62, #67, #70 | Open, all labeled |
| G14 Concurrency | #220 | Open, labeled |
**Label pass completed:** All open issues in groups 6–14 now have `system:*`, `priority:tier*`, and type labels applied.
### Post-triage new issues (#312–#316, created 2026-04-19)
These appeared after the triage document was written and have been labeled in the same session:
| Issue | Title | Labels Applied |
|-------|-------|----------------|
| #312 | Extend fuzz coverage to remaining API surface | Minor Feature, system:performance, priority:tier2-foundation |
| #313 | Migrate UIEntity::grid to shared_ptr\<GridData\> | Refactoring & Cleanup, system:grid, system:python-binding, priority:tier1-active |
| #314 | API audit documentation follow-through | Documentation, system:documentation, priority:tier1-active |
| #316 | Sparse perspective writeback in updateVisibility | Minor Feature, system:performance, system:grid, priority:tier2-foundation, workflow:needs-benchmark |

162
docs/WASM_TROUBLESHOOTING.md
View file

@ -1,162 +0,0 @@
# WASM / Emscripten Troubleshooting Guide
Practical solutions for common issues when building, testing, and deploying McRogueFace as a WebAssembly application.
## Build Issues
### "emcmake not found"
The Emscripten SDK must be activated in your current shell before building:
```bash
source ~/emsdk/emsdk_env.sh
make wasm
```
This sets `PATH`, `EMSDK`, and other environment variables. You need to re-run it for each new terminal session.
### Build fails during CMake configure
If CMake fails during the Emscripten configure step, delete the build directory and re-configure:
```bash
rm -rf build-emscripten
make wasm
```
The Makefile targets skip CMake if a `Makefile` already exists in the build directory. Stale CMake caches from a prior SDK version or changed options cause configure errors.
### "memory access out of bounds" at startup
Usually caused by insufficient stack or memory. The build defaults to a 2 MB stack (`-sSTACK_SIZE=2097152`) and growable heap (`-sALLOW_MEMORY_GROWTH=1`). If you hit stack limits with deep recursion (e.g. during Python import), increase the stack size in `CMakeLists.txt`:
```cmake
-sSTACK_SIZE=4194304 # 4 MB
```
### Link errors about undefined symbols
The Emscripten build uses `-sERROR_ON_UNDEFINED_SYMBOLS=0` because some libc/POSIX symbols are stubbed. If you add new C++ code that calls missing POSIX APIs, you will get a runtime error rather than a link error. Check the browser console for `Aborted(Assertion failed: missing function: ...)`.
## Runtime Issues
### Python import errors
The WASM build bundles a filtered Python stdlib at build time via `--preload-file`. If a Python module is missing at runtime:
1. Check `wasm_stdlib/lib/` — this is the preloaded stdlib tree
2. If the module should be included, add it to `tools/stdlib_modules.yaml` under the appropriate category
3. Rebuild: `rm -rf build-emscripten && make wasm`
Some modules (like `socket`, `ssl`, `multiprocessing`) are intentionally excluded because they require OS features unavailable in the browser.
### "Synchronous XMLHttpRequest on the main thread is deprecated"
This warning appears when Python code triggers synchronous file I/O during module import. It's harmless but can cause slight UI freezes. The engine preloads all files into Emscripten's virtual filesystem before Python starts, so actual network requests don't happen.
### IndexedDB / persistent storage errors
The build uses `-lidbfs.js` for persistent storage (save games, user preferences). Common issues:
- **"mkdir failed" on first load**: The engine calls `FS.mkdir('/idbfs')` during initialization. If the path already exists from a prior version, this fails silently. The `emscripten_pre.js` file patches this.
- **Data not persisting**: Call `FS.syncfs(false, callback)` from JavaScript to flush changes to IndexedDB. The C++ side exposes `sync_storage()` via `Module.ccall`.
- **Private browsing**: IndexedDB is unavailable in some private/incognito modes. The engine falls back gracefully but data won't persist.
### Black screen / no rendering
Check the browser's developer console (F12) for errors. Common causes:
- **WebGL 2 not supported**: The build requires WebGL 2 (`-sMIN_WEBGL_VERSION=2`). Very old browsers or software renderers may not support it.
- **Canvas size is zero**: If the HTML container has no explicit size, the canvas may render at 0x0. The custom `shell.html` handles this, but custom embedding needs to set canvas dimensions.
- **Exception during init**: A Python error during `game.py` execution will abort rendering. Check console for Python tracebacks.
### Audio not working
Audio is stubbed in the WASM build. `SoundBuffer`, `Sound`, and `Music` objects exist but do nothing. This is documented in the Web Build Constraints table in CLAUDE.md.
## Debugging
### Enable debug builds
Use the debug WASM targets for full DWARF symbols and source maps:
```bash
make wasm-debug # Full game with debug info
make playground-debug # REPL with debug info
```
These produce larger binaries but enable:
- **Source-level debugging** in Chrome DevTools (via DWARF and source maps)
- **Readable stack traces** (via `--emit-symbol-map`)
### Reading WASM stack traces
Production WASM stack traces show mangled names like `$_ZN7UIFrame6renderEv`. To demangle:
1. Use the debug build which emits a `.symbols` file
2. Or pipe through `c++filt`: `echo '_ZN7UIFrame6renderEv' | c++filt`
3. Or use Chrome's DWARF extension for inline source mapping
### Browser developer tools
- **Chrome**: DevTools > Sources > shows C++ source files with DWARF debug builds
- **Firefox**: Debugger > limited DWARF support, better with source maps
- **Console**: All `printf`/`std::cout` output goes to the browser console
- **Network**: Check that `.data` (preloaded files) and `.wasm` loaded successfully
- **Memory**: Use Chrome's Memory tab to profile WASM heap usage
### Assertions
The build enables `-sASSERTIONS=2` and `-sSTACK_OVERFLOW_CHECK=2` by default (both debug and release). These catch:
- Null pointer dereferences in WASM memory
- Stack overflow before it corrupts the heap
- Invalid Emscripten API usage
## Deployment
### File sizes
Typical build sizes:
| Build | .wasm | .data | .js | Total |
|-------|-------|-------|-----|-------|
| Release | ~15 MB | ~25 MB | ~200 KB | ~40 MB |
| Debug | ~40 MB | ~25 MB | ~300 KB | ~65 MB |
The `.data` file contains the Python stdlib and game assets. Use the "light" stdlib preset to reduce it.
### Serving requirements
WASM files require specific HTTP headers:
- `Content-Type: application/wasm` for `.wasm` files
- CORS headers if serving from a CDN
The `make serve` targets use Python's `http.server` which handles MIME types correctly for local development.
### Embedding in custom pages
The build produces an HTML file from `shell.html` (or `shell_game.html`). To embed in your own page, you need:
1. The `.js`, `.wasm`, and `.data` files from the build directory
2. A canvas element with `id="canvas"`
3. Load the `.js` file, which bootstraps everything:
```html
<canvas id="canvas" width="1024" height="768"></canvas>
<script src="mcrogueface.js"></script>
```
### Game shell vs playground shell
- `make wasm` / `make wasm-game`: Uses `shell.html` or `shell_game.html` — includes REPL widget or fullscreen canvas
- `make playground`: Uses `shell.html` with REPL chrome — intended for interactive testing
- Set `MCRF_GAME_SHELL=ON` in CMake for fullscreen-only (no REPL)
## Known Limitations
1. **No dynamic module loading**: All Python modules must be preloaded at build time
2. **No threading**: JavaScript is single-threaded; Python's `threading` module is non-functional
3. **No filesystem writes to disk**: Writes go to an in-memory filesystem (optionally synced to IndexedDB)
4. **No audio**: Sound API is fully stubbed
5. **No ImGui console**: The debug overlay is desktop-only
6. **Input differences**: Some keyboard shortcuts are intercepted by the browser (Ctrl+W, F5, etc.)

994
docs/api-audit-2026-04.md
View file

@ -1,994 +0,0 @@
# McRogueFace Python API Consistency Audit
**Date**: 2026-04-09
**Version**: 0.2.6-prerelease
**Purpose**: Catalog the full public API surface, identify inconsistencies and issues before 1.0 API freeze.
---
## Table of Contents
1. [Executive Summary](#executive-summary)
2. [Module-Level API](#module-level-api)
3. [Core Value Types](#core-value-types)
4. [UI Drawable Types](#ui-drawable-types)
5. [Grid System](#grid-system)
6. [Entity System](#entity-system)
7. [Collections](#collections)
8. [Audio Types](#audio-types)
9. [Procedural Generation](#procedural-generation)
10. [Pathfinding](#pathfinding)
11. [Shader System](#shader-system)
12. [Tiled/LDtk Import](#tiledldtk-import)
13. [3D/Experimental Types](#3dexperimental-types)
14. [Enums](#enums)
15. [Findings: Naming Inconsistencies](#findings-naming-inconsistencies)
16. [Findings: Missing Functionality](#findings-missing-functionality)
17. [Findings: Deprecations to Resolve](#findings-deprecations-to-resolve)
18. [Findings: Documentation Gaps](#findings-documentation-gaps)
19. [Recommendations](#recommendations)
---
## Executive Summary
The McRogueFace Python API exposes **46 exported types**, **14 internal types**, **10 enums**, **13 module-level functions**, **7 module-level properties**, and **5 singleton instances** through the `mcrfpy` module.
Overall, the API is remarkably consistent. Properties and methods use snake_case throughout the type system. The major inconsistencies are concentrated in a few areas:
1. **4 module-level functions use camelCase** (`setScale`, `findAll`, `getMetrics`, `setDevConsole`)
2. **Terse/placeholder docstrings** on 5 core types (Vector, Font, Texture, GridPoint, GridPointState)
3. **Deprecated property aliases** still exposed (`sprite_number`)
4. **Color property naming split**: some types use `fill_color`/`outline_color`, others use `color`
5. **Redundant position aliases** on Entity (`grid_pos` vs `cell_pos` for the same data)
---
## Module-Level API
### Functions (`mcrfpy.*`)
| Function | Signature | Notes |
|----------|-----------|-------|
| `step` | `(dt: float = None) -> float` | Advance simulation (headless mode) |
| `exit` | `() -> None` | Shutdown engine |
| `find` | `(name: str, scene: str = None) -> Drawable \| None` | Find UI element by name |
| `lock` | `() -> _LockContext` | Thread-safe UI update context manager |
| `bresenham` | `(start, end, *, include_start=True, include_end=True) -> list[tuple]` | Line algorithm |
| `start_benchmark` | `() -> None` | Begin benchmark capture |
| `end_benchmark` | `() -> str` | End benchmark, return filename |
| `log_benchmark` | `(message: str) -> None` | Add benchmark annotation |
| `_sync_storage` | `() -> None` | WASM persistent storage flush |
| **`setScale`** | `(multiplier: float) -> None` | **CAMELCASE - deprecated** |
| **`findAll`** | `(pattern: str, scene: str = None) -> list` | **CAMELCASE** |
| **`getMetrics`** | `() -> dict` | **CAMELCASE** |
| **`setDevConsole`** | `(enabled: bool) -> None` | **CAMELCASE** |
### Properties (`mcrfpy.*`)
| Property | Type | Writable | Notes |
|----------|------|----------|-------|
| `current_scene` | `Scene \| None` | Yes | Active scene |
| `scenes` | `dict[str, Scene]` | No | All registered scenes |
| `timers` | `list[Timer]` | No | Active timers |
| `animations` | `list[Animation]` | No | Active animations |
| `default_transition` | `Transition` | Yes | Scene transition effect |
| `default_transition_duration` | `float` | Yes | Transition duration |
| `save_dir` | `str` | No | Platform-specific save path |
### Singletons
| Name | Type | Notes |
|------|------|-------|
| `keyboard` | `Keyboard` | Modifier key state |
| `mouse` | `Mouse` | Position and button state |
| `window` | `Window` | Window properties |
| `default_font` | `Font` | JetBrains Mono |
| `default_texture` | `Texture` | Kenney Tiny Dungeon (16x16) |
### Constants
| Name | Type | Value |
|------|------|-------|
| `__version__` | `str` | Build version string |
| `default_fov` | `FOV` | `FOV.BASIC` |
### Submodules
| Name | Contents |
|------|----------|
| `automation` | Screenshot, click simulation, testing utilities |
---
## Core Value Types
### `Color`
```
Color(r: int = 0, g: int = 0, b: int = 0, a: int = 255)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `r`, `g`, `b`, `a` | int (0-255) | R/W |
| Methods | Signature |
|---------|-----------|
| `from_hex` | `(cls, hex_string: str) -> Color` (classmethod) |
| `to_hex` | `() -> str` |
| `lerp` | `(other: Color, t: float) -> Color` |
Protocols: `__repr__`, `__hash__`
### `Vector`
```
Vector(x: float = 0, y: float = 0)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `x`, `y` | float | R/W |
| `int` | tuple[int, int] | R |
| Methods | Signature |
|---------|-----------|
| `magnitude` | `() -> float` |
| `magnitude_squared` | `() -> float` |
| `normalize` | `() -> Vector` |
| `dot` | `(other: Vector) -> float` |
| `distance_to` | `(other: Vector) -> float` |
| `angle` | `() -> float` |
| `copy` | `() -> Vector` |
| `floor` | `() -> Vector` |
Protocols: `__repr__`, `__hash__`, `__eq__`/`__ne__`, arithmetic (`+`, `-`, `*`, `/`, `-x`, `abs`), sequence (`len`, `[0]`/`[1]`)
### `Font`
```
Font(filename: str)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `family` | str | R |
| `source` | str | R |
Methods: None
Protocols: `__repr__`
### `Texture`
```
Texture(filename: str, sprite_width: int, sprite_height: int)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `sprite_width`, `sprite_height` | int | R |
| `sheet_width`, `sheet_height` | int | R |
| `sprite_count` | int | R |
| `source` | str | R |
| Methods | Signature |
|---------|-----------|
| `from_bytes` | `(cls, data, w, h, sprite_w, sprite_h, name=...) -> Texture` (classmethod) |
| `composite` | `(cls, layers, sprite_w, sprite_h, name=...) -> Texture` (classmethod) |
| `hsl_shift` | `(hue_shift, sat_shift=0, lit_shift=0) -> Texture` |
Protocols: `__repr__`, `__hash__`
---
## UI Drawable Types
### Base: `Drawable` (abstract)
Cannot be instantiated directly.
| Properties | Type | R/W | Notes |
|-----------|------|-----|-------|
| `on_click` | callable | R/W | `(pos, button, action)` |
| `z_index` | int | R/W | Render order |
| `visible` | bool | R/W | |
| `opacity` | float | R/W | 0.0-1.0 |
| `name` | str | R/W | |
| `pos` | Vector | R/W | |
| `parent` | Drawable | R | |
| `align` | Alignment | R/W | |
| `margin`, `horiz_margin`, `vert_margin` | float | R/W | |
| `shader` | Shader | R/W | |
| `uniforms` | UniformCollection | R | |
| `rotation` | float | R/W | |
| `origin` | Vector | R/W | |
| Methods | Signature |
|---------|-----------|
| `move` | `(dx, dy)` or `(delta)` |
| `resize` | `(w, h)` or `(size)` |
| `animate` | `(property, target, duration, easing, ...)` |
### `Frame`
```
Frame(pos=None, size=None, **kwargs)
```
Additional properties beyond Drawable:
| Properties | Type | R/W |
|-----------|------|-----|
| `x`, `y`, `w`, `h` | float | R/W |
| `fill_color` | Color | R/W |
| `outline_color` | Color | R/W |
| `outline` | float | R/W |
| `children` | UICollection | R |
| `clip_children` | bool | R/W |
| `cache_subtree` | bool | R/W |
| `grid_pos`, `grid_size` | Vector | R/W |
### `Caption`
```
Caption(pos=None, font=None, text='', **kwargs)
```
Additional properties beyond Drawable:
| Properties | Type | R/W |
|-----------|------|-----|
| `x`, `y` | float | R/W |
| `w`, `h` | float | R (computed) |
| `size` | Vector | R (computed) |
| `text` | str | R/W |
| `font_size` | float | R/W |
| `fill_color` | Color | R/W |
| `outline_color` | Color | R/W |
| `outline` | float | R/W |
### `Sprite`
```
Sprite(pos=None, texture=None, sprite_index=0, **kwargs)
```
Additional properties beyond Drawable:
| Properties | Type | R/W | Notes |
|-----------|------|-----|-------|
| `x`, `y` | float | R/W | |
| `w`, `h` | float | R (computed) | |
| `scale` | float | R/W | Uniform scale |
| `scale_x`, `scale_y` | float | R/W | Per-axis scale |
| `sprite_index` | int | R/W | |
| `sprite_number` | int | R/W | **DEPRECATED alias** |
| `texture` | Texture | R/W | |
### `Line`
```
Line(start=None, end=None, thickness=1.0, color=None, **kwargs)
```
| Properties | Type | R/W | Notes |
|-----------|------|-----|-------|
| `start` | Vector | R/W | |
| `end` | Vector | R/W | |
| `color` | Color | R/W | **Not `fill_color`** |
| `thickness` | float | R/W | |
### `Circle`
```
Circle(radius=0, center=None, fill_color=None, outline_color=None, outline=0, **kwargs)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `radius` | float | R/W |
| `center` | Vector | R/W |
| `fill_color` | Color | R/W |
| `outline_color` | Color | R/W |
| `outline` | float | R/W |
### `Arc`
```
Arc(center=None, radius=0, start_angle=0, end_angle=90, color=None, thickness=1, **kwargs)
```
| Properties | Type | R/W | Notes |
|-----------|------|-----|-------|
| `center` | Vector | R/W | |
| `radius` | float | R/W | |
| `start_angle`, `end_angle` | float | R/W | Degrees |
| `color` | Color | R/W | **Not `fill_color`** |
| `thickness` | float | R/W | |
---
## Grid System
### `Grid` (also available as `GridView`)
```
Grid(grid_size=None, pos=None, size=None, texture=None, **kwargs)
```
| Properties | Type | R/W | Notes |
|-----------|------|-----|-------|
| `grid_size`, `grid_w`, `grid_h` | tuple/int | R | |
| `x`, `y`, `w`, `h` | float | R/W | |
| `pos`, `position` | Vector | R/W | `position` is redundant alias |
| `center` | Vector | R/W | Camera center (pixels) |
| `center_x`, `center_y` | float | R/W | |
| `zoom` | float | R/W | |
| `camera_rotation` | float | R/W | |
| `fill_color` | Color | R/W | |
| `texture` | Texture | R | |
| `entities` | EntityCollection | R | |
| `children` | UICollection | R | |
| `layers` | tuple | R | |
| `perspective`, `perspective_enabled` | various | R/W | |
| `fov`, `fov_radius` | various | R/W | |
| `on_cell_enter`, `on_cell_exit`, `on_cell_click` | callable | R/W | |
| `hovered_cell` | tuple | R | |
| `grid_data` | _GridData | R/W | Internal grid reference |
| Methods | Signature |
|---------|-----------|
| `at` | `(x, y)` or `(pos)` -> GridPoint |
| `compute_fov` | `(pos, radius, light_walls, algorithm)` |
| `is_in_fov` | `(pos) -> bool` |
| `find_path` | `(start, end, diagonal_cost, collide) -> AStarPath` |
| `get_dijkstra_map` | `(root, diagonal_cost, collide) -> DijkstraMap` |
| `clear_dijkstra_maps` | `()` |
| `add_layer` | `(layer)` |
| `remove_layer` | `(name_or_layer)` |
| `layer` | `(name) -> ColorLayer \| TileLayer` |
| `entities_in_radius` | `(pos, radius) -> list` |
| `center_camera` | `(pos)` -- tile coordinates |
| `apply_threshold` | `(source, range, walkable, transparent)` |
| `apply_ranges` | `(source, ranges)` |
| `step` | `(n, turn_order)` -- turn management |
### `GridPoint` (internal, returned by `Grid.at()`)
| Properties | Type | R/W |
|-----------|------|-----|
| `walkable` | bool | R/W |
| `transparent` | bool | R/W |
| `entities` | list | R |
| `grid_pos` | tuple | R |
Dynamic attributes: named layer data via `__getattr__`/`__setattr__`
### `GridPointState` (internal, returned by entity gridstate)
| Properties | Type | R/W |
|-----------|------|-----|
| `visible` | bool | R/W |
| `discovered` | bool | R/W |
| `point` | GridPoint | R |
### `ColorLayer`
```
ColorLayer(z_index=-1, name=None, grid_size=None)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `z_index` | int | R/W |
| `visible` | bool | R/W |
| `grid_size` | tuple | R |
| `name` | str | R |
| `grid` | Grid | R/W |
| Methods | Signature |
|---------|-----------|
| `at` | `(x, y)` or `(pos) -> Color` |
| `set` | `(pos, color)` |
| `fill` | `(color)` |
| `fill_rect` | `(pos, size, color)` |
| `draw_fov` | `(source, radius, fov, visible, discovered, unknown)` |
| `apply_perspective` | `(entity, visible, discovered, unknown)` |
| `update_perspective` | `()` |
| `clear_perspective` | `()` |
| `apply_threshold` | `(source, range, color)` |
| `apply_gradient` | `(source, range, color_low, color_high)` |
| `apply_ranges` | `(source, ranges)` |
### `TileLayer`
```
TileLayer(z_index=-1, name=None, texture=None, grid_size=None)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `z_index` | int | R/W |
| `visible` | bool | R/W |
| `texture` | Texture | R/W |
| `grid_size` | tuple | R |
| `name` | str | R |
| `grid` | Grid | R/W |
| Methods | Signature |
|---------|-----------|
| `at` | `(x, y)` or `(pos) -> int` |
| `set` | `(pos, index)` |
| `fill` | `(index)` |
| `fill_rect` | `(pos, size, index)` |
| `apply_threshold` | `(source, range, tile)` |
| `apply_ranges` | `(source, ranges)` |
---
## Entity System
### `Entity`
```
Entity(grid_pos=None, texture=None, sprite_index=0, **kwargs)
```
| Properties | Type | R/W | Notes |
|-----------|------|-----|-------|
| `pos`, `x`, `y` | Vector/float | R/W | Pixel position |
| `cell_pos`, `cell_x`, `cell_y` | Vector/int | R/W | Integer cell coords |
| `grid_pos`, `grid_x`, `grid_y` | Vector/int | R/W | **Same as cell_pos** |
| `draw_pos` | Vector | R/W | Fractional tile position |
| `sprite_index` | int | R/W | |
| `sprite_number` | int | R/W | **DEPRECATED alias** |
| `sprite_offset`, `sprite_offset_x`, `sprite_offset_y` | Vector/float | R/W | |
| `grid` | Grid | R/W | |
| `gridstate` | GridPointState | R | |
| `labels` | frozenset | R/W | |
| `step` | callable | R/W | Turn callback |
| `default_behavior` | Behavior | R/W | |
| `behavior_type` | Behavior | R | |
| `turn_order` | int | R/W | |
| `move_speed` | float | R/W | |
| `target_label` | str | R/W | |
| `sight_radius` | int | R/W | |
| `visible`, `opacity`, `name` | various | R/W | |
| `shader`, `uniforms` | various | R/W | |
| Methods | Signature |
|---------|-----------|
| `at` | `(x, y)` or `(pos) -> GridPoint` |
| `index` | `() -> int` |
| `die` | `()` |
| `path_to` | `(x, y)` or `(target) -> AStarPath` |
| `find_path` | `(target, diagonal_cost, collide) -> AStarPath` |
| `update_visibility` | `()` |
| `visible_entities` | `(fov, radius) -> list` |
| `animate` | `(property, target, duration, easing, ...)` |
---
## Collections
### `UICollection` (internal, returned by `Frame.children` / `Scene.children`)
| Methods | Signature |
|---------|-----------|
| `append` | `(element)` |
| `extend` | `(iterable)` |
| `insert` | `(index, element)` |
| `remove` | `(element)` |
| `pop` | `([index]) -> Drawable` |
| `index` | `(element) -> int` |
| `count` | `(element) -> int` |
| `find` | `(name, recursive=False) -> Drawable \| None` |
Protocols: `len`, `[]`, slicing, iteration
### `EntityCollection` (internal, returned by `Grid.entities`)
Same methods as UICollection. Protocols: `len`, `[]`, slicing, iteration.
---
## Audio Types
### `Sound`
```
Sound(source: str | SoundBuffer)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `volume` | float (0-100) | R/W |
| `loop` | bool | R/W |
| `playing` | bool | R |
| `duration` | float | R |
| `source` | str | R |
| `pitch` | float | R/W |
| `buffer` | SoundBuffer | R |
| Methods | Signature |
|---------|-----------|
| `play` | `()` |
| `pause` | `()` |
| `stop` | `()` |
| `play_varied` | `(pitch_range=0.1, volume_range=3.0)` |
### `SoundBuffer`
```
SoundBuffer(filename: str)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `duration` | float | R |
| `sample_count` | int | R |
| `sample_rate` | int | R |
| `channels` | int | R |
| `sfxr_params` | dict | R |
| Methods | Signature | Notes |
|---------|-----------|-------|
| `from_samples` | `(cls, data, channels, sample_rate) -> SoundBuffer` | classmethod |
| `tone` | `(cls, frequency, duration, waveform='sine', ...) -> SoundBuffer` | classmethod |
| `sfxr` | `(cls, preset, seed=None) -> SoundBuffer` | classmethod |
| `concat` | `(cls, buffers) -> SoundBuffer` | classmethod |
| `mix` | `(cls, buffers) -> SoundBuffer` | classmethod |
| `pitch_shift` | `(semitones) -> SoundBuffer` | returns new |
| `low_pass` | `(cutoff) -> SoundBuffer` | returns new |
| `high_pass` | `(cutoff) -> SoundBuffer` | returns new |
| `echo` | `(delay, decay) -> SoundBuffer` | returns new |
| `reverb` | `(room_size) -> SoundBuffer` | returns new |
| `distortion` | `(gain) -> SoundBuffer` | returns new |
| `bit_crush` | `(bits) -> SoundBuffer` | returns new |
| `gain` | `(amount) -> SoundBuffer` | returns new |
| `normalize` | `() -> SoundBuffer` | returns new |
| `reverse` | `() -> SoundBuffer` | returns new |
| `slice` | `(start, end) -> SoundBuffer` | returns new |
| `sfxr_mutate` | `(amount) -> SoundBuffer` | returns new |
### `Music`
```
Music(filename: str)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `volume` | float (0-100) | R/W |
| `loop` | bool | R/W |
| `playing` | bool | R |
| `duration` | float | R |
| `position` | float | R/W |
| `source` | str | R |
| Methods | Signature |
|---------|-----------|
| `play` | `()` |
| `pause` | `()` |
| `stop` | `()` |
---
## Procedural Generation
### `HeightMap`
```
HeightMap(size: tuple[int, int], fill: float = 0.0)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `size` | tuple | R |
46 methods covering: fill/clear, get/set (via `[]`), math operations, noise, erosion, BSP integration, kernel operations, binary operations.
Protocols: `[x, y]` subscript (get/set)
### `DiscreteMap`
```
DiscreteMap(size: tuple[int, int], fill: int = 0, enum: type[IntEnum] = None)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `size` | tuple | R |
| `enum_type` | type | R/W |
22 methods covering: fill/clear, get/set (via `[]`), math, bitwise, statistics, conversion.
Protocols: `[x, y]` subscript (get/set)
### `BSP`
```
BSP(pos: tuple[int, int], size: tuple[int, int])
```
| Properties | Type | R/W |
|-----------|------|-----|
| `bounds`, `pos`, `size` | tuple | R |
| `root` | BSPNode | R |
| `adjacency` | BSPAdjacency | R |
| Methods | Signature |
|---------|-----------|
| `split_once` | `(...)` |
| `split_recursive` | `(...)` |
| `clear` | `()` |
| `leaves` | `() -> list[BSPNode]` |
| `traverse` | `(order) -> BSPIter` |
| `find` | `(pos) -> BSPNode` |
| `get_leaf` | `(index) -> BSPNode` |
| `to_heightmap` | `() -> HeightMap` |
### `NoiseSource`
```
NoiseSource(dimensions=2, algorithm='simplex', hurst=0.5, lacunarity=2.0, seed=None)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `dimensions`, `algorithm`, `hurst`, `lacunarity`, `seed` | various | R |
| Methods | Signature |
|---------|-----------|
| `get` | `(pos) -> float` |
| `fbm` | `(pos, octaves=4) -> float` |
| `turbulence` | `(pos, octaves=4) -> float` |
| `sample` | `(size, world_origin, world_size, mode, octaves) -> HeightMap` |
---
## Pathfinding
### `AStarPath`
| Properties | Type | R/W |
|-----------|------|-----|
| `origin` | tuple | R |
| `destination` | tuple | R |
| `remaining` | int | R |
| Methods | Signature |
|---------|-----------|
| `walk` | `() -> Vector` |
| `peek` | `() -> Vector` |
Protocols: `len`, `bool`, iteration
### `DijkstraMap`
| Properties | Type | R/W |
|-----------|------|-----|
| `root` | tuple | R |
| Methods | Signature |
|---------|-----------|
| `distance` | `(pos) -> float \| None` |
| `path_from` | `(pos) -> AStarPath` |
| `step_from` | `(pos) -> Vector \| None` |
| `to_heightmap` | `(size=None, unreachable=-1.0) -> HeightMap` |
---
## Shader System
### `Shader`
```
Shader(fragment_source: str, dynamic: bool = False)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `dynamic` | bool | R/W |
| `source` | str | R |
| `is_valid` | bool | R |
| Methods | Signature |
|---------|-----------|
| `set_uniform` | `(name: str, value: float \| tuple)` |
### `PropertyBinding`
```
PropertyBinding(target: Drawable, property: str)
```
| Properties | Type | R/W |
|-----------|------|-----|
| `target` | Drawable | R |
| `property` | str | R |
| `value` | float | R |
| `is_valid` | bool | R |
### `CallableBinding`
```
CallableBinding(callable: Callable[[], float])
```
| Properties | Type | R/W |
|-----------|------|-----|
| `callable` | callable | R |
| `value` | float | R |
| `is_valid` | bool | R |
### `UniformCollection` (internal, returned by `drawable.uniforms`)
Dict-like container. Supports `[]`, `del`, `in`, `keys()`, `values()`, `items()`, `clear()`.
---
## Tiled/LDtk Import
### `TileSetFile`
```
TileSetFile(path: str)
```
Properties (all R): `name`, `tile_width`, `tile_height`, `tile_count`, `columns`, `margin`, `spacing`, `image_source`, `properties`, `wang_sets`
Methods: `to_texture()`, `tile_info(id)`, `wang_set(name)`
### `TileMapFile`
```
TileMapFile(path: str)
```
Properties (all R): `width`, `height`, `tile_width`, `tile_height`, `orientation`, `properties`, `tileset_count`, `tile_layer_names`, `object_layer_names`
Methods: `tileset(index)`, `tile_layer_data(name)`, `resolve_gid(gid)`, `object_layer(name)`, `apply_to_tile_layer(layer, name)`
### `WangSet` (factory-created from TileSetFile)
Properties (all R): `name`, `type`, `color_count`, `colors`
Methods: `terrain_enum()`, `resolve(discrete_map)`, `apply(discrete_map, tile_layer)`
### `LdtkProject`
```
LdtkProject(path: str)
```
Properties (all R): `version`, `tileset_names`, `ruleset_names`, `level_names`, `enums`
Methods: `tileset(name)`, `ruleset(name)`, `level(name)`
### `AutoRuleSet` (factory-created from LdtkProject)
Properties (all R): `name`, `grid_size`, `value_count`, `values`, `rule_count`, `group_count`
Methods: `terrain_enum()`, `resolve(discrete_map)`, `apply(discrete_map, tile_layer)`
---
## 3D/Experimental Types
> These are exempt from the 1.0 API freeze per ROADMAP.md.
Viewport3D, Entity3D, EntityCollection3D, Model3D, Billboard, VoxelGrid, VoxelRegion, VoxelPoint, Camera3D (via Viewport3D properties).
---
## Enums
| Enum | Values | Notes |
|------|--------|-------|
| `Key` | 42+ keyboard keys | Legacy string comparison (`Key.ESCAPE == "Escape"`) |
| `InputState` | `PRESSED`, `RELEASED` | Legacy: `"start"`, `"end"` |
| `MouseButton` | `LEFT`, `RIGHT`, `MIDDLE`, `X1`, `X2` | Legacy: `"left"`, `"right"`, `"middle"` |
| `Easing` | 32 easing functions | Linear, Quad, Cubic, etc. |
| `Transition` | Scene transition effects | |
| `Traversal` | BSP traversal orders | |
| `Alignment` | 9 positions + NONE | TOP_LEFT through BOTTOM_RIGHT |
| `Behavior` | 11 entity behaviors | For `grid.step()` turn system |
| `Trigger` | 3 trigger types | Entity step callbacks |
| `FOV` | FOV algorithms | Maps to libtcod |
---
## Findings: Naming Inconsistencies
### F1: Module-level camelCase functions (CRITICAL)
Four module-level functions use camelCase while everything else uses snake_case:
| Current | Should Be | Status |
|---------|-----------|--------|
| `setScale` | `set_scale` | Deprecated anyway (use `Window.resolution`) |
| `findAll` | `find_all` | Active, needs alias |
| `getMetrics` | `get_metrics` | Active, needs alias |
| `setDevConsole` | `set_dev_console` | Active, needs alias |
**Resolution**: Add snake_case aliases. Keep camelCase temporarily for backward compatibility. Remove camelCase in 1.0.
### F2: Color property naming split
Filled shapes (Frame, Caption, Circle) use `fill_color`/`outline_color`. Stroke-only shapes (Line, Arc) use `color`. This is actually semantically correct -- Line and Arc don't have a "fill" concept. **No change needed**, but worth documenting.
### F3: Redundant Entity position aliases
Entity exposes the same cell position data under two names:
- `grid_pos`, `grid_x`, `grid_y`
- `cell_pos`, `cell_x`, `cell_y`
Both exist because `grid_pos` is the constructor parameter name and `cell_pos` is more descriptive. **Recommendation**: Keep both but document `grid_pos` as the canonical name (matches constructor).
### F4: Grid `position` alias
`Grid.position` is a redundant alias for `Grid.pos`. All other types use only `pos`. **Recommendation**: Deprecate `position`, keep `pos`.
### F5: Iterator type naming
- `UICollectionIter` -- has "UI" prefix
- `UIEntityCollectionIter` -- has "UI" prefix
- `EntityCollection3DIter` -- no "UI" prefix
The "UI" prefix is an internal detail leaking into type names. Since these are internal types (not exported), this is cosmetic but worth noting.
---
## Findings: Missing Functionality
### F6: No `__eq__` on Color
`Color` has `__hash__` but no `__eq__`/`__ne__`. Two colors with the same RGBA values may not compare equal. This is a bug.
### F7: No `Music.pitch`
`Sound` has a `pitch` property but `Music` does not, despite SFML supporting it. Minor omission.
### F8: No `Font` methods
`Font` has no methods at all -- not even a way to query available sizes or get text metrics. This limits text layout capabilities.
### F9: GridPoint has no `__init__`
`GridPoint` cannot be constructed from Python (`tp_new = NULL`). This is intentional (it's a view into grid data) but should be clearly documented.
### F10: Animation direct construction deprecated but not marked
The `Animation` class can still be instantiated directly even though `.animate()` on drawables is preferred. No deprecation warning is emitted.
---
## Findings: Deprecations to Resolve
### F11: `sprite_number` on Sprite and Entity
Both types expose `sprite_number` as a deprecated alias for `sprite_index`. This should be removed before 1.0.
### F12: `setScale` module function
Deprecated in favor of `Window.resolution`. Should be removed before 1.0.
### F13: Legacy string enum comparisons
`Key`, `InputState`, `MouseButton` support comparing to legacy string values (e.g., `Key.ESCAPE == "Escape"`, `InputState.PRESSED == "start"`). This backward compatibility layer should be removed before 1.0.
---
## Findings: Documentation Gaps
### F14: Terse docstrings on core types
Several types have placeholder-quality `tp_doc` strings:
| Type | Current tp_doc | Should be |
|------|---------------|-----------|
| `Vector` | `"SFML Vector Object"` | Full constructor docs with args |
| `Font` | `"SFML Font Object"` | Full constructor docs |
| `Texture` | `"SFML Texture Object"` | Full constructor docs |
| `GridPoint` | `"UIGridPoint object"` | Description of purpose and access pattern |
| `GridPointState` | `"UIGridPointState object"` | Description of purpose |
### F15: Missing MCRF_* macro usage
Some types use raw string docstrings for methods instead of MCRF_METHOD macros. This means the documentation pipeline may miss them.
---
## Recommendations
### Before 1.0 (Breaking Changes)
1. **Remove camelCase functions**: `setScale`, `findAll`, `getMetrics`, `setDevConsole`
2. **Remove `sprite_number`** deprecated alias from Sprite and Entity
3. **Remove legacy string enum comparisons** from Key, InputState, MouseButton
4. **Remove `Grid.position`** redundant alias (keep `pos`)
5. **Add `__eq__`/`__ne__` to Color** type
### Immediate (Non-Breaking)
1. **Add snake_case aliases** for the 4 camelCase module functions
2. **Improve docstrings** on Vector, Font, Texture, GridPoint, GridPointState
3. **Document `grid_pos` vs `cell_pos`** -- state that `grid_pos` is canonical
### Future Considerations
1. Add `pitch` to `Music`
2. Add basic text metrics to `Font`
3. Consider deprecation warnings for `Animation()` direct construction
4. Unify iterator type naming (remove "UI" prefix from internal types)
---
## Statistics
| Category | Count |
|----------|-------|
| Exported types | 46 |
| Internal types | 14 |
| Enums | 10 |
| Module functions | 13 |
| Module properties | 7 |
| Singletons | 5 |
| **Total public API surface** | **~93 named items** |
| Naming inconsistencies found | 5 |
| Missing functionality items | 5 |
| Deprecations to resolve | 3 |
| Documentation gaps | 2 |
| **Total findings** | **15** |
---
## #314 Freeze Decisions (2026-06, recorded before generating the API-surface snapshot)
The April audit body above is partly stale. Live introspection of the built module gives the
authoritative counts, and the snapshot test (`tests/unit/api_surface_snapshot_test.py`) is built
from live introspection, not from this document.
**Corrected live counts (2026-06):** 12 enums (audit said 10 — adds `Perspective` #294 and
`Heuristic` #315), 46 exported classes, 12 module functions, 7 singletons/constants (incl. the
`automation` submodule), 1 submodule. `GridPointState` was removed in #294 (its F14 row is moot).
**Per-finding final status:** F1, F4, F6, F10, F11, F13, F14 = RESOLVED (verified in source via
#304–#308). F2 = correct-by-design (docs-only). F5, F9 = cosmetic, unchanged. F7 (`Music.pitch`),
F8 (`Font` methods) = Future, explicitly NOT 1.0 blockers.
**Decisions locked for the freeze (the snapshot golden enshrines these):**
- **F3 (cell-position canonical name):** `grid_pos` is canonical (matches the `grid_pos=`
constructor argument); `cell_pos`/`cell_x`/`cell_y` are documented aliases. Both share the same
getter/setter and remain interchangeable. Docstrings aligned at `src/UIEntity.cpp` getsetters.
- **F12 (`set_scale`):** KEPT in the 1.0 surface as a documented-deprecated function. Removing it
now would itself be a new breaking change; the snapshot locks it in.
- **`mcrfpy.automation`:** PyAutoGUI-compatibility camelCase (`moveRel`/`dragTo`/etc.) is EXEMPT
from the snake_case rule. The snapshot records it in a clearly-labeled section.
- **`entity.texture` (new in #313):** additive read/write property; getter returns the entity's
real texture, `None` only in the degenerate (default_texture-null) case — never re-derefs a null
default_texture. Added to the frozen contract + stubs + docs when #313 lands (golden gains exactly
one line). Known edges, frozen as-is (2026-06-11 adversarial review): the getter mints a NEW
Texture wrapper per access and Texture has no `__eq__`, so `e.texture == e.texture` is False —
compare `.source`/sprite dims instead (same behavior as the pre-existing `Sprite.texture`);
setting does not re-validate `sprite_index` against the new atlas; setter rejects non-Texture
(TypeError), null-data Texture wrappers (ValueError, mirrors `Sprite.texture`), and deletion.
**1.0 freeze scope — class classification** (the snapshot segregates FROZEN vs EXPERIMENTAL):
- **FROZEN (stable 1.0):** core value types (Color, Vector, Font, Texture), UI drawables
(Drawable [root], Frame, Caption, Sprite, Line, Circle, Arc), Grid/GridView/Entity, Scene,
Window, Timer, Keyboard, Mouse, audio (Sound, SoundBuffer, Music), procgen (BSP, HeightMap,
NoiseSource, DijkstraMap, AStarPath), all 12 enums, the snake_case module functions, and the
singletons. (Provisionally frozen, flagged for confirmation at golden review: `ColorLayer`,
`TileLayer` [core grid layers, distinct from Tiled import], `DiscreteMap` [#293/#294 grid data].)
- **EXPERIMENTAL (exempt, may change post-1.0):** 3D/Voxel (Billboard, Entity3D,
EntityCollection3D[Iter], Model3D, Viewport3D, VoxelGrid, VoxelRegion), Tiled import (TileSetFile,
TileMapFile, WangSet), LDtk import (LdtkProject, AutoRuleSet), Shader, and binding helpers
(CallableBinding, PropertyBinding).
The snapshot test FAILS on any exported class not classified (forces a deliberate decision for
future additions).

93
docs/api-stability.md
View file

@ -1,93 +0,0 @@
# API Stability Policy (1.0 Contract)
This document records the compatibility promises that freeze at McRogueFace 1.0.
Each item below is a **semantic decision that cannot be changed compatibly after
1.0** and was ratified in the 2026-07-02 pre-1.0 memory-model review. Additive,
opt-in APIs may still be introduced after 1.0; the guarantees here constrain
what those additions may assume.
The public `mcrfpy` surface (types, methods, properties, enum members,
functions, singletons, and each property's inferred type / read-only flag) is
guarded against accidental drift by
`tests/unit/api_surface_snapshot_test.py`, which diffs the live module against
a committed golden file. That test is the mechanical enforcement of this policy;
any intentional change to the surface must be a reviewed re-baseline of the
golden file.
## Value semantics for Color and Vector (#326)
`mcrfpy.Color` and `mcrfpy.Vector` are **value types, forever.** They store
their data inline (no shared references), and every property that returns a
Color or Vector returns a **fresh copy**. This matches how `sf::Color` /
`sf::Vector2f` behave in C++ and carries zero implementation risk. Write-through
proxy views were considered and rejected: they would freeze dangling-parent
lifetime and identity questions into the contract.
The direct consequence is that mutating a component of a color/vector obtained
from a property does **not** affect the original — it mutates a throwaway copy:
```python
frame.fill_color.r = 255 # NO-OP: mutates a temporary copy, then discards it
```
The two supported idioms are:
```python
# 1. Read-modify-writeback
c = frame.fill_color # copy
c.r = 255 # mutate the copy
frame.fill_color = c # write the whole value back
# 2. Whole-value assignment
frame.fill_color = mcrfpy.Color(255, 0, 0)
```
This behavior is stated normatively in the `Color` and `Vector` class
docstrings and will not change in the 1.x series.
## Bulk-edit convention for writable views (#328)
Future zero-copy **writable** views over cell/buffer data (numpy / buffer
protocol) are exposed **only** through an `edit()` context manager:
```python
with layer.edit() as view:
view[...] = ... # write freely
# on __exit__, the affected state is conservatively invalidated
# (whole-layer markDirty / full TCOD resync + generation bump as applicable)
```
Forgetting to synchronize is impossible by construction, which applies the
project's fail-early principle to the API surface. The engine performs **no
automatic change detection**; invalidation is triggered unconditionally on
`__exit__`.
- **Read-only** zero-copy views (for example `DiscreteMap.mask()`) may still be
exposed directly, with no `edit()` ceremony, because they cannot violate
engine invariants.
- **No always-writable raw view + `mark_dirty()` primitives will be shipped.**
If a genuine need for persistent writable views emerges after 1.0, such
primitives can be added additively without breaking the `edit()` idiom.
This convention is the pattern that the buffer-protocol work must follow:
ColorLayer/TileLayer buffers (#335), related view APIs (#334), and any future
`grid.walkable` array view (which depends on the GridData SoA refactor, #332).
## Subinterpreters are out of scope for 1.x (#330)
Running mcrfpy in a Python subinterpreter is unsupported in 1.x; the module
declares `m_size = -1` and will refuse or misbehave. See #220.
The module uses single-phase initialization with global state (static
`PyTypeObject`s, cached enum singletons, and global engine pointers), so it
cannot be safely instantiated per-subinterpreter. Excluding subinterpreters from
the 1.0 promise leaves the later refactor (#220: heap types, multi-phase init,
per-interpreter module state) unconstrained. Heap-type conversion does not change
Python-visible behavior, so nothing else needs to be reserved for it.
## Related
- [Threading model](threading-model.md) — the `mcrfpy.lock()` off-main-thread
contract, which likewise freezes at 1.0.
- `tests/unit/api_surface_snapshot_test.py` — enforcement mechanism for the
public API surface.

1751
docs/api_reference_complete.html
View file

File diff suppressed because it is too large Load diff

6191
docs/api_reference_dynamic.html
View file

File diff suppressed because it is too large Load diff

6155
docs/mcrfpy.3
View file

File diff suppressed because it is too large Load diff

187
docs/sprint-perf-342-348.md
View file

@ -1,187 +0,0 @@
# Sprint plan: hot-path perf batch (#342, #343, #344, #348)
Durable plan to fix four profiler-identified hot paths in one sprint, with
**before/after measurement built in**. Baselines below were captured on the
pre-fix tree (master tip `03c9b9f`/`ef0408e`, engine code = #331 `182da62`).
Companion: `docs/profiling.md` (the rig). All four share the same class of fix as
#331 — stop re-doing per-call ceremony on a hot path.
## Measurement protocol (run identically before and after)
One harness drives all four scenarios in isolation, step()-only / pure-loop (no
screenshots — avoids the PNG trap):
```
tests/benchmarks/sprint_perf_baseline.py
```
**Wall-clock** (Release `build/`, representative -O3), median of 3 runs:
```
./build/mcrogueface --headless --exec tests/benchmarks/sprint_perf_baseline.py
```
**Instruction counts** (profile `build-profile/`, deterministic), one run:
```
valgrind --tool=callgrind --callgrind-out-file=build-profile/cg_baseline.out \
build-profile/mcrogueface --headless --exec tests/benchmarks/sprint_perf_baseline.py
callgrind_annotate [--inclusive=yes] build-profile/cg_baseline.out
```
Re-run both after each fix; compare the same `BASELINE | ...` lines and the same
per-function Ir. Callgrind is the primary acceptance signal (deterministic);
wall-clock is the sanity check.
**Note on wall-clock vs Callgrind divergence** (observed here, expected): the key
scenario dominates wall-clock (`automation.keyDown` blocks on X11/OS syscalls — many
ms, few app instructions) while the animation scenario dominates Callgrind
(`Animation::update` ~42% of instructions — pure CPU, no syscalls). Judge each fix by
the metric that reflects its cost: CPU-bound fixes (#342/#348) by Callgrind Ir;
per-event fixes (#344) by Callgrind Ir (wall-clock too noisy).
## Baseline numbers (pre-fix)
Harness scale: anim=500 frames×2 props/200 steps, update=5000 steps, keys=3000
events, grid=120²×5 passes. Callgrind harness total = **3,522,924,932 Ir**.
| Issue | Wall-clock (median/3) | Target function (Callgrind) | Self Ir | % harness | calls (observed) |
|---|---|---|---|---|---|
| #342 | 6.68 ms / 200k ops | `UIFrame::setProperty(string,float)` | 158.6M | 4.50% | 5.2M |
| #343 | 172 ms / 5k steps | `PyObject_GetAttrString("update")`/step | ~3.4k Ir/call | small | ~1/step |
| #344 | 665 ms / 6k ops (noisy 98–135 µs) | `PyObject_CallFunction` (enum ctor) | 19.18M | 0.54% | 6,000 |
| #348 | 81.8 ms / 72k ops | `UIGridView::get_grid` | 72.5M | 2.06% | 72,000 |
Supporting inclusive costs for #342: `Animation::applyValue` 621M (17.63%),
`Animation::update` 1,479M (41.99%). For #344: `injectKeyEvent` 308M (8.76%) is
harness-only injection machinery, **not** a fix target. For #348: `get_grid`'s
callees `lookup`/`registerObject`/`PyWeakref_NewRef` each fire 72,000× → **0% cache
hit** (the defect).
## Suggested order
1. **#348** first — biggest self-cost after animation, cleanest fix, clearest
Callgrind signal (cache-hit rate 0%→~100%), and it de-risks the shim for the
others touching the grid.
2. **#342** — highest % of harness instructions; the win is CPU/frame-budget, which
Callgrind reveals even though wall-clock hid it.
3. **#343**, **#344** — smaller, mechanical; batch together.
## Per-issue plan
### #348 — get_grid re-allocates a Grid wrapper + weakref per call
- **Root cause** (`src/UIGridView.cpp` `get_grid`): allocates a temporary `UIGrid`
wrapper, caches only a **weakref** to it, returns it; caller drops it → weakref
dies → next call misses and re-allocates. 0% cache hit.
- **Fix direction**: give the shim a persistent handle to its own `UIGrid` wrapper
(strong ref owned for the shim's lifetime, or store `PyObject*` on GridData and
return it INCREF'd) so repeated access reuses one object. Mind the Grid↔wrapper
ownership cycle — reuse the `#251` tp_dealloc `use_count()` break pattern.
- **Files**: `src/UIGridView.cpp`, `src/UIGrid.h/.cpp`, possibly `PythonObjectCache`.
- **Acceptance**: `get_grid` inclusive Ir drops sharply; `PyWeakref_NewRef` /
`registerObject` call counts under `get_grid` fall from 72,000 to ~1;
`348_grid_churn` wall-clock improves. No new leak (ASan test suite clean).
- **Risk**: ownership/lifetime — must not leak the grid or double-free. Add a
regression test that hammers `grid.at()` in a loop and checks refcount stability.
### #342 — Animation applyValue → setProperty strcmp cascade
- **Root cause** (`src/Animation.cpp` `applyValue` → `UIDrawable::setProperty(const
std::string&, ...)`, e.g. `src/UIFrame.cpp:843`): per active animation per frame,
the property name is re-resolved by a linear string-compare chain.
- **Fix direction**: resolve the property **once** at `Animation::start()` to a
stable handle — an enum id or a cached setter (member function pointer / small
dispatch struct) — and call that each frame. Keep the string path for the
one-time resolve and for `setProperty`'s existing public callers.
- **Files**: `src/Animation.h/.cpp`, `src/UIFrame.cpp`, `src/UICaption.cpp`,
`src/UISprite.cpp`, `src/UIEntity.cpp`, `src/UIGrid.cpp` (each `setProperty`).
- **Acceptance**: `setProperty` self Ir (158.6M) and `applyValue` inclusive (621M)
drop; `342_anim_dispatch` wall-clock improves modestly. Common props (`x`,
`opacity`) already hit early in the cascade, so expect a larger win on deep props
(`fill_color.a`) — consider adding a deep-property variant to the harness.
- **Risk**: keep behavior identical for every property/type combination; the
existing animation tests must stay green.
### #343 — PyScene::update GetAttrString every frame
- **Root cause** (`src/PySceneObject.cpp:405`): `PyObject_GetAttrString(self,
"update")` every frame even when the subclass doesn't override `update`.
- **Fix direction**: resolve once whether the subclass overrides `update` (cache a
bool or the bound method), skip the lookup otherwise — mirror the hover fast-path
guard at `src/PyScene.cpp:361` (`is_python_subclass`).
- **Files**: `src/PySceneObject.cpp` (+ wherever per-scene override state lives).
- **Acceptance**: per-step `GetAttrString("update")` calls → 0 for a plain scene;
`343_scene_update` wall-clock improves. Overriding scenes still get `update` called.
- **Risk**: must still invoke `update` when it IS overridden (incl. late-bound /
monkeypatched). Test both a plain and an overriding scene.
### #344 — on_key rebuilds Key/InputState enum members per event
- **Root cause** (`src/PySceneObject.cpp:362,373`): `PyObject_CallFunction(enum_class,
"i", value)` per event constructs the IntEnum member via Python `EnumMeta.__call__`.
- **Fix direction**: pre-build the enum members once into a lookup table (array/dict
keyed by int value) and return the cached `PyObject*` (INCREF) per event.
- **Files**: `src/PySceneObject.cpp`, `src/PyKey.cpp`, `src/PyInputState*` (enum defs).
- **Acceptance**: `PyObject_CallFunction` (enum ctor) 6,000-call / 19.18M-Ir cost →
near-zero (replaced by an array index + INCREF). Wall-clock is too noisy to gate on.
- **Risk**: cached members must stay valid for the interpreter lifetime; hold strong
refs. Enum identity/equality must be unchanged (existing `test_callback_enums.py`).
## Post-fix results (Callgrind A/B, same harness)
Harness total Ir: **3,522,924,932 → 3,211,532,731 (−311.4M, −8.8%)**.
| Issue | Metric | Baseline | Post-fix | Delta |
|---|---|---|---|---|
| #348 | `get_grid` inclusive Ir | 72,575,645 | 2,089,926 | **−97%** |
| #342 | `Animation::update` inclusive Ir | 1,479,186,800 | 1,207,377,600 | **−18.4%** (−272M) |
| #342 | variant `__do_visit` self Ir (anim) | 473.2M | 98.8M | **−79%** |
| #344 | enum-ctor `PyObject_CallFunction` self Ir | 19.18M | 0.228M | **−99%** |
| #343 | `call_update` Ir **per frame** (real doFrame loop) | 4,559 | 3 | **−99.9%** (~1,500×) |
Wall-clock (Release, median/3): #342 6.68→5.91 ms, #343 172→157 ms, #344 665→454 ms
(−31%), #348 81.8→72 ms. Wall-clock understates the CPU wins (the animation win is
hidden behind syscalls in other scenarios; judge by Callgrind Ir).
### #343 measured in the real game loop (not `step()`)
`mcrfpy.step()` bypasses `updatePythonScenes()`, so #343 is invisible headless-`step()`.
It was instead measured on the actual `doFrame()` loop (headless `run()`, which shares
the exact `doFrame` path with the windowed loop), driven by a keep-alive timer and
bounded with `timeout -s INT`. Normalized by `call_update` **Ir/call** so the (necessarily
uncontrolled) frame count doesn't matter:
- Without fix: `call_update` inclusive **794,878,505 Ir / 174,375 frames = 4,559 Ir/frame**
— 35% of *all* instructions in a bare-scene loop (the failed `GetAttrString("update")`
+ `PyErr_Clear` every frame).
- With fix: **1,114,407 Ir / 371,469 frames ≈ 3 Ir/frame** (just the type-pointer compare).
- Throughput: **2.1× more frames** completed in the same Callgrind budget (371k vs 174k).
Bench script: `scratchpad/bench343.py` (bare Scene + keep-alive timer, no `step()`).
## Profiling course-corrections (the sprint working as intended)
The rig **disproved two guessed hot spots** and found the real ones. Recorded here so
we don't re-guess:
- **#342 — the strcmp cascade was cold.** `std::string == const char*` short-circuits
on length, so each `setProperty` name compare is ~0.15%. The real cost was the two
per-frame `std::variant` `std::visit` dispatches (`interpolate()` + `applyValue()`),
**~473M Ir / 13.4% of the harness**. A first attempt to cut the `update()` weak_ptr
triple-lock was also a dead end — locking an *empty* weak_ptr is nearly free (no
control block, no atomics), so it was a wash/slight regression and was reverted. The
shipped fix is a scalar-float fast path (`isSimpleFloatAnim`) that interpolates and
applies directly, bypassing both visits: variant machinery **473M → 99M**.
- **#343 — real but off the `step()` path.** `GameEngine::step()` (headless) does **not**
call `updatePythonScenes()`, so `call_update` never runs under `mcrfpy.step()` — the
sprint's step()-only harness can't see it. Measured instead on the real `doFrame()`
loop (see "#343 measured in the real game loop" above): **4,559 → 3 Ir/frame**, a large,
genuine win — it was just measured with the wrong tool at first. (Verified a base
`mcrfpy.Scene` has no `__dict__`, so `update` can only come from a subclass → the guard
is safe.)
- **#344 / #348 — guesses confirmed.** Cached enum members and a persistent Grid
wrapper are large, clean wins exactly as predicted.
## Definition of done
- [x] All four target metrics measured in `cg_*` A/B (above); #343 documented as
correct-but-not-headless-measurable.
- [x] `cd tests && python3 run_tests.py` green (307/307), incl. new regression tests
`issue_34{2,4,8}_*`.
- [x] Each commit references its issue (`closes #NNN`); this doc carries post-fix numbers.

126
docs/threading-model.md
View file

@ -1,126 +0,0 @@
# Threading Model
This page describes how McRogueFace coordinates the C++ render loop with Python
threads, and states the **normative rule** for touching engine objects from a
thread other than the one running the game loop.
## The rule
> **Any access to `mcrfpy` objects from a non-main thread must happen inside a
> `with mcrfpy.lock():` block. Behavior outside the lock is undefined.**
This is a frozen part of the 1.0 contract. It holds identically under the
default GIL build and under free-threaded (`--disable-gil`) builds. Reads and
writes both require the lock: creating, mutating, or even reading properties of
Frames, Captions, Sprites, Grids, Entities, Colors, Vectors, Scenes, Timers, or
any other `mcrfpy` object from a worker thread is only defined while the lock is
held.
Code that runs on the main thread — script initialization, `Scene.on_key` and
other input handlers, `Timer` callbacks, animation callbacks, grid/cell
callbacks — is already synchronized with the render loop and does **not** need
the lock. `with mcrfpy.lock():` is a no-op there, so the same helper function
can be called safely from both contexts.
## Why the lock exists
The engine is single-threaded by design: the game loop owns all `mcrfpy` state
and mutates it freely between frames without internal locking. Rather than make
every engine object internally thread-safe, McRogueFace exposes one coarse
synchronization point. Worker threads park on it until the main loop opens a
safe window; the main loop never has to defend against concurrent mutation.
This coarse-grained contract is what keeps a future free-threading port
(tracked in #336) tractable — the promise to callers never changes, and the
engine never has to become internally thread-safe.
## How it works
The mechanism is `FrameLock` (`src/GameEngine.cpp`) driving the
`PyLockContext` context manager returned by `mcrfpy.lock()`
(`src/PyLock.cpp`):
1. **GIL released around `display()`.** Each frame, the engine releases the GIL
while it presents the frame (`Py_BEGIN_ALLOW_THREADS` around
`window->display()` / `headless_renderer->display()`,
`src/GameEngine.cpp:475-499`). This lets waiting Python worker threads get
scheduled.
2. **Safe window between frames.** After `display()` and before the next
frame's processing, if any thread is waiting on the frame lock, the engine
opens a window (`frameLock.openWindow()`), releases the GIL so the waiting
threads can run their `with mcrfpy.lock():` bodies, then closes the window
once they finish (`frameLock.closeWindow()`).
3. **`mcrfpy.lock()` on a worker thread blocks (GIL released) until that
window opens.** `PyLockContext.__enter__` calls `FrameLock::acquire()`,
which waits on a condition variable with the GIL released
(`src/GameEngine.cpp:29-40`). When the block exits, `__exit__` releases the
frame lock.
4. **`mcrfpy.lock()` on the main thread is a no-op.** `__enter__` detects the
main thread and returns immediately without acquiring anything
(`src/PyLock.cpp:70-85`), because the main thread is already the sole owner
of engine state.
5. **Engine → Python callbacks re-acquire the GIL.** All callbacks the engine
invokes into Python (animations, scene/input handlers, timers) wrap their
work in `PyGILState_Ensure` / `PyGILState_Release`
(`src/Animation.cpp`, `src/PySceneObject.cpp`), so they run correctly
regardless of which thread advanced the frame.
## Worker-thread example
A background thread computes something expensive (pathfinding, procedural
generation, network I/O) and then updates the UI. Only the UI update needs the
lock; the computation itself does not touch `mcrfpy` objects and runs freely.
```python
import mcrfpy
import threading
scene = mcrfpy.Scene("game")
label = mcrfpy.Caption(text="working...", pos=(10, 10))
scene.children.append(label)
mcrfpy.current_scene = scene
def worker():
# Heavy work OUTSIDE the lock: touches no mcrfpy objects.
result = expensive_computation()
# Touching mcrfpy objects: MUST be inside the lock on a worker thread.
with mcrfpy.lock():
label.text = f"done: {result}"
label.fill_color = mcrfpy.Color(0, 255, 0)
# Read-modify-writeback for value types (Color/Vector) also goes
# inside the lock when done off the main thread:
with mcrfpy.lock():
c = label.fill_color # copy
c.a = 128 # mutate the copy
label.fill_color = c # write it back
threading.Thread(target=worker, daemon=True).start()
```
### What not to do
```python
def worker_bad():
result = expensive_computation()
# WRONG: mutating an mcrfpy object from a worker thread without the lock.
# Behavior is undefined — visual glitches, torn reads, or a crash.
label.text = f"done: {result}"
```
Keep lock blocks short. The main loop is stalled for the duration of every open
window, so do heavy work outside the lock and hold it only for the actual
engine-object access.
## Related
- `mcrfpy.lock()` — the API entry point (see its docstring).
- [API stability policy](api-stability.md) — the frozen 1.0 contract.
- #219 — original concurrency design.
- #336 — free-threading hardening (implementation may harden; the contract above does not change).
- #220 — subinterpreters, the *other* concurrency model, which is out of scope for 1.x (see the stability policy).

2067
stubs/mcrfpy.pyi
View file

File diff suppressed because it is too large Load diff

137
tests/unit/api_manifest_test.py Normal file
View file

@ -0,0 +1,137 @@
#!/usr/bin/env python3
"""
Direct-execution test for the API manifest generator (tools/generate_api_manifest.py).
Verifies:
- the generator runs inside the engine and writes api/manifest.json
- the manifest carries the expected schema keys and structure
- output is deterministic across two consecutive runs
Run:
cd build && ./mcrogueface --headless --exec ../tests/unit/api_manifest_test.py
"""
import os
import sys
import json
import importlib.util
import subprocess
def fail(msg):
print("FAIL: " + msg)
sys.exit(1)
def find_repo_root():
try:
out = subprocess.run(
["git", "rev-parse", "--show-toplevel"],
cwd=os.getcwd(), stdout=subprocess.PIPE, stderr=subprocess.PIPE)
if out.returncode == 0 and out.stdout.strip():
return out.stdout.decode("utf-8").strip()
except Exception:
pass
# Fallback: two levels up from this test file (tests/unit/ -> repo).
return os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
def load_generator(repo):
gen_path = os.path.join(repo, "tools", "generate_api_manifest.py")
if not os.path.exists(gen_path):
fail("generator not found at " + gen_path)
spec = importlib.util.spec_from_file_location("gen_api_manifest", gen_path)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
return mod
def main():
repo = find_repo_root()
manifest_path = os.path.join(repo, "api", "manifest.json")
full_path = os.path.join(repo, "docs", "generated", "api_full.json")
gen = load_generator(repo)
# First run.
gen.main()
if not os.path.exists(manifest_path):
fail("manifest.json not written")
if not os.path.exists(full_path):
fail("api_full.json not written")
with open(manifest_path, "rb") as f:
run1 = f.read()
with open(full_path, "rb") as f:
run1_full = f.read()
# Second run -- must be byte-identical (deterministic).
gen.main()
with open(manifest_path, "rb") as f:
run2 = f.read()
with open(full_path, "rb") as f:
run2_full = f.read()
if run1 != run2:
fail("manifest.json is not deterministic across two runs")
if run1_full != run2_full:
fail("api_full.json is not deterministic across two runs")
# Schema / structure checks.
manifest = json.loads(run1.decode("utf-8"))
required = ["schema", "version", "commit", "dirty", "seeded", "objects", "functions"]
for key in required:
if key not in manifest:
fail("manifest missing top-level key: " + key)
if manifest["schema"] != 1:
fail("schema is not 1")
if not isinstance(manifest["objects"], dict) or not manifest["objects"]:
fail("objects is empty or not a dict")
if not isinstance(manifest["functions"], dict) or not manifest["functions"]:
fail("functions is empty or not a dict")
if not isinstance(manifest["dirty"], bool):
fail("dirty is not a bool")
if not isinstance(manifest["seeded"], bool):
fail("seeded is not a bool")
# A known object with expected member/lifecycle structure.
if "Grid" not in manifest["objects"]:
fail("expected object 'Grid' missing")
grid = manifest["objects"]["Grid"]
for key in ["kind", "doc_sha", "lifecycle", "members"]:
if key not in grid:
fail("Grid missing key: " + key)
if grid["kind"] != "class":
fail("Grid kind is not 'class'")
if "since" not in grid["lifecycle"]:
fail("Grid lifecycle missing 'since'")
if len(grid["doc_sha"]) != 16:
fail("doc_sha is not 16 hex chars")
# Every member carries the required fields and a valid kind.
valid_kinds = {"method", "classmethod", "staticmethod", "property"}
for oname, obj in manifest["objects"].items():
for mname, member in obj["members"].items():
for key in ["kind", "signature", "doc_sha", "lifecycle"]:
if key not in member:
fail("%s.%s missing key: %s" % (oname, mname, key))
if member["kind"] not in valid_kinds:
fail("%s.%s has invalid kind: %s" % (oname, mname, member["kind"]))
if len(member["doc_sha"]) != 16:
fail("%s.%s doc_sha not 16 hex" % (oname, mname))
# Functions carry required fields.
for fname, fn in manifest["functions"].items():
for key in ["kind", "signature", "doc_sha", "lifecycle"]:
if key not in fn:
fail("function %s missing key: %s" % (fname, key))
if fn["kind"] != "function":
fail("function %s kind is not 'function'" % fname)
print("PASS: manifest deterministic, schema valid, %d objects / %d functions"
% (len(manifest["objects"]), len(manifest["functions"])))
sys.exit(0)
if __name__ == "__main__":
main()

397
tools/api_delta.py Executable file
View file

@ -0,0 +1,397 @@
#!/usr/bin/env python3
"""
api_delta.py -- diff two McRogueFace API manifests.
Plain python3, standard library only. Loads api/manifest.json from two git
refs (or the working tree via ".") and reports what changed in the API surface.
Usage:
api_delta.py REF1 REF2 [--format md|json|gitea] [--site-dir PATH]
REF1 / REF2:
A git ref (branch, tag, commit) whose api/manifest.json is read via
`git show REF:api/manifest.json`, or "." to read the working-tree file
at api/manifest.json.
--format:
md (default) human-readable Markdown tables
json machine-readable delta structure
gitea a checklist-style issue body grouped by object ("- [ ]" items)
--site-dir PATH:
Scan a Jekyll site for pages whose YAML frontmatter carries an
`mcrf.objects` list; affected pages are attached to each object entry.
"""
import os
import re
import sys
import json
import argparse
import subprocess
# ---------------------------------------------------------------------------
# Manifest loading
# ---------------------------------------------------------------------------
def git_output(args, cwd):
proc = subprocess.run(
["git"] + args, cwd=cwd,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
return proc.returncode, proc.stdout.decode("utf-8", "replace"), \
proc.stderr.decode("utf-8", "replace")
def repo_root():
rc, out, _ = git_output(["rev-parse", "--show-toplevel"], os.getcwd())
if rc == 0 and out.strip():
return out.strip()
return os.getcwd()
def load_manifest(ref, repo):
"""Load api/manifest.json for a ref, or from the working tree when ref == '.'."""
if ref == ".":
path = os.path.join(repo, "api", "manifest.json")
with open(path, "r") as f:
return json.load(f)
rc, out, err = git_output(["show", "%s:api/manifest.json" % ref], repo)
if rc != 0:
raise SystemExit("error: cannot read api/manifest.json at ref '%s': %s"
% (ref, err.strip()))
return json.loads(out)
# ---------------------------------------------------------------------------
# Jekyll frontmatter scan (minimal, no deps)
# ---------------------------------------------------------------------------
def frontmatter_lines(raw):
lines = raw.splitlines()
if not lines or lines[0].strip() != "---":
return None
for i in range(1, len(lines)):
if lines[i].strip() == "---":
return lines[1:i]
return None
def extract_objects_list(hay):
names = []
# inline form: objects: [A, B, "C"]
m = re.search(r"objects:\s*\[([^\]]*)\]", hay)
if m:
for part in m.group(1).split(","):
p = part.strip().strip("'\"")
if p:
names.append(p)
return names
# block form: objects:\n - A\n - B
m = re.search(r"(?m)^\s*objects:\s*$", hay)
if m:
for line in hay[m.end():].splitlines():
if line.strip() == "":
continue
lm = re.match(r"^\s*-\s*(.+?)\s*$", line)
if lm:
names.append(lm.group(1).strip().strip("'\""))
continue
# a non-list, non-blank line ends the list
break
return names
def objects_from_frontmatter(raw):
fm = frontmatter_lines(raw)
if fm is None:
return []
text = "\n".join(fm)
m = re.search(r"(?m)^mcrf:[ \t]*(.*)$", text)
if not m:
return []
inline_after = m.group(1).strip()
block = []
for line in text[m.end():].splitlines():
if line.strip() == "":
block.append(line)
continue
if re.match(r"^\s", line):
block.append(line)
else:
break
hay = inline_after + "\n" + "\n".join(block)
return extract_objects_list(hay)
def scan_site(site_dir):
"""Return {object_name: [relative page paths]}."""
mapping = {}
exts = (".md", ".markdown", ".html", ".htm")
for root, _dirs, files in os.walk(site_dir):
for fname in files:
if not fname.lower().endswith(exts):
continue
path = os.path.join(root, fname)
try:
with open(path, "r", errors="replace") as f:
raw = f.read(8192)
except Exception:
continue
objs = objects_from_frontmatter(raw)
if not objs:
continue
rel = os.path.relpath(path, site_dir)
for name in objs:
mapping.setdefault(name, set()).add(rel)
return {k: sorted(v) for k, v in mapping.items()}
# ---------------------------------------------------------------------------
# Delta computation
# ---------------------------------------------------------------------------
def member_changed(a, b):
"""Classify a member change: 'signature' or 'doc' or None."""
if a.get("kind") != b.get("kind"):
return "signature"
if a.get("signature", "") != b.get("signature", ""):
return "signature"
if a.get("doc_sha") != b.get("doc_sha"):
return "doc"
return None
def compute_delta(a, b, object_pages):
a_objs = a.get("objects", {})
b_objs = b.get("objects", {})
a_fns = a.get("functions", {})
b_fns = b.get("functions", {})
def pages_for(name):
return object_pages.get(name, []) if object_pages else []
objects_added = sorted(set(b_objs) - set(a_objs))
objects_removed = sorted(set(a_objs) - set(b_objs))
objects_changed = {}
for name in sorted(set(a_objs) & set(b_objs)):
a_mem = a_objs[name].get("members", {})
b_mem = b_objs[name].get("members", {})
added = sorted(set(b_mem) - set(a_mem))
removed = sorted(set(a_mem) - set(b_mem))
sig_changed = []
doc_changed = []
for m in sorted(set(a_mem) & set(b_mem)):
kind = member_changed(a_mem[m], b_mem[m])
if kind == "signature":
sig_changed.append(m)
elif kind == "doc":
doc_changed.append(m)
obj_doc_changed = a_objs[name].get("doc_sha") != b_objs[name].get("doc_sha")
if added or removed or sig_changed or doc_changed or obj_doc_changed:
objects_changed[name] = {
"members_added": added,
"members_removed": removed,
"members_signature_changed": sig_changed,
"members_doc_changed": doc_changed,
"object_doc_changed": obj_doc_changed,
"pages": pages_for(name),
}
functions_added = sorted(set(b_fns) - set(a_fns))
functions_removed = sorted(set(a_fns) - set(b_fns))
functions_sig_changed = []
functions_doc_changed = []
for m in sorted(set(a_fns) & set(b_fns)):
kind = member_changed(a_fns[m], b_fns[m])
if kind == "signature":
functions_sig_changed.append(m)
elif kind == "doc":
functions_doc_changed.append(m)
return {
"ref1": None,
"ref2": None,
"objects_added": [{"name": n, "pages": pages_for(n)} for n in objects_added],
"objects_removed": [{"name": n, "pages": pages_for(n)} for n in objects_removed],
"objects_changed": objects_changed,
"functions_added": functions_added,
"functions_removed": functions_removed,
"functions_signature_changed": functions_sig_changed,
"functions_doc_changed": functions_doc_changed,
}
def delta_is_empty(d):
return not (d["objects_added"] or d["objects_removed"] or d["objects_changed"]
or d["functions_added"] or d["functions_removed"]
or d["functions_signature_changed"] or d["functions_doc_changed"])
# ---------------------------------------------------------------------------
# Rendering
# ---------------------------------------------------------------------------
def render_json(d):
return json.dumps(d, sort_keys=True, indent=2)
def _pages_suffix(pages):
return (" (pages: %s)" % ", ".join(pages)) if pages else ""
def render_md(d):
out = []
out.append("# API delta: %s -> %s" % (d["ref1"], d["ref2"]))
out.append("")
if delta_is_empty(d):
out.append("No API changes.")
out.append("")
return "\n".join(out)
if d["objects_added"]:
out.append("## Objects added")
out.append("")
out.append("| object | pages |")
out.append("| --- | --- |")
for o in d["objects_added"]:
out.append("| `%s` | %s |" % (o["name"], ", ".join(o["pages"]) or "-"))
out.append("")
if d["objects_removed"]:
out.append("## Objects removed")
out.append("")
out.append("| object | pages |")
out.append("| --- | --- |")
for o in d["objects_removed"]:
out.append("| `%s` | %s |" % (o["name"], ", ".join(o["pages"]) or "-"))
out.append("")
if d["objects_changed"]:
out.append("## Objects changed")
out.append("")
for name in sorted(d["objects_changed"]):
info = d["objects_changed"][name]
out.append("### %s%s" % (name, _pages_suffix(info["pages"])))
out.append("")
if info.get("object_doc_changed"):
out.append("- class docstring changed")
out.append("| change | members |")
out.append("| --- | --- |")
rows = [
("added", info["members_added"]),
("removed", info["members_removed"]),
("signature changed", info["members_signature_changed"]),
("doc changed", info["members_doc_changed"]),
]
for label, members in rows:
if members:
out.append("| %s | %s |"
% (label, ", ".join("`%s`" % m for m in members)))
out.append("")
fn_rows = [
("added", d["functions_added"]),
("removed", d["functions_removed"]),
("signature changed", d["functions_signature_changed"]),
("doc changed", d["functions_doc_changed"]),
]
if any(members for _, members in fn_rows):
out.append("## Functions")
out.append("")
out.append("| change | functions |")
out.append("| --- | --- |")
for label, members in fn_rows:
if members:
out.append("| %s | %s |"
% (label, ", ".join("`%s`" % m for m in members)))
out.append("")
return "\n".join(out)
def render_gitea(d):
out = []
out.append("## API changes: %s..%s" % (d["ref1"], d["ref2"]))
out.append("")
if delta_is_empty(d):
out.append("No API changes.")
out.append("")
return "\n".join(out)
for o in d["objects_added"]:
out.append("### %s (new object)%s" % (o["name"], _pages_suffix(o["pages"])))
out.append("- [ ] document new object `%s`" % o["name"])
out.append("")
for o in d["objects_removed"]:
out.append("### %s (removed object)%s" % (o["name"], _pages_suffix(o["pages"])))
out.append("- [ ] remove references to `%s`" % o["name"])
out.append("")
for name in sorted(d["objects_changed"]):
info = d["objects_changed"][name]
out.append("### %s%s" % (name, _pages_suffix(info["pages"])))
if info.get("object_doc_changed"):
out.append("- [ ] review class docstring change")
for m in info["members_added"]:
out.append("- [ ] member added: `%s`" % m)
for m in info["members_removed"]:
out.append("- [ ] member removed: `%s`" % m)
for m in info["members_signature_changed"]:
out.append("- [ ] signature changed: `%s`" % m)
for m in info["members_doc_changed"]:
out.append("- [ ] doc changed: `%s`" % m)
out.append("")
fn_items = (
[("added", m) for m in d["functions_added"]]
+ [("removed", m) for m in d["functions_removed"]]
+ [("signature changed", m) for m in d["functions_signature_changed"]]
+ [("doc changed", m) for m in d["functions_doc_changed"]]
)
if fn_items:
out.append("### Functions")
for label, m in fn_items:
out.append("- [ ] %s: `%s`" % (label, m))
out.append("")
return "\n".join(out)
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
def main(argv=None):
parser = argparse.ArgumentParser(
description="Diff two McRogueFace API manifests.")
parser.add_argument("ref1", help="first git ref, or '.' for the working tree")
parser.add_argument("ref2", help="second git ref, or '.' for the working tree")
parser.add_argument("--format", choices=["md", "json", "gitea"], default="md")
parser.add_argument("--site-dir", default=None,
help="Jekyll site dir to scan for mcrf.objects frontmatter")
args = parser.parse_args(argv)
repo = repo_root()
a = load_manifest(args.ref1, repo)
b = load_manifest(args.ref2, repo)
object_pages = scan_site(args.site_dir) if args.site_dir else None
delta = compute_delta(a, b, object_pages)
delta["ref1"] = args.ref1
delta["ref2"] = args.ref2
if args.format == "json":
print(render_json(delta))
elif args.format == "gitea":
print(render_gitea(delta))
else:
print(render_md(delta))
return 0
if __name__ == "__main__":
sys.exit(main())

374
tools/generate_api_manifest.py Normal file
View file

@ -0,0 +1,374 @@
#!/usr/bin/env python3
"""
API manifest generator for McRogueFace.
Runs INSIDE the engine (mcrfpy embedded interpreter) and introspects the
compiled `mcrfpy` module to emit two artifacts:
api/manifest.json -- compact, committed, deterministic baseline
docs/generated/api_full.json -- same tree plus full docstring text
Invoke:
cd build && ./mcrogueface --headless --exec ../tools/generate_api_manifest.py
The manifest tracks per-object / per-member lifecycle (since / modified) by
diffing against the previous committed manifest (git show HEAD:api/manifest.json).
Renderers must tolerate optional future lifecycle keys "deprecated" and "removed".
Pure ASCII source (embedded interpreter constraint).
"""
import os
import sys
import json
import types
import hashlib
import subprocess
try:
import mcrfpy
except ImportError:
print("Error: this script must be run with McRogueFace as the interpreter")
print("Usage: ./build/mcrogueface --headless --exec ../tools/generate_api_manifest.py")
sys.exit(1)
SCHEMA_VERSION = 1
# ---------------------------------------------------------------------------
# Git / version helpers
# ---------------------------------------------------------------------------
def run_git(args, repo):
"""Return (returncode, stdout_text). Never raises."""
try:
proc = subprocess.run(
["git"] + args,
cwd=repo,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
return proc.returncode, proc.stdout.decode("utf-8", "replace")
except Exception:
return 1, ""
def find_repo_root():
rc, out = run_git(["rev-parse", "--show-toplevel"], os.getcwd())
if rc == 0 and out.strip():
return out.strip()
# Fall back to the tools/ directory's parent.
here = os.path.dirname(os.path.abspath(__file__))
return os.path.dirname(here)
def parse_base_version(repo):
"""Parse MCRFPY_VERSION from src/McRogueFaceVersion.h."""
header = os.path.join(repo, "src", "McRogueFaceVersion.h")
with open(header, "r") as f:
text = f.read()
marker = "#define MCRFPY_VERSION"
for line in text.splitlines():
line = line.strip()
if line.startswith(marker):
q1 = line.find('"')
q2 = line.find('"', q1 + 1)
if q1 != -1 and q2 != -1:
return line[q1 + 1:q2]
raise RuntimeError("Could not parse MCRFPY_VERSION from " + header)
def compute_version_info(repo):
base = parse_base_version(repo)
rc, _ = run_git(["describe", "--tags", "--exact-match"], repo)
exact_tag = (rc == 0)
version = base if exact_tag else (base + "-dev")
rc, out = run_git(["rev-parse", "--short", "HEAD"], repo)
commit = out.strip() if rc == 0 else ""
rc, out = run_git(["status", "--porcelain", "--", "src/"], repo)
dirty = bool(out.strip()) if rc == 0 else False
return base, version, commit, dirty
def load_previous_manifest(repo):
"""Return the previous committed manifest dict, or None on first run."""
rc, out = run_git(["show", "HEAD:api/manifest.json"], repo)
if rc != 0 or not out.strip():
return None
try:
return json.loads(out)
except Exception:
return None
# ---------------------------------------------------------------------------
# Introspection helpers
# ---------------------------------------------------------------------------
def doc_hash(doc):
"""First 16 hex chars of sha256 of the docstring (hash empty string if absent)."""
return hashlib.sha256((doc or "").encode("utf-8")).hexdigest()[:16]
def looks_like_signature(line):
"""A line looks like a signature if it is an optional identifier followed by
a parenthesised argument list: e.g. 'step(dt: float) -> float' or '(a, b)'."""
s = line.strip()
i = 0
n = len(s)
# optional leading identifier
while i < n and (s[i].isalnum() or s[i] == "_"):
i += 1
# optional spaces
while i < n and s[i] == " ":
i += 1
if i >= n or s[i] != "(":
return False
return ")" in s[i:]
def extract_signature(doc, text_signature):
"""First docstring line when it looks like a signature, else __text_signature__,
else empty string."""
if doc:
first = doc.strip().split("\n", 1)[0].strip()
if looks_like_signature(first):
return first
if text_signature:
return text_signature
return ""
def raw_descriptor(cls, name):
for base in getattr(cls, "__mro__", (cls,)):
if name in base.__dict__:
return base.__dict__[name]
return None
def classify_member(cls, name, attr):
"""Return one of method|classmethod|staticmethod|property, or None if the
attribute is not a documented API member (e.g. a plain enum value)."""
raw = raw_descriptor(cls, name)
raw_type = type(raw).__name__
if isinstance(raw, staticmethod):
return "staticmethod"
if raw_type == "classmethod_descriptor":
return "classmethod"
if isinstance(raw, property):
return "property"
if isinstance(raw, (types.GetSetDescriptorType, types.MemberDescriptorType)):
return "property"
if raw_type in ("method_descriptor", "wrapper_descriptor",
"builtin_function_or_method", "function") and callable(attr):
return "method"
return None
def member_entry(cls, name):
"""Build a full member entry (with 'doc'), or None if not an API member."""
try:
attr = getattr(cls, name)
except Exception:
return None
kind = classify_member(cls, name, attr)
if kind is None:
return None
doc = getattr(attr, "__doc__", None) or ""
text_sig = getattr(attr, "__text_signature__", None)
sig = extract_signature(doc, text_sig)
return {
"kind": kind,
"signature": sig,
"doc": doc,
"doc_sha": doc_hash(doc),
}
def collect_classes():
"""Return {class_name: full_entry_without_lifecycle}."""
classes = {}
for name in sorted(dir(mcrfpy)):
if name.startswith("_"):
continue
obj = getattr(mcrfpy, name)
if not isinstance(obj, type):
continue
doc = obj.__doc__ or ""
members = {}
for mname in sorted(dir(obj)):
if mname.startswith("__"):
continue
entry = member_entry(obj, mname)
if entry is not None:
members[mname] = entry
classes[name] = {
"kind": "class",
"doc": doc,
"doc_sha": doc_hash(doc),
"members": members,
}
return classes
def collect_functions():
"""Return {func_name: full_entry_without_lifecycle}."""
functions = {}
for name in sorted(dir(mcrfpy)):
if name.startswith("_"):
continue
obj = getattr(mcrfpy, name)
if isinstance(obj, type):
continue
if not callable(obj):
continue
doc = getattr(obj, "__doc__", None) or ""
text_sig = getattr(obj, "__text_signature__", None)
sig = extract_signature(doc, text_sig)
functions[name] = {
"kind": "function",
"signature": sig,
"doc": doc,
"doc_sha": doc_hash(doc),
}
return functions
# ---------------------------------------------------------------------------
# Lifecycle assignment
# ---------------------------------------------------------------------------
def assign_lifecycle(cur, prev, new_since, version, commit, has_signature):
"""Attach a 'lifecycle' dict to cur based on the previous entry.
- new (prev is None) -> {"since": new_since}
- doc_sha or signature changed -> copy prev lifecycle, set modified={version,commit}
- unchanged -> copy prev lifecycle verbatim
"""
if prev is None:
return {"since": new_since}
prev_lc = prev.get("lifecycle", {})
lifecycle = json.loads(json.dumps(prev_lc)) # deep copy, verbatim
if "since" not in lifecycle:
lifecycle["since"] = new_since
changed = prev.get("doc_sha") != cur["doc_sha"]
if has_signature:
changed = changed or (prev.get("signature", "") != cur.get("signature", ""))
if changed:
lifecycle["modified"] = {"version": version, "commit": commit}
return lifecycle
def build_tree(classes, functions, prev, first_run, base_version, version, commit):
"""Produce the full tree (entries retain 'doc'); lifecycle is computed here."""
new_since = base_version if first_run else version
prev_objects = (prev or {}).get("objects", {})
prev_functions = (prev or {}).get("functions", {})
objects_out = {}
for cname, centry in classes.items():
prev_obj = prev_objects.get(cname)
obj_lifecycle = assign_lifecycle(
centry, prev_obj, new_since, version, commit, has_signature=False)
prev_members = (prev_obj or {}).get("members", {})
members_out = {}
for mname, mentry in centry["members"].items():
prev_member = prev_members.get(mname)
mentry = dict(mentry)
mentry["lifecycle"] = assign_lifecycle(
mentry, prev_member, new_since, version, commit, has_signature=True)
members_out[mname] = mentry
objects_out[cname] = {
"kind": "class",
"doc": centry["doc"],
"doc_sha": centry["doc_sha"],
"lifecycle": obj_lifecycle,
"members": members_out,
}
functions_out = {}
for fname, fentry in functions.items():
prev_fn = prev_functions.get(fname)
fentry = dict(fentry)
fentry["lifecycle"] = assign_lifecycle(
fentry, prev_fn, new_since, version, commit, has_signature=True)
functions_out[fname] = fentry
return objects_out, functions_out
def strip_docs(node):
"""Recursively remove 'doc' keys, returning a new structure for the compact manifest."""
if isinstance(node, dict):
return {k: strip_docs(v) for k, v in node.items() if k != "doc"}
if isinstance(node, list):
return [strip_docs(v) for v in node]
return node
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
def main():
repo = find_repo_root()
base_version, version, commit, dirty = compute_version_info(repo)
prev = load_previous_manifest(repo)
first_run = prev is None
classes = collect_classes()
functions = collect_functions()
objects_out, functions_out = build_tree(
classes, functions, prev, first_run, base_version, version, commit)
full = {
"schema": SCHEMA_VERSION,
"version": version,
"commit": commit,
"dirty": dirty,
"seeded": first_run,
"objects": objects_out,
"functions": functions_out,
}
manifest = strip_docs(full)
api_dir = os.path.join(repo, "api")
gen_dir = os.path.join(repo, "docs", "generated")
os.makedirs(api_dir, exist_ok=True)
os.makedirs(gen_dir, exist_ok=True)
manifest_path = os.path.join(api_dir, "manifest.json")
full_path = os.path.join(gen_dir, "api_full.json")
with open(manifest_path, "w") as f:
f.write(json.dumps(manifest, sort_keys=True, separators=(",", ":")))
f.write("\n")
with open(full_path, "w") as f:
f.write(json.dumps(full, sort_keys=True, indent=2))
f.write("\n")
n_obj = len(objects_out)
n_fn = len(functions_out)
n_mem = sum(len(o["members"]) for o in objects_out.values())
print("manifest: %s" % manifest_path)
print("full: %s" % full_path)
print("version=%s commit=%s dirty=%s seeded=%s" % (version, commit, dirty, first_run))
print("objects=%d members=%d functions=%d" % (n_obj, n_mem, n_fn))
if __name__ == "__main__":
main()
sys.exit(0)

104
tools/hooks/pre-commit Executable file
View file

@ -0,0 +1,104 @@
#!/bin/bash
#
# McRogueFace pre-commit hook
#
# Purpose: keep the committed API manifest (api/manifest.json) in lockstep with
# the C++ binding surface, and never let a commit that touches src/ land with a
# broken build or a regressed frozen-docstring surface.
#
# Fast path: if no staged path is under src/, this hook exits 0 immediately --
# doc/asset/test-only commits pay no build cost.
#
# Slow path (staged changes under src/):
# 1. Incremental `make -j$(nproc)` from the repo root. Build failure aborts
# the commit (log tail printed).
# 2. Frozen-docstring gate, run exactly the way tools/generate_all_docs.sh
# runs it (audit_pymethoddef.py + check_frozen_docstrings.sh). A regressed
# frozen docstring aborts the commit.
# 3. Regenerate the rendered docs via tools/generate_all_docs.sh.
# 4. Regenerate api/manifest.json from the freshly-built binary.
# 5. `git add api/manifest.json` (the ONLY index mutation this hook performs).
# 6. Best-effort one-line API delta summary (non-fatal).
#
# Install with: make install-hooks
#
set -uo pipefail
# Resolve repo root; all subsequent steps run from there.
REPO_ROOT="$(git rev-parse --show-toplevel)"
cd "$REPO_ROOT" || exit 1
# ---------------------------------------------------------------------------
# Fast path: bail out unless something under src/ is staged.
# ---------------------------------------------------------------------------
STAGED="$(git diff --cached --name-only)"
if ! grep -q '^src/' <<< "$STAGED"; then
exit 0
fi
echo "[pre-commit] src/ changes staged -- running build + docs/manifest refresh"
JOBS="$(nproc 2>/dev/null || echo 4)"
BUILD_LOG="$(mktemp -t mcrf-precommit-build.XXXXXX.log)"
trap 'rm -f "$BUILD_LOG"' EXIT
# ---------------------------------------------------------------------------
# 1. Incremental build. Failure aborts the commit.
# ---------------------------------------------------------------------------
echo "[pre-commit] make -j$JOBS ..."
if ! make -j"$JOBS" > "$BUILD_LOG" 2>&1; then
echo "[pre-commit] BUILD FAILED -- commit aborted. Last 40 lines:" >&2
tail -n 40 "$BUILD_LOG" >&2
exit 1
fi
# ---------------------------------------------------------------------------
# 2. Frozen-docstring gate (mirrors tools/generate_all_docs.sh).
# Only runs when the .venv-audit tooling is present, exactly as the doc
# generator gates it. A failure here aborts the commit.
# ---------------------------------------------------------------------------
if [ -x "./.venv-audit/bin/python3" ] && [ -f "./tools/audit_pymethoddef.py" ]; then
echo "[pre-commit] PyMethodDef/PyGetSetDef macro compliance audit ..."
./.venv-audit/bin/python3 ./tools/audit_pymethoddef.py --quiet || true
echo "[pre-commit] frozen docstring gate (strict) ..."
if ! ./tools/check_frozen_docstrings.sh; then
echo "[pre-commit] FROZEN DOCSTRING GATE FAILED -- commit aborted." >&2
exit 1
fi
else
echo "[pre-commit] (skipping frozen docstring gate: .venv-audit not found)"
fi
# ---------------------------------------------------------------------------
# 3. Regenerate rendered docs (gitignored; shipped inside release artifacts).
# ---------------------------------------------------------------------------
echo "[pre-commit] regenerating rendered docs ..."
if ! bash tools/generate_all_docs.sh; then
echo "[pre-commit] doc generation FAILED -- commit aborted." >&2
exit 1
fi
# ---------------------------------------------------------------------------
# 4. Regenerate the committed compact manifest from the fresh binary.
# ---------------------------------------------------------------------------
echo "[pre-commit] regenerating api/manifest.json ..."
if ! ( cd build && ./mcrogueface --headless --exec ../tools/generate_api_manifest.py ); then
echo "[pre-commit] manifest generation FAILED -- commit aborted." >&2
exit 1
fi
# ---------------------------------------------------------------------------
# 5. Stage the refreshed manifest. This is the ONLY git-index write in the hook.
# ---------------------------------------------------------------------------
git add api/manifest.json
# ---------------------------------------------------------------------------
# 6. Best-effort one-line API delta summary (non-fatal if the tool is absent).
# ---------------------------------------------------------------------------
if [ -f "./tools/api_delta.py" ]; then
python3 tools/api_delta.py HEAD . --format md 2>/dev/null || true
fi
echo "[pre-commit] done -- api/manifest.json staged."
exit 0

40
tools/package.sh
View file

@ -102,6 +102,40 @@ create_stdlib_zip() {
--output "$output_dir"
}
# Copy the rendered API docs + committed manifest into <package>/docs/.
# These are generated (rendered docs live outside git as of the generated-docs
# .gitignore block) but MUST ship inside every release artifact.
copy_release_docs() {
local package_dir=$1
local docs_out="$package_dir/docs"
log_info "Bundling API documentation into docs/"
mkdir -p "$docs_out"
# "src (relative to PROJECT_ROOT) : dest (relative to <package>/docs)"
local pairs=(
"docs/API_REFERENCE_DYNAMIC.md:API_REFERENCE_DYNAMIC.md"
"docs/api_reference_dynamic.html:api_reference_dynamic.html"
"docs/mcrfpy.3:mcrfpy.3"
"stubs/mcrfpy.pyi:mcrfpy.pyi"
"docs/generated/api_full.json:generated/api_full.json"
"api/manifest.json:manifest.json"
)
local pair src dst
for pair in "${pairs[@]}"; do
src="${pair%%:*}"
dst="${pair#*:}"
if [ -f "$PROJECT_ROOT/$src" ]; then
mkdir -p "$(dirname "$docs_out/$dst")"
cp "$PROJECT_ROOT/$src" "$docs_out/$dst"
else
log_warn "Release doc missing (NOT bundled): $src"
log_warn " run 'bash tools/generate_all_docs.sh' and regenerate the manifest before packaging"
fi
done
}
package_windows() {
local preset=$1
local build_dir="$PROJECT_ROOT/build-windows"
@ -165,6 +199,9 @@ package_windows() {
cp "$build_dir/python314.zip" "$package_dir/"
fi
# Bundle rendered API docs + manifest
copy_release_docs "$package_dir"
# Create the distribution archive
log_info "Creating archive: ${package_name}.zip"
(cd "$DIST_DIR" && zip -r "${package_name}.zip" "$package_name")
@ -279,6 +316,9 @@ package_linux() {
find "$package_dir/lib/Python/Lib" -name "*_test.py" -delete 2>/dev/null || true
fi
# Bundle rendered API docs + manifest
copy_release_docs "$package_dir"
# Create run script
cat > "$package_dir/run.sh" << 'EOF'
#!/bin/bash

103
tools/untrack_generated_docs.sh Executable file
View file

@ -0,0 +1,103 @@
#!/bin/bash
#
# untrack_generated_docs.sh
#
# One-shot migration helper: stops tracking the generated/rendered docs and
# personal-notes docs that the new .gitignore "generated-docs" block now
# ignores, WITHOUT deleting them from your working tree.
#
# It reads the pattern block delimited by the BEGIN/END markers below out of
# the repo's .gitignore, then asks git which CURRENTLY-TRACKED files those
# patterns match (via `git ls-files --cached -i -X <patterns>`). Because the
# pattern source is the .gitignore block itself, this can never drift from what
# is actually ignored.
#
# Usage:
# tools/untrack_generated_docs.sh # dry run: print the git rm --cached commands
# tools/untrack_generated_docs.sh --apply # actually run them
#
# NOTE: `git rm --cached` only removes the files from the index (they stay on
# disk). You still need to `git commit` afterwards to record the removal.
#
set -euo pipefail
REPO_ROOT="$(git rev-parse --show-toplevel)"
GITIGNORE="$REPO_ROOT/.gitignore"
BEGIN_MARKER='# >>> generated-docs (untrack via tools/untrack_generated_docs.sh) >>>'
END_MARKER='# <<< generated-docs <<<'
APPLY=0
case "${1:-}" in
--apply) APPLY=1 ;;
""|--dry-run) APPLY=0 ;;
-h|--help)
echo "Usage: $0 [--apply]"
echo " (no args) dry run -- print the git rm --cached commands"
echo " --apply execute the git rm --cached commands"
exit 0
;;
*)
echo "Unknown argument: $1 (use --apply or no argument)" >&2
exit 1
;;
esac
if [ ! -f "$GITIGNORE" ]; then
echo "ERROR: $GITIGNORE not found" >&2
exit 1
fi
# Extract the pattern lines strictly between the markers, dropping comments and
# blank lines, into a temp exclude file that git can consume.
PATTERNS_FILE="$(mktemp -t mcrf-untrack-patterns.XXXXXX)"
trap 'rm -f "$PATTERNS_FILE"' EXIT
awk -v b="$BEGIN_MARKER" -v e="$END_MARKER" '
$0 == b { inblock=1; next }
$0 == e { inblock=0; next }
inblock {
line=$0
# strip leading/trailing whitespace
gsub(/^[ \t]+|[ \t]+$/, "", line)
if (line == "" || line ~ /^#/) next
print line
}
' "$GITIGNORE" > "$PATTERNS_FILE"
if [ ! -s "$PATTERNS_FILE" ]; then
echo "ERROR: no patterns found between markers in .gitignore" >&2
echo " expected block delimited by:" >&2
echo " $BEGIN_MARKER" >&2
echo " $END_MARKER" >&2
exit 1
fi
# Ask git which tracked files match those ignore patterns.
# -c = cached (tracked), -i = show ignored, -X = exclude-pattern file.
mapfile -t TRACKED < <(cd "$REPO_ROOT" && git ls-files --cached -i -X "$PATTERNS_FILE" | sort -u)
if [ "${#TRACKED[@]}" -eq 0 ]; then
echo "No currently-tracked files match the generated-docs ignore block. Nothing to do."
exit 0
fi
echo "The following ${#TRACKED[@]} tracked file(s) are now covered by the generated-docs .gitignore block:"
echo ""
for f in "${TRACKED[@]}"; do
echo " git rm --cached -- \"$f\""
done
echo ""
if [ "$APPLY" -eq 0 ]; then
echo "Dry run only. Re-run with --apply to execute, then commit the removals."
echo "(Files remain on disk; only the git index entries are removed.)"
exit 0
fi
echo "Applying: git rm --cached ..."
# Remove in one batch; --cached keeps the working-tree files.
( cd "$REPO_ROOT" && git rm --cached --quiet -- "${TRACKED[@]}" )
echo ""
echo "Done. ${#TRACKED[@]} file(s) removed from the index (still present on disk)."
echo "Review with 'git status', then commit the removals."