john/McRogueFace
1
0
Fork 0
Code Issues 35 Pull requests Projects Releases Wiki Activity
Page: Proposal: Hybrid Scene Serialization
Pages
AI and Pathfinding API Stability Contract Adding Python Bindings Animation System Design Proposals Development Workflow Entity Management Grid Interaction Patterns Grid Rendering Pipeline Grid System Grid TCOD Integration Headless Mode Home Input and Events LLM Agent Testbed Architecture Performance and Profiling Plan: Roguelike Tutorial Blueprint Plan: mcrfpy Stdlib Widgets Procedural-Generation Proposal: Hybrid Scene Serialization Proposal: Next Generation Grid & Entity System Python Binding Layer Rendering and Visuals Strategic Direction Threading Model UI Component Hierarchy UI Widget Patterns Writing Tests
2 Proposal: Hybrid Scene Serialization
John McCardle edited this page 2026-07-11 19:51:54 +00:00
Table of Contents

Proposal: Hybrid Scene Serialization

Last modified: 2026-07-11 00:03

mcrf-meta -- objects: Frame,Caption,Sprite,Grid,Texture,DiscreteMap | type: plan | status: current | verified: 0.2.8 (b6720f6)

Proposal: Hybrid Scene Serialization

Status: Design Phase Complexity: Major Impact: system:rendering, system:ui-hierarchy, system:grid, system:python-binding, system:performance

Related Pages:

Related Issues: #347 (renderer parity — replay fidelity dependency), #255 (perf umbrella), #345 (profiling rig that motivated this)

Problem Statement

Capturing what McRogueFace draws currently means rasterizing to PNG via automation.screenshot() → sf::Image::saveToFile (src/McRFPy_Automation.cpp:218). Profiling the #345 rig showed that ~96.6% of instructions in a screenshot-per-frame workload are spent in PNG DEFLATE compression (libsfml stbi_zlib_compress), not in the engine. PNG is convenient to redistribute but far too costly to encode frequently.

Two distinct needs are being served — badly — by one expensive raster mechanism:

  1. Test oracle / regression — "did the UI end up in the state we expected?" Pixel-diffing PNGs is slow and brittle: antialiasing, GPU/driver variance, and backend differences (SFML desktop vs SDL2/WebGL, see #347) cause false failures.
  2. Frequent capture — profiling frames, recording sessions, bug-report snapshots — where per-frame raster cost is prohibitive.

Current Limitations

  • automation.screenshot() is the only capture path; it always rasterizes and always DEFLATE-compresses.
  • No way to snapshot, diff, inspect, persist, or re-render UI state — only pixels.
  • No scene save/load exists (confirmed: no issue or proposal covers it as of 2026-07-10).
  • Pixel comparison is the wrong oracle for layout/logic regressions (the majority), yet it is all that is available.

Proposed Solution

McRogueFace is already a retained-mode engine: a rendered frame is a pure function of (UIDrawable tree + referenced assets). Therefore the tree can be serialized declaratively — SVG-analogous — capturing render inputs instead of outputs.

Core mechanism: recursively walk the scene's UIDrawable hierarchy (the walk already exists in ImGuiSceneExplorer, dispatching on derived_type() over children collections) and emit each node's type + properties + asset references. A 100-element scene becomes a few KB of text at roughly memcpy-of-properties cost — cheaper than even BMP, because the framebuffer is never touched.

Hybrid rasterization escape-hatch: nodes whose pixels are not reducible to an asset path — render-texture Frames with opacity compositing, Texture.composite/hsl_shift outputs, shader results — export a small QOI/TGA tile that the scene file references. Symbolic where an asset path exists; raster only where it must.

Lazy rasterization (this is what solves the frequency problem): serialize cheaply every frame during a run; rasterize to PNG/QOI only the frames you choose to look at, by feeding the scene file back through the engine. MCRF itself is the perfect replayer. Frequent capture becomes nearly free; pixel cost is paid on demand.

Serialization target: round-trip to Python constructor kwargs — each node serializes to the arguments needed to reconstruct it (Frame(pos=..., size=..., fill_color=...), children nested). This makes the artifact a genuine scene save/load format, not merely a screenshot substitute — usable for authoring scenes, persisting UI, and attaching reproducible state to bug reports.

The governing distinction

The format records render inputs, not outputs. Consequences, stated plainly:

  • Superior for layout/logic/state regression: deterministic, diffable (f3.x: 100→104), immune to GPU/AA/driver/backend variance.
  • Blind to rendering bugs — a wrong shader, blend mode, z-order, clip rect, or atlas offset will not appear, because the intent was recorded, not the pixels. This complements screenshots; it does not replace them. Raster capture remains the tool for visual/shader correctness.

Fidelity edges (feasibility-determining)

  • Procedural textures → raster escape-hatch (above).
  • Grids are the largest/most complex node: thousands of cells + color/tile layers + entities + camera/zoom/FOV state. Sparse/RLE-encode cell state (reuse the DiscreteMap byte-serialization path). Still far smaller than pixels.
  • Text (Caption): layout depends on font metrics/kerning/outline — replay requires the same font referenced by path.
  • Shaders / render-to-texture / z-order: the replayer must match the engine exactly. Because the replayer is the engine, this is automatic within one engine version and backend; cross-version/cross-backend replay drifts, which is exactly the surface #347 addresses.

Migration Path

Purely additive — no change to existing rendering or the current automation.screenshot() API.

  1. Serializer (read-only): add to_dict()/serialize() on UIDrawable (recursive), initially covering Frame/Caption/Sprite; a new mcrfpy entry point (e.g. automation.snapshot_scene(path)).
  2. Grid + layers: sparse/RLE cell + layer serialization, reusing DiscreteMap bytes.
  3. Raster escape-hatch: detect non-asset textures, export QOI/TGA sidecars, reference them.
  4. Replayer: load a scene file → reconstruct the tree via the Python API → optional rasterize-on-demand to PNG/QOI.
  5. Test oracle: structural-diff helper for tests/; opt-in per test. Existing screenshot tests remain valid for visual coverage.

Performance Impact

  • Capture: O(nodes) property emission, no compression, no rasterization — expected orders of magnitude below the current per-frame PNG cost (which profiling measured at ~96.6% of a screenshot-heavy workload).
  • Storage: KB-scale text/binary vs MB-scale pixels; grids dominate and are RLE/sparse-bounded.
  • Test suite: removes GPU/driver/backend flakiness from logic/layout tests; faster than image decode+compare.
  • On-demand raster: unchanged PNG/QOI cost, but paid only for chosen frames.

Implementation Complexity

Major, but no piece is novel to the codebase:

  • Tree walk — pattern exists (ImGuiSceneExplorer).
  • Byte serialization — exists (DiscreteMap, .mcvg voxel RLE).
  • Python-API round-trip construction — exists (LDtk/Tiled import build trees from data).

Risk areas: grid/layer serialization size; text layout fidelity on replay; the raster escape-hatch boundary; and cross-backend replay parity (dependency on #347). Recommend incremental delivery per the Migration Path, Frame/Caption/Sprite first, Grid second, replayer third.

Open Questions

  • Serialization encoding: JSON (diffable, verbose) vs a compact binary vs Python-source emission (executable, re-runnable). Possibly JSON for the oracle use, binary for save/load.
  • Should the replayer be in-engine only, or is a lightweight standalone viewer worthwhile (the SVG-in-browser analogue)?
  • Versioning/compatibility policy for stored scene files across engine versions.