From dd575af7177f070f28aa259300625d624d26136f Mon Sep 17 00:00:00 2001
From: John McCardle <john@ffwf.net>
Date: Tue, 16 Dec 2025 21:24:44 +0000
Subject: [PATCH] Add Grid Rendering Pipeline

---
 Grid-Rendering-Pipeline.md | 374 +++++++++++++++++++++++++++++++++++++
 1 file changed, 374 insertions(+)
 create mode 100644 Grid-Rendering-Pipeline.md

diff --git a/Grid-Rendering-Pipeline.md b/Grid-Rendering-Pipeline.md
new file mode 100644
index 0000000..2b8d72f
--- /dev/null
+++ b/Grid-Rendering-Pipeline.md
@@ -0,0 +1,374 @@
+# Grid Rendering Pipeline
+
+## Overview
+
+The Grid rendering pipeline handles multi-layer tilemap rendering with chunk-based caching for optimal performance. It supports arbitrary numbers of rendering layers, viewport culling, zoom, and entity rendering.
+
+**Parent Page:** [[Grid-System]]
+
+**Related Pages:**
+- [[Performance-and-Profiling]] - Metrics and profiling tools
+- [[Entity-Management]] - Entity rendering within grids
+- [[Grid-TCOD-Integration]] - FOV and pathfinding (partially in transition)
+
+**Key Files:**
+- `src/UIGrid.cpp::render()` - Main rendering orchestration
+- `src/GridLayers.cpp` - ColorLayer and TileLayer rendering
+- `src/UIGrid.cpp::renderChunk()` - Per-chunk RenderTexture management
+
+**Related Issues:**
+- [#123](../issues/123) - Chunk-based Grid Rendering (Closed - Implemented)
+- [#148](../issues/148) - Dirty Flag RenderTexture Caching (Closed - Implemented)
+- [#147](../issues/147) - Dynamic Layer System (Closed - Implemented)
+- [#113](../issues/113) - Batch Operations API (Open - includes FOV access discussion)
+- [#115](../issues/115) - SpatialHash for entity culling (Open)
+
+---
+
+## Architecture Overview
+
+### Layer-Based Rendering
+
+Grids render content through **layers** rather than per-cell properties. Each layer is either a `ColorLayer` (solid colors) or `TileLayer` (texture sprites).
+
+**Render order is determined by z_index:**
+- Layers with `z_index < 0` render **below** entities
+- Entities render at the z=0 boundary
+- Layers with `z_index >= 0` render **above** entities (overlays)
+
+Within each group, lower z_index values render first (behind higher values).
+
+```
+z_index: -3    -2    -1    0    +1    +2
+         ↓     ↓     ↓     ↓     ↓     ↓
+       [background] [tiles] [ENTITIES] [fog] [UI overlay]
+```
+
+**Implementation:** See `UIGrid::render()` which iterates layers in z_index order, inserting entity rendering at the 0 boundary.
+
+### Chunk-Based Caching
+
+Large grids are divided into **chunks** (regions of cells). Each chunk maintains its own `sf::RenderTexture` that caches the rendered result.
+
+**Key concepts:**
+- Chunk size is implementation-defined (currently ~256 cells per dimension)
+- Only chunks intersecting the viewport are considered for rendering
+- Each chunk tracks whether its content is "dirty" (needs redraw)
+- Static content renders once, then the cached texture is reused
+
+**Implementation:** See `UIGrid::renderChunk()` for chunk texture management.
+
+### Dirty Flag Propagation
+
+When layer content changes, only affected chunks are marked dirty:
+
+1. `layer.set(x, y, value)` marks the containing chunk as dirty
+2. On next render, dirty chunks redraw to their RenderTexture
+3. Clean chunks simply blit their cached texture
+
+This means a 1000x1000 grid with one changing cell redraws only ~1 chunk, not 1,000,000 cells.
+
+**Implementation:** Dirty flags propagate through `UIGrid::markDirty()` and are checked in `UIGrid::render()`.
+
+---
+
+## Render Pipeline Stages
+
+### Stage 1: Viewport Calculation
+
+Calculate which chunks and cells are visible based on camera position, zoom, and grid dimensions.
+
+**Key properties:**
+- `center` / `center_x`, `center_y` - Camera position in pixel coordinates (within grid space)
+- `zoom` - Scale factor (1.0 = normal, 2.0 = 2x magnification)
+- `size` - Viewport dimensions in screen pixels
+
+**Implementation:** Viewport bounds calculated at start of `UIGrid::render()`.
+
+### Stage 2: Below-Entity Layers
+
+For each layer with `z_index < 0`, sorted by z_index:
+1. Determine which chunks intersect viewport
+2. For each visible chunk:
+   - If dirty: redraw layer content to chunk's RenderTexture
+   - Draw chunk's cached texture to output
+
+### Stage 3: Entity Rendering
+
+Entities render at the z=0 boundary:
+1. Iterate entity collection
+2. Cull entities outside viewport bounds
+3. Draw visible entity sprites at interpolated positions
+
+**Note:** Entity culling is currently O(n). SpatialHash optimization planned in [#115](../issues/115).
+
+### Stage 4: Above-Entity Layers
+
+For each layer with `z_index >= 0`, sorted by z_index:
+- Same chunk-based rendering as Stage 2
+- These layers appear as overlays (fog, highlights, UI elements)
+
+### Stage 5: Final Compositing
+
+All rendered content exists in the grid's output RenderTexture, which is drawn to the window in a single operation.
+
+---
+
+## Performance Characteristics
+
+**Static Grids:**
+- Near-zero CPU cost after initial render
+- Cached chunk textures reused frame-to-frame
+- Only viewport calculation and texture blitting
+
+**Dynamic Grids:**
+- Cost proportional to number of dirty chunks
+- Single-cell changes affect only one chunk
+- Bulk operations should batch changes before render
+
+**Large Grids:**
+- 1000x1000+ grids render efficiently
+- Only visible chunks processed
+- Memory scales with grid size (chunk textures)
+
+**Profiling:** Use `mcrfpy.getMetrics()` or the F3 overlay to see render times. See [[Performance-and-Profiling]].
+
+---
+
+## FOV and Perspective System (In Transition)
+
+**Current Status:** The FOV computation (`compute_fov()`, `is_in_fov()`) works correctly for pathfinding and visibility queries. However, the automatic fog-of-war overlay system is **not currently connected** to the new layer architecture.
+
+**What works:**
+- `grid.compute_fov(x, y, radius)` - Computes which cells are visible
+- `grid.is_in_fov(x, y)` - Queries visibility of a specific cell
+- Pathfinding uses walkable/transparent properties correctly
+
+**What's in transition:**
+- Per-entity perspective (`UIGridPointState`) not yet exposed to Python
+- Automatic fog overlay rendering disconnected from layer system
+- Batch FOV data access being designed ([#113 discussion](../issues/113))
+
+**Current workaround - Manual fog layer:**
+
+```python
+# Create a fog overlay layer (positive z_index = above entities)
+fog_layer = grid.add_layer("color", z_index=1)
+
+# After computing FOV, update fog colors
+grid.compute_fov(player.grid_x, player.grid_y, radius=10)
+
+for x in range(grid.grid_x):
+    for y in range(grid.grid_y):
+        if not grid.is_in_fov(x, y):
+            # Dim color for non-visible cells
+            fog_layer.set(x, y, mcrfpy.Color(0, 0, 0, 192))
+        else:
+            # Transparent for visible cells
+            fog_layer.set(x, y, mcrfpy.Color(0, 0, 0, 0))
+```
+
+**Limitation:** This O(n²) iteration is inefficient for large grids. Batch operations ([#113](../issues/113)) will address this with patterns like `cells_in_radius()` iterators.
+
+---
+
+## Layer Techniques
+
+### Multiple Overlay Layers
+
+You can pre-create multiple overlay layers and toggle between them:
+
+```python
+# Create several overlay options
+highlight_layer = grid.add_layer("color", z_index=1)
+danger_layer = grid.add_layer("color", z_index=2)
+fog_layer = grid.add_layer("color", z_index=3)
+
+# Populate each with different data...
+# highlight_layer shows selected cells
+# danger_layer shows enemy threat zones
+# fog_layer shows visibility
+
+# Toggle visibility to switch which overlay shows
+def show_danger_zones():
+    highlight_layer.visible = False
+    danger_layer.visible = True
+    fog_layer.visible = False
+
+def show_fog_of_war():
+    highlight_layer.visible = False
+    danger_layer.visible = False
+    fog_layer.visible = True
+```
+
+### Z-Index Reordering
+
+Change layer order at runtime by modifying z_index:
+
+```python
+# Bring a layer to front
+important_layer.z_index = 100
+
+# Send a layer behind entities
+background_layer.z_index = -10
+```
+
+**Note:** Changing z_index marks the layer dirty, triggering re-render of affected chunks.
+
+### Constructor `layers={}` Limitations
+
+The `layers={}` constructor argument is convenient but limited:
+
+```python
+# This creates layers, but ALL get negative z_index (below entities)
+grid = mcrfpy.Grid(
+    grid_size=(50, 50),
+    layers={"ground": "color", "terrain": "tile", "overlay": "color"}
+)
+# Result: ground=-3, terrain=-2, overlay=-1 (all below entities!)
+```
+
+**For overlays above entities, use `add_layer()` explicitly:**
+
+```python
+grid = mcrfpy.Grid(grid_size=(50, 50), layers={})
+ground = grid.add_layer("color", z_index=-2)
+terrain = grid.add_layer("tile", z_index=-1)
+overlay = grid.add_layer("color", z_index=1)  # Above entities!
+```
+
+---
+
+## Migration from Legacy API
+
+Prior to the layer system, grids had built-in per-cell rendering via `UIGridPoint` properties. Here's how to recreate that behavior:
+
+### Legacy Pattern (Pre-November 2025)
+
+```python
+# OLD API (no longer works):
+grid = mcrfpy.Grid(50, 50, 16, 16)
+cell = grid.at(x, y)
+cell.tilesprite = 42        # Sprite index
+cell.color = (255, 0, 0)    # Background color
+```
+
+### Equivalent Modern Pattern
+
+```python
+# NEW API - explicit layers:
+grid = mcrfpy.Grid(grid_size=(50, 50), pos=(0, 0), size=(400, 400))
+
+# Grid creates a default TileLayer; access it:
+tile_layer = grid.layers[0]
+tile_layer.set(x, y, 42)    # Sprite index
+
+# For background colors, add a ColorLayer behind tiles:
+color_layer = grid.add_layer("color", z_index=-2)  # Behind default tile layer
+color_layer.set(x, y, mcrfpy.Color(255, 0, 0))
+```
+
+### Recreating Old Three-Layer Rendering
+
+The legacy system rendered: background color → tile sprite → FOV overlay.
+
+```python
+# Create grid with no default layers
+grid = mcrfpy.Grid(
+    grid_size=(50, 50),
+    pos=(100, 100),
+    size=(400, 400),
+    texture=tileset,
+    layers={}
+)
+
+# Layer 1: Background colors (z=-2, behind everything)
+background = grid.add_layer("color", z_index=-2)
+
+# Layer 2: Tile sprites (z=-1, above background, below entities)
+tiles = grid.add_layer("tile", z_index=-1, texture=tileset)
+
+# Layer 3: FOV overlay (z=+1, above entities)
+fog = grid.add_layer("color", z_index=1)
+
+# Now populate layers as needed
+background.fill(mcrfpy.Color(20, 20, 30))  # Dark blue background
+
+for x in range(50):
+    for y in range(50):
+        tiles.set(x, y, calculate_tile_index(x, y))
+
+# FOV overlay updated after compute_fov() calls
+```
+
+---
+
+## Common Issues
+
+### Issue: Layer Changes Don't Appear
+
+**Cause:** Layer content changed but chunk not marked dirty.
+
+**Fix:** Use layer methods (`set()`, `fill()`) which automatically mark dirty. Direct property manipulation may bypass dirty flagging.
+
+### Issue: Overlay Appears Behind Entities
+
+**Cause:** Layer z_index is negative.
+
+**Fix:** Use `z_index >= 0` for overlays:
+```python
+overlay = grid.add_layer("color", z_index=1)  # Not z_index=-1
+```
+
+### Issue: Performance Degrades with Many Changes
+
+**Cause:** Each `set()` call can mark a chunk dirty; many scattered changes = many chunk redraws.
+
+**Fix:** Batch logically-related changes. For bulk updates, consider:
+```python
+# Less efficient: 10,000 individual calls
+for x in range(100):
+    for y in range(100):
+        layer.set(x, y, value)
+
+# More efficient when available: bulk fill
+layer.fill(mcrfpy.Color(0, 0, 0))  # Single operation
+```
+
+Batch operations API ([#113](../issues/113)) will provide more patterns.
+
+---
+
+## API Quick Reference
+
+**Grid Properties:**
+- `center`, `center_x`, `center_y` - Camera position (pixels in grid space)
+- `zoom` - Scale factor
+- `layers` - List of layer objects, sorted by z_index
+- `fill_color` - Grid background (behind all layers)
+
+**Grid Methods:**
+- `add_layer(type, z_index, texture)` - Create new layer
+- `remove_layer(layer)` - Remove a layer
+
+**Layer Properties:**
+- `z_index` - Render order
+- `visible` - Show/hide layer
+- `grid_size` - Dimensions (read-only)
+
+**Layer Methods:**
+- `set(x, y, value)` - Set cell (color or sprite index)
+- `at(x, y)` - Get cell value
+- `fill(value)` - Fill all cells
+
+---
+
+**Navigation:**
+- [[Grid-System]] - Parent page, layer concepts
+- [[Grid-TCOD-Integration]] - FOV computation details
+- [[Performance-and-Profiling]] - Metrics and optimization
+- [[Entity-Management]] - Entity rendering
+
+---
+
+*Last updated: 2025-11-29*
\ No newline at end of file