GridView and Grid, FOVLayer and PathLayer #252

New issue

Closed

opened 2026-03-03 04:05:03 +00:00 by john · 4 comments

john commented 2026-03-03 04:05:03 +00:00

Owner

Copy link

7DRL finding. The requirement to size and position a Grid when it's really serving as a container of walkable/visible data, the requirement to use a single visible/walkable dataset, and the requirement to allocate the walkable/visible layers regardless of if or how they're being used, are all huge disappointments with the current API.

first, an investigation.

Then, in "complete harmony" with the HeightMap and DiscreteMap classes and libtcod's underlying walkable, visible datastructures - let's provide the current Grid experience as a GridView with a default Grid with default visible and walkable layers. Directly instantiating a Grid means shared map data, with multiple or no tcod map layers inside of it; multiple GridView instances can share the texture cache with independent camera positions.

7DRL finding. The requirement to size and position a Grid when it's really serving as a container of walkable/visible data, the requirement to use a single visible/walkable dataset, and the requirement to allocate the walkable/visible layers regardless of if or how they're being used, are all huge disappointments with the current API. first, an investigation. Then, in "complete harmony" with the `HeightMap` and `DiscreteMap` classes and libtcod's underlying walkable, visible datastructures - let's provide the current `Grid` experience as a `GridView` with a default `Grid` with default `visible` and `walkable` layers. Directly instantiating a `Grid` means shared map data, with multiple or no tcod map layers inside of it; multiple `GridView` instances can share the texture cache with independent camera positions.

john added the

Major Feature

priority:tier1-active

labels 2026-03-03 04:05:03 +00:00

john commented 2026-03-08 18:53:30 +00:00

Author

Owner

Copy link

Draft Plan: GridMap/GridView/Grid Split (#252)

Context

During 7DRL 2026, the monolithic UIGrid class proved frustrating: every grid requires positioning/sizing/zoom even when it's just a data container, walkable/transparent are hardcoded to one-each-per-grid, and there's no way to share map data between views. Issue #252 calls for splitting Grid into a data container and a rendering viewport, while modernizing how walkable/transparent/cost maps integrate with the procgen pipeline (DiscreteMap, HeightMap, BSP, Noise, WangSet).

Three-Type Design

Type	Role	Inherits
GridMap	Pure data container: coordinate space, named DiscreteMap slots, layers, entities, spatial hash, pathfinding/FOV. Not a UIDrawable.	—
GridView	Rendering viewport into a GridMap. Camera, zoom, renderTexture, perspective, cell callbacks.	UIDrawable
Grid	Compatibility shim. Constructs a GridView + auto-created GridMap. Returns a GridView.	GridView (or factory)

Grid preserves the current API: mcrfpy.Grid(grid_size=..., pos=..., size=...) works as before but the returned object is a GridView with grid.gridmap exposing the underlying GridMap. Existing code doesn't break. Power users access GridMap directly for multi-view or shared-data scenarios.

DiscreteMap as Universal Substrate

Walkable, transparent, and cost data become named DiscreteMap slots on GridMap. No new types needed. TCODMap is rebuilt lazily from named slots when FOV/pathfinding is requested.

Default slots created automatically: "walkable" (fill=1), "transparent" (fill=1).

Multiple maps per grid for different entity types:

gridmap = mcrfpy.GridMap(grid_size=(50, 50), texture=tileset)

# Default maps
gridmap.map("walkable").fill(1)
gridmap.map("walkable")[5, 3] = 0

# Custom maps for different unit types
gridmap.add_map("flying_walkable", fill=1)    # flying units
gridmap.add_map("aquatic_walkable", fill=0)   # water-only units
gridmap.add_map("darkvision_transparent", fill=1)  # x-ray vision

# Pathfinding uses named maps
path = gridmap.find_path(start, end)                              # default "walkable"
path = gridmap.find_path(start, end, walkable="flying_walkable")  # custom
path = gridmap.find_path(start, end, cost="terrain_cost")         # weighted

# FOV uses named maps
gridmap.compute_fov((25,25), radius=10)                                  # default "transparent"
gridmap.compute_fov((25,25), radius=10, transparent="darkvision_transparent")  # custom

# Procgen pipeline feeds directly into grid maps
terrain = mcrfpy.DiscreteMap.from_heightmap(hmap, ranges=...)
gridmap.set_map("terrain", terrain)
wang_set.resolve(terrain)  # already works

Python API

# === GridMap: pure data ===
gridmap = mcrfpy.GridMap(grid_size=(50, 50), texture=tileset)
gridmap.map("walkable")                # DiscreteMap view (zero-copy)
gridmap.add_map("cost", fill=1)        # named map
gridmap.at(5, 5).walkable = True       # backwards-compat proxy
gridmap.entities                       # EntityCollection
gridmap.layers                         # TileLayer/ColorLayer list
gridmap.children                       # UIDrawables in grid-world coords
gridmap.compute_fov(...)               # FOV on data
gridmap.find_path(...)                 # pathfinding on data

# === GridView: rendering viewport (IS a UIDrawable) ===
view = mcrfpy.GridView(gridmap=gridmap, pos=(0,0), size=(800,600))
view.zoom = 2.0
view.center_camera((25, 25))
view.perspective = player_entity
view.gridmap                           # the GridMap reference
scene.children.append(view)

# Multiple views of same gridmap
minimap = mcrfpy.GridView(gridmap=gridmap, pos=(600,0), size=(200,200))
minimap.zoom = 0.25
scene.children.append(minimap)

# === Grid: compatibility shim (returns GridView) ===
# Current API preserved — creates GridView + auto-created GridMap
grid = mcrfpy.Grid(grid_size=(50,50), pos=(0,0), size=(800,600), texture=tileset)
# grid IS a GridView. grid.gridmap IS a GridMap.
grid.zoom = 2.0                        # GridView property
grid.gridmap.map("walkable")[5,3] = 0  # GridMap through view
grid.at(5,5).walkable                  # delegates to gridmap
grid.compute_fov(...)                  # delegates to gridmap
scene.children.append(grid)            # works — it's a GridView

# Entity perspective with custom maps
entity.perspective_walkable = "flying_walkable"       # which walkable map to use
entity.perspective_transparent = "darkvision_transparent"  # which FOV map to use

Architecture

What Goes Where

Member	Current (UIGrid)	New Location	Notes
grid_w, grid_h	UIGrid	GridMap	Dimensions
ptex (texture)	UIGrid	GridMap	Determines cell size
tcod_map	UIGrid	GridMap (lazy cache)	Rebuilt from DiscreteMap slots
points[] / chunk_manager	UIGrid	Removed	Replaced by named DiscreteMap slots
entities	UIGrid	GridMap	Entity list + spatial hash
spatial_hash	UIGrid	GridMap	O(1) entity queries
children	UIGrid	GridMap	UIDrawables in grid-world coords (shared across views)
layers	UIGrid	GridMap	TileLayer/ColorLayer (data)
dijkstra_maps	UIGrid	GridMap	Cached pathfinding
renderTexture	UIGrid	GridView	Render pipeline
rotationTexture	UIGrid	GridView	Camera rotation
box, fill_color	UIGrid	GridView	Screen display
center_x/y, zoom	UIGrid	GridView	Camera state
camera_rotation	UIGrid	GridView	Rotation angle
perspective_entity/enabled	UIGrid	GridView	FOV overlay
on_cell_* callbacks	UIGrid	GridView	View-specific interaction
screenToCell()	UIGrid	GridView	Coordinate conversion
UIDrawable inheritance	UIGrid	GridView	pos, size, visible, etc.

Layer Chunk Caches: Shared, Not Per-View

Chunk rendering depends on layer data + cell dimensions (from GridMap's texture), NOT on camera position. Camera transform is applied AFTER chunks are drawn. So layers keep their existing chunk cache architecture — multiple GridViews rendering the same GridMap reuse the same chunk textures. No per-view cache needed.

The only change: GridLayer::parent_grid changes from UIGrid* to GridMap*.

GridPoint Becomes a Proxy

UIGridPoint loses its bool walkable, transparent members. Instead, gridmap.at(x,y) returns a lightweight proxy that reads/writes the GridMap's named DiscreteMap slots:

// gridmap.at(5, 3).walkable = True  ->  gridmap.map("walkable")[5, 3] = 1
// Already has dynamic layer access via getattro/setattro (#150)

The proxy stores (GridMap*, x, y) instead of owning data.

DiscreteMap Zero-Copy Binding

Add an owner field to PyDiscreteMapObject so GridMap slots can be exposed as DiscreteMap views without copying:

typedef struct {
    PyObject_HEAD
    uint8_t* values;
    int w, h;
    PyObject* enum_type;
    PyObject* owner;       // NEW: if non-NULL, don't free values in dealloc
} PyDiscreteMapObject;

GridMap stores raw uint8_t* arrays for its map slots. gridmap.map("walkable") returns a DiscreteMap that wraps the GridMap's internal array. Mutations trigger a dirty flag for lazy TCODMap rebuild.

Cost-Weighted Pathfinding

Use libtcod's ITCODPathCallback to read from a DiscreteMap cost field:

class DiscreteMapCostCallback : public ITCODPathCallback {
    const uint8_t* walkable_data;
    const uint8_t* cost_data;  // optional
    int width;
    float getWalkCost(int xFrom, int yFrom, int xTo, int yTo, void*) const override {
        int idx = xTo + yTo * width;
        if (!walkable_data[idx]) return 0.0f;
        return cost_data ? (float)cost_data[idx] : 1.0f;
    }
};

Entity Changes

UIEntity::grid changes from shared_ptr<UIGrid> to shared_ptr<GridMap>. Entity is already conceptualized as "a perspective on Grid data" (see UIEntity.h comment block). The gridstate (visible/discovered per cell) stays on Entity.

Entity gains optional per-entity map preferences:

• perspective_walkable — name of walkable map for this entity's pathfinding
• perspective_transparent — name of transparent map for this entity's FOV

Perspective System

Currently duplicated: ColorLayer has perspective binding AND UIGrid has perspective_entity. In the new design, perspective is purely a GridView concern. GridView's render() applies FOV overlay based on its perspective_entity, using that entity's perspective_transparent map for FOV computation. ColorLayer's perspective methods (apply_perspective, update_perspective) are deprecated in favor of view-level perspective.

Grid Compatibility Shim

mcrfpy.Grid becomes either:

• A subclass of GridView whose __init__ accepts the combined parameter set (grid_size + pos + size + texture + ...) and auto-creates a GridMap, OR
• A factory function that returns a plain GridView

Subclass approach is cleaner: isinstance(obj, GridView) is True, existing code that checks type still works, and Grid can override __init__ to accept the legacy parameter signature while delegating to GridView.

// Grid's __init__ creates a GridMap, then calls GridView.__init__ with it
// Grid(grid_size=..., pos=..., size=..., texture=...)
//   -> self.gridmap = GridMap(grid_size, texture)
//   -> GridView.__init__(self, gridmap=self.gridmap, pos=pos, size=size)

Methods like grid.compute_fov(), grid.find_path(), grid.at(), grid.map(), grid.entities, grid.layers delegate to self.gridmap.

Files

New Files

File	Purpose
src/GridMap.h	GridMap class — data container
src/GridMap.cpp	Implementation (map slots, layers, entities, pathfinding)
src/PyGridMap.h	Python type mcrfpy.GridMap
src/PyGridMap.cpp	Python bindings for GridMap
src/GridView.h	GridView class — UIDrawable viewport
src/GridView.cpp	Render pipeline (extracted from UIGrid.cpp)
src/PyGridView.h	Python type mcrfpy.GridView + mcrfpy.Grid shim
src/PyGridView.cpp	Python bindings for GridView and Grid

Modified Files

File	Change
src/PyDiscreteMap.h/cpp	Add owner field, dirty callback
src/UIEntity.h/cpp	shared_ptr<UIGrid> -> shared_ptr<GridMap>, add perspective map names
src/UIEntityCollection.h/cpp	Grid reference type change
src/UIGridPoint.h/cpp	Remove bool members, become proxy into GridMap maps
src/GridLayers.h/cpp	UIGrid* -> GridMap* parent reference
src/UIGridPathfinding.h/cpp	Accept named map params, add cost callback
src/SpatialHash.h/cpp	No change (stays with GridMap)
src/McRFPy_API.cpp	Register GridMap + GridView + Grid types, remove old UIGrid
src/UIDrawable.h	Add UIGRIDVIEW to PyObjectsEnum
src/Scene.h/cpp	UIGRID -> UIGRIDVIEW for rendering dispatch
CMakeLists.txt	Add new source files

Removed Files

File	Replaced By
src/UIGrid.h	src/GridMap.h + src/GridView.h
src/UIGrid.cpp	src/GridMap.cpp + src/GridView.cpp
src/GridChunk.h/cpp	GridPoint storage replaced by DiscreteMap slots

Implementation Phases

Phase 0: Foundation (non-breaking)

1. Add owner field to PyDiscreteMapObject (view support, dirty callback)
2. All existing DiscreteMap tests must pass unchanged

Phase 1: GridMap Class

3. Create GridMap with: dimensions, named map slots (default "walkable"/"transparent"), layer management, entity list + spatial hash, children collection
4. Default map slots: walkable (fill=1), transparent (fill=1)
5. Python bindings: mcrfpy.GridMap — constructor takes grid_size, texture
6. Methods: at(), map(), add_map(), set_map(), remove_map(), add_layer(), remove_layer(), layer()

Phase 2: Pathfinding on GridMap

7. Move FOV/pathfinding from UIGrid to GridMap
8. Lazy TCODMap cache: rebuild from walkable+transparent DiscreteMap slots when dirty
9. Add walkable=, transparent=, cost= named-map parameters to compute_fov(), find_path(), get_dijkstra_map()
10. Implement DiscreteMapCostCallback for weighted pathfinding

Phase 3: GridView Class

11. Create GridView inheriting UIDrawable, referencing shared_ptr<GridMap>
12. Move render pipeline from UIGrid: renderTexture, camera, zoom, rotation, fill_color
13. Move cell callbacks, perspective system, screenToCell
14. gridmap property exposes the underlying GridMap
15. Python type: mcrfpy.GridView with all current Grid rendering properties

Phase 4: Grid Compatibility Shim

16. mcrfpy.Grid as subclass of GridView
17. Constructor accepts combined params: grid_size + pos + size + texture + all current kwargs
18. Auto-creates GridMap internally, delegates data methods to self.gridmap
19. Forwarding properties/methods: at(), map(), compute_fov(), find_path(), entities, layers, children
20. All existing tests pass WITHOUT modification (this is the key validation)

Phase 5: Entity + Layer Migration

21. Change UIEntity::grid from shared_ptr<UIGrid> to shared_ptr<GridMap>
22. Add perspective_walkable, perspective_transparent string properties to Entity
23. Change GridLayer::parent_grid from UIGrid* to GridMap*
24. Update UIEntityCollection references
25. Refactor UIGridPoint to proxy into GridMap map slots

Phase 6: Cleanup

26. Remove UIGrid.h/cpp (all functionality now in GridMap + GridView)
27. Remove GridChunk.h/cpp (GridPoint storage replaced by DiscreteMap slots)
28. Update Scene rendering: UIGRIDVIEW replaces UIGRID
29. Update McRFPy_API type registration

Verification

Unit Tests (new)

• GridMap: map("walkable") returns writable DiscreteMap, modifications reflected in pathfinding
• GridMap: add_map("custom") creates slot, usable in find_path(walkable="custom")
• GridMap: at(x,y).walkable proxy reads/writes walkable DiscreteMap slot
• GridView(gridmap=...) renders correctly
• Two GridViews sharing one GridMap show same data with different cameras
• Cost-weighted pathfinding: cells with cost=3 are avoided vs cost=1
• Custom transparent map in compute_fov() produces different FOV
• Entity perspective_walkable/perspective_transparent used by perspective system
• Animations on GridView (center_x, zoom) work

Backwards Compatibility (critical)

• All existing tests pass unchanged via Grid compatibility shim
• mcrfpy.Grid(grid_size=..., pos=..., size=...) returns working GridView
• grid.at(x,y).walkable, grid.compute_fov(), grid.find_path() all work
• grid.entities, grid.layers, grid.children all work
• scene.children.append(grid) works (Grid IS-A GridView IS-A UIDrawable)

Integration Tests

• Full procgen pipeline: Noise -> HeightMap -> DiscreteMap -> gridmap.set_map -> WangSet -> TileLayer
• Multiple entities with different walkable maps on same grid
• Minimap + main view sharing one gridmap
• Perspective rendering with per-entity transparent maps

Manual Verification

• make && cd build && ./mcrogueface --headless --exec ../tests/unit/grid_view_test.py
• Run existing test suite: cd tests && python3 run_tests.py

Risks and Concerns

This is a complex architectural change which has the potential to undermine a huge amount of memory safety progress that McRogueFace has made since the plan was drafted. See these make options:

# Debug/sanitizer build options
option(MCRF_SANITIZE_ADDRESS "Build with AddressSanitizer" OFF)
option(MCRF_SANITIZE_UNDEFINED "Build with UBSan" OFF)
option(MCRF_SANITIZE_THREAD "Build with ThreadSanitizer" OFF)
option(MCRF_DEBUG_PYTHON "Link against debug CPython from __lib_debug/" OFF)

While rearchitecting, we should perform sanitization and debug builds incrementally. Waiting until the end to perform memory safety checks isn't a good strategy - every compilation attempt during the architecture overhaul is also a good opportunity to find memory errors, including instrumentation with libtcod, since the grid system is so deeply connected to that library.

I considered the plan above basically ready to implement, but instead I started working on instrumentation to find memory safety issues. I might execute this plan in conjunction with some RAII templates or macros to encourage memory and refcount safety in all of McRogueFace's C++ to Python interactions.

# Draft Plan: GridMap/GridView/Grid Split (#252) ## Context During 7DRL 2026, the monolithic `UIGrid` class proved frustrating: every grid requires positioning/sizing/zoom even when it's just a data container, walkable/transparent are hardcoded to one-each-per-grid, and there's no way to share map data between views. Issue #252 calls for splitting Grid into a data container and a rendering viewport, while modernizing how walkable/transparent/cost maps integrate with the procgen pipeline (DiscreteMap, HeightMap, BSP, Noise, WangSet). ## Three-Type Design | Type | Role | Inherits | |------|------|----------| | **`GridMap`** | Pure data container: coordinate space, named DiscreteMap slots, layers, entities, spatial hash, pathfinding/FOV. Not a UIDrawable. | — | | **`GridView`** | Rendering viewport into a GridMap. Camera, zoom, renderTexture, perspective, cell callbacks. | UIDrawable | | **`Grid`** | Compatibility shim. Constructs a GridView + auto-created GridMap. Returns a GridView. | GridView (or factory) | `Grid` preserves the current API: `mcrfpy.Grid(grid_size=..., pos=..., size=...)` works as before but the returned object is a GridView with `grid.gridmap` exposing the underlying GridMap. Existing code doesn't break. Power users access GridMap directly for multi-view or shared-data scenarios. ## DiscreteMap as Universal Substrate Walkable, transparent, and cost data become **named DiscreteMap slots** on GridMap. No new types needed. TCODMap is rebuilt lazily from named slots when FOV/pathfinding is requested. Default slots created automatically: `"walkable"` (fill=1), `"transparent"` (fill=1). Multiple maps per grid for different entity types: ```python gridmap = mcrfpy.GridMap(grid_size=(50, 50), texture=tileset) # Default maps gridmap.map("walkable").fill(1) gridmap.map("walkable")[5, 3] = 0 # Custom maps for different unit types gridmap.add_map("flying_walkable", fill=1) # flying units gridmap.add_map("aquatic_walkable", fill=0) # water-only units gridmap.add_map("darkvision_transparent", fill=1) # x-ray vision # Pathfinding uses named maps path = gridmap.find_path(start, end) # default "walkable" path = gridmap.find_path(start, end, walkable="flying_walkable") # custom path = gridmap.find_path(start, end, cost="terrain_cost") # weighted # FOV uses named maps gridmap.compute_fov((25,25), radius=10) # default "transparent" gridmap.compute_fov((25,25), radius=10, transparent="darkvision_transparent") # custom # Procgen pipeline feeds directly into grid maps terrain = mcrfpy.DiscreteMap.from_heightmap(hmap, ranges=...) gridmap.set_map("terrain", terrain) wang_set.resolve(terrain) # already works ``` ## Python API ```python # === GridMap: pure data === gridmap = mcrfpy.GridMap(grid_size=(50, 50), texture=tileset) gridmap.map("walkable") # DiscreteMap view (zero-copy) gridmap.add_map("cost", fill=1) # named map gridmap.at(5, 5).walkable = True # backwards-compat proxy gridmap.entities # EntityCollection gridmap.layers # TileLayer/ColorLayer list gridmap.children # UIDrawables in grid-world coords gridmap.compute_fov(...) # FOV on data gridmap.find_path(...) # pathfinding on data # === GridView: rendering viewport (IS a UIDrawable) === view = mcrfpy.GridView(gridmap=gridmap, pos=(0,0), size=(800,600)) view.zoom = 2.0 view.center_camera((25, 25)) view.perspective = player_entity view.gridmap # the GridMap reference scene.children.append(view) # Multiple views of same gridmap minimap = mcrfpy.GridView(gridmap=gridmap, pos=(600,0), size=(200,200)) minimap.zoom = 0.25 scene.children.append(minimap) # === Grid: compatibility shim (returns GridView) === # Current API preserved — creates GridView + auto-created GridMap grid = mcrfpy.Grid(grid_size=(50,50), pos=(0,0), size=(800,600), texture=tileset) # grid IS a GridView. grid.gridmap IS a GridMap. grid.zoom = 2.0 # GridView property grid.gridmap.map("walkable")[5,3] = 0 # GridMap through view grid.at(5,5).walkable # delegates to gridmap grid.compute_fov(...) # delegates to gridmap scene.children.append(grid) # works — it's a GridView # Entity perspective with custom maps entity.perspective_walkable = "flying_walkable" # which walkable map to use entity.perspective_transparent = "darkvision_transparent" # which FOV map to use ``` ## Architecture ### What Goes Where | Member | Current (UIGrid) | New Location | Notes | |--------|-------------------|-------------|-------| | `grid_w, grid_h` | UIGrid | GridMap | Dimensions | | `ptex` (texture) | UIGrid | GridMap | Determines cell size | | `tcod_map` | UIGrid | GridMap (lazy cache) | Rebuilt from DiscreteMap slots | | `points[]` / `chunk_manager` | UIGrid | **Removed** | Replaced by named DiscreteMap slots | | `entities` | UIGrid | GridMap | Entity list + spatial hash | | `spatial_hash` | UIGrid | GridMap | O(1) entity queries | | `children` | UIGrid | GridMap | UIDrawables in grid-world coords (shared across views) | | `layers` | UIGrid | GridMap | TileLayer/ColorLayer (data) | | `dijkstra_maps` | UIGrid | GridMap | Cached pathfinding | | `renderTexture` | UIGrid | GridView | Render pipeline | | `rotationTexture` | UIGrid | GridView | Camera rotation | | `box, fill_color` | UIGrid | GridView | Screen display | | `center_x/y, zoom` | UIGrid | GridView | Camera state | | `camera_rotation` | UIGrid | GridView | Rotation angle | | `perspective_entity/enabled` | UIGrid | GridView | FOV overlay | | `on_cell_*` callbacks | UIGrid | GridView | View-specific interaction | | `screenToCell()` | UIGrid | GridView | Coordinate conversion | | UIDrawable inheritance | UIGrid | GridView | pos, size, visible, etc. | ### Layer Chunk Caches: Shared, Not Per-View Chunk rendering depends on layer data + cell dimensions (from GridMap's texture), NOT on camera position. Camera transform is applied AFTER chunks are drawn. So **layers keep their existing chunk cache architecture** — multiple GridViews rendering the same GridMap reuse the same chunk textures. No per-view cache needed. The only change: `GridLayer::parent_grid` changes from `UIGrid*` to `GridMap*`. ### GridPoint Becomes a Proxy `UIGridPoint` loses its `bool walkable, transparent` members. Instead, `gridmap.at(x,y)` returns a lightweight proxy that reads/writes the GridMap's named DiscreteMap slots: ```cpp // gridmap.at(5, 3).walkable = True -> gridmap.map("walkable")[5, 3] = 1 // Already has dynamic layer access via getattro/setattro (#150) ``` The proxy stores `(GridMap*, x, y)` instead of owning data. ### DiscreteMap Zero-Copy Binding Add an `owner` field to `PyDiscreteMapObject` so GridMap slots can be exposed as DiscreteMap views without copying: ```cpp typedef struct { PyObject_HEAD uint8_t* values; int w, h; PyObject* enum_type; PyObject* owner; // NEW: if non-NULL, don't free values in dealloc } PyDiscreteMapObject; ``` GridMap stores raw `uint8_t*` arrays for its map slots. `gridmap.map("walkable")` returns a DiscreteMap that wraps the GridMap's internal array. Mutations trigger a dirty flag for lazy TCODMap rebuild. ### Cost-Weighted Pathfinding Use libtcod's `ITCODPathCallback` to read from a DiscreteMap cost field: ```cpp class DiscreteMapCostCallback : public ITCODPathCallback { const uint8_t* walkable_data; const uint8_t* cost_data; // optional int width; float getWalkCost(int xFrom, int yFrom, int xTo, int yTo, void*) const override { int idx = xTo + yTo * width; if (!walkable_data[idx]) return 0.0f; return cost_data ? (float)cost_data[idx] : 1.0f; } }; ``` ### Entity Changes `UIEntity::grid` changes from `shared_ptr<UIGrid>` to `shared_ptr<GridMap>`. Entity is already conceptualized as "a perspective on Grid data" (see UIEntity.h comment block). The gridstate (visible/discovered per cell) stays on Entity. Entity gains optional per-entity map preferences: - `perspective_walkable` — name of walkable map for this entity's pathfinding - `perspective_transparent` — name of transparent map for this entity's FOV ### Perspective System Currently duplicated: ColorLayer has perspective binding AND UIGrid has perspective_entity. In the new design, perspective is purely a **GridView** concern. GridView's `render()` applies FOV overlay based on its `perspective_entity`, using that entity's `perspective_transparent` map for FOV computation. ColorLayer's perspective methods (`apply_perspective`, `update_perspective`) are deprecated in favor of view-level perspective. ### Grid Compatibility Shim `mcrfpy.Grid` becomes either: - A **subclass of GridView** whose `__init__` accepts the combined parameter set (grid_size + pos + size + texture + ...) and auto-creates a GridMap, OR - A **factory function** that returns a plain GridView Subclass approach is cleaner: `isinstance(obj, GridView)` is True, existing code that checks type still works, and Grid can override `__init__` to accept the legacy parameter signature while delegating to GridView. ```cpp // Grid's __init__ creates a GridMap, then calls GridView.__init__ with it // Grid(grid_size=..., pos=..., size=..., texture=...) // -> self.gridmap = GridMap(grid_size, texture) // -> GridView.__init__(self, gridmap=self.gridmap, pos=pos, size=size) ``` Methods like `grid.compute_fov()`, `grid.find_path()`, `grid.at()`, `grid.map()`, `grid.entities`, `grid.layers` delegate to `self.gridmap`. ## Files ### New Files | File | Purpose | |------|---------| | `src/GridMap.h` | `GridMap` class — data container | | `src/GridMap.cpp` | Implementation (map slots, layers, entities, pathfinding) | | `src/PyGridMap.h` | Python type `mcrfpy.GridMap` | | `src/PyGridMap.cpp` | Python bindings for GridMap | | `src/GridView.h` | `GridView` class — UIDrawable viewport | | `src/GridView.cpp` | Render pipeline (extracted from UIGrid.cpp) | | `src/PyGridView.h` | Python type `mcrfpy.GridView` + `mcrfpy.Grid` shim | | `src/PyGridView.cpp` | Python bindings for GridView and Grid | ### Modified Files | File | Change | |------|--------| | `src/PyDiscreteMap.h/cpp` | Add `owner` field, dirty callback | | `src/UIEntity.h/cpp` | `shared_ptr<UIGrid>` -> `shared_ptr<GridMap>`, add perspective map names | | `src/UIEntityCollection.h/cpp` | Grid reference type change | | `src/UIGridPoint.h/cpp` | Remove bool members, become proxy into GridMap maps | | `src/GridLayers.h/cpp` | `UIGrid*` -> `GridMap*` parent reference | | `src/UIGridPathfinding.h/cpp` | Accept named map params, add cost callback | | `src/SpatialHash.h/cpp` | No change (stays with GridMap) | | `src/McRFPy_API.cpp` | Register GridMap + GridView + Grid types, remove old UIGrid | | `src/UIDrawable.h` | Add `UIGRIDVIEW` to PyObjectsEnum | | `src/Scene.h/cpp` | UIGRID -> UIGRIDVIEW for rendering dispatch | | `CMakeLists.txt` | Add new source files | ### Removed Files | File | Replaced By | |------|-------------| | `src/UIGrid.h` | `src/GridMap.h` + `src/GridView.h` | | `src/UIGrid.cpp` | `src/GridMap.cpp` + `src/GridView.cpp` | | `src/GridChunk.h/cpp` | GridPoint storage replaced by DiscreteMap slots | ## Implementation Phases ### Phase 0: Foundation (non-breaking) 1. Add `owner` field to PyDiscreteMapObject (view support, dirty callback) 2. All existing DiscreteMap tests must pass unchanged ### Phase 1: GridMap Class 3. Create `GridMap` with: dimensions, named map slots (default "walkable"/"transparent"), layer management, entity list + spatial hash, children collection 4. Default map slots: `walkable` (fill=1), `transparent` (fill=1) 5. Python bindings: `mcrfpy.GridMap` — constructor takes `grid_size`, `texture` 6. Methods: `at()`, `map()`, `add_map()`, `set_map()`, `remove_map()`, `add_layer()`, `remove_layer()`, `layer()` ### Phase 2: Pathfinding on GridMap 7. Move FOV/pathfinding from UIGrid to GridMap 8. Lazy TCODMap cache: rebuild from walkable+transparent DiscreteMap slots when dirty 9. Add `walkable=`, `transparent=`, `cost=` named-map parameters to `compute_fov()`, `find_path()`, `get_dijkstra_map()` 10. Implement `DiscreteMapCostCallback` for weighted pathfinding ### Phase 3: GridView Class 11. Create `GridView` inheriting UIDrawable, referencing `shared_ptr<GridMap>` 12. Move render pipeline from UIGrid: renderTexture, camera, zoom, rotation, fill_color 13. Move cell callbacks, perspective system, screenToCell 14. `gridmap` property exposes the underlying GridMap 15. Python type: `mcrfpy.GridView` with all current Grid rendering properties ### Phase 4: Grid Compatibility Shim 16. `mcrfpy.Grid` as subclass of GridView 17. Constructor accepts combined params: `grid_size` + `pos` + `size` + `texture` + all current kwargs 18. Auto-creates GridMap internally, delegates data methods to `self.gridmap` 19. Forwarding properties/methods: `at()`, `map()`, `compute_fov()`, `find_path()`, `entities`, `layers`, `children` 20. All existing tests pass WITHOUT modification (this is the key validation) ### Phase 5: Entity + Layer Migration 21. Change `UIEntity::grid` from `shared_ptr<UIGrid>` to `shared_ptr<GridMap>` 22. Add `perspective_walkable`, `perspective_transparent` string properties to Entity 23. Change `GridLayer::parent_grid` from `UIGrid*` to `GridMap*` 24. Update UIEntityCollection references 25. Refactor UIGridPoint to proxy into GridMap map slots ### Phase 6: Cleanup 26. Remove UIGrid.h/cpp (all functionality now in GridMap + GridView) 27. Remove GridChunk.h/cpp (GridPoint storage replaced by DiscreteMap slots) 28. Update Scene rendering: UIGRIDVIEW replaces UIGRID 29. Update McRFPy_API type registration ## Verification ### Unit Tests (new) - `GridMap`: `map("walkable")` returns writable DiscreteMap, modifications reflected in pathfinding - `GridMap`: `add_map("custom")` creates slot, usable in `find_path(walkable="custom")` - `GridMap`: `at(x,y).walkable` proxy reads/writes walkable DiscreteMap slot - `GridView(gridmap=...)` renders correctly - Two `GridView`s sharing one `GridMap` show same data with different cameras - Cost-weighted pathfinding: cells with cost=3 are avoided vs cost=1 - Custom transparent map in `compute_fov()` produces different FOV - Entity `perspective_walkable`/`perspective_transparent` used by perspective system - Animations on GridView (center_x, zoom) work ### Backwards Compatibility (critical) - **All existing tests pass unchanged** via Grid compatibility shim - `mcrfpy.Grid(grid_size=..., pos=..., size=...)` returns working GridView - `grid.at(x,y).walkable`, `grid.compute_fov()`, `grid.find_path()` all work - `grid.entities`, `grid.layers`, `grid.children` all work - `scene.children.append(grid)` works (Grid IS-A GridView IS-A UIDrawable) ### Integration Tests - Full procgen pipeline: Noise -> HeightMap -> DiscreteMap -> gridmap.set_map -> WangSet -> TileLayer - Multiple entities with different walkable maps on same grid - Minimap + main view sharing one gridmap - Perspective rendering with per-entity transparent maps ### Manual Verification - `make && cd build && ./mcrogueface --headless --exec ../tests/unit/grid_view_test.py` - Run existing test suite: `cd tests && python3 run_tests.py` ## Risks and Concerns This is a complex architectural change which has the potential to undermine a huge amount of memory safety progress that McRogueFace has made since the plan was drafted. See these make options: ``` # Debug/sanitizer build options option(MCRF_SANITIZE_ADDRESS "Build with AddressSanitizer" OFF) option(MCRF_SANITIZE_UNDEFINED "Build with UBSan" OFF) option(MCRF_SANITIZE_THREAD "Build with ThreadSanitizer" OFF) option(MCRF_DEBUG_PYTHON "Link against debug CPython from __lib_debug/" OFF) ``` While rearchitecting, we should perform sanitization and debug builds incrementally. Waiting until the end to perform memory safety checks isn't a good strategy - every compilation attempt during the architecture overhaul is also a good opportunity to find memory errors, including instrumentation with libtcod, since the grid system is so deeply connected to that library. I considered the plan above basically ready to implement, but instead I started working on instrumentation to find memory safety issues. I might execute this plan in conjunction with some RAII templates or macros to encourage memory and refcount safety in all of McRogueFace's C++ to Python interactions.

john commented 2026-03-08 20:18:29 +00:00

Author

Owner

Copy link

7DRL Findings: Additional scope for this overhaul

Post-jam analysis identified several grid/entity pain points that belong under this issue rather than as standalone work:

Entity Gridstate Lifecycle (Items 1, 5, 6 from jam feedback)

Problem: ensureGridstate() silently destroys exploration data on grid resize, retains stale data for same-size grids, and there's no save/load mechanism. entity.gridstate returns detached SimpleNamespace copies while entity.at() returns live proxies — no batch operations exist, forcing O(w*h) Python iteration for save/restore.

Resolution under #252: With the Grid/GridView split, gridstate data lives on the Grid (data container). Entity perspective becomes a DiscreteMap reference on the entity, with convenience methods for swapping/overwriting when entities change grids. Game developers who want per-grid memory caching can manage DiscreteMap instances in Python — the engine provides the primitives (correctly-sized DiscreteMap creation, assignment with validation, automatic disabling on grid departure) rather than a built-in cache. See planned issue for Entity.gridstate as DiscreteMap.

FOV Query / Overlay Decoupling (Items 2, 3 from jam feedback)

Problem: grid.perspective = entity conflates two independent concerns: (1) FOV spatial queries (is_in_fov) and (2) per-frame alpha overlay rendering. When using ColorLayer fog exclusively, you can't get FOV queries without the overlay. When both systems are active, alpha values stack (ColorLayer α=150 + perspective overlay α=192 = double-darkening).

Resolution under #252: The FOVLayer concept resolves this architecturally — FOV visualization becomes an explicit, optional layer rather than an implicit overlay baked into grid.perspective. The Grid owns FOV computation data (which entity is the perspective source); the GridView controls whether/how it renders.

## 7DRL Findings: Additional scope for this overhaul Post-jam analysis identified several grid/entity pain points that belong under this issue rather than as standalone work: ### Entity Gridstate Lifecycle (Items 1, 5, 6 from jam feedback) **Problem:** `ensureGridstate()` silently destroys exploration data on grid resize, retains stale data for same-size grids, and there's no save/load mechanism. `entity.gridstate` returns detached SimpleNamespace copies while `entity.at()` returns live proxies — no batch operations exist, forcing O(w*h) Python iteration for save/restore. **Resolution under #252:** With the Grid/GridView split, gridstate data lives on the Grid (data container). Entity perspective becomes a DiscreteMap reference on the entity, with convenience methods for swapping/overwriting when entities change grids. Game developers who want per-grid memory caching can manage DiscreteMap instances in Python — the engine provides the primitives (correctly-sized DiscreteMap creation, assignment with validation, automatic disabling on grid departure) rather than a built-in cache. See planned issue for `Entity.gridstate` as DiscreteMap. ### FOV Query / Overlay Decoupling (Items 2, 3 from jam feedback) **Problem:** `grid.perspective = entity` conflates two independent concerns: (1) FOV spatial queries (`is_in_fov`) and (2) per-frame alpha overlay rendering. When using ColorLayer fog exclusively, you can't get FOV queries without the overlay. When both systems are active, alpha values stack (ColorLayer α=150 + perspective overlay α=192 = double-darkening). **Resolution under #252:** The `FOVLayer` concept resolves this architecturally — FOV visualization becomes an explicit, optional layer rather than an implicit overlay baked into grid.perspective. The Grid owns FOV computation data (which entity is the perspective source); the GridView controls whether/how it renders.

john referenced this issue 2026-03-08 20:18:41 +00:00

Deduplicate redundant FOV computation #292

john referenced this issue 2026-03-08 20:18:54 +00:00

DiscreteMap serialization via bytes #293

john referenced this issue 2026-03-08 20:19:11 +00:00

Entity.gridstate as DiscreteMap reference #294

john added a new dependency 2026-03-08 21:07:33 +00:00

#270 [Bugfix] GridLayer::parent_grid is a dangling raw pointer when grid is destroyed

john added a new dependency 2026-03-08 21:07:33 +00:00

#271 [Bugfix] UIGridPoint::parent_grid is a dangling raw pointer

john added a new dependency 2026-03-08 21:07:34 +00:00

#277 [Bugfix] GridChunk::parent_grid raw pointer can dangle

john referenced this issue 2026-03-08 21:07:50 +00:00

[Bugfix] GridLayer::parent_grid is a dangling raw pointer when grid is destroyed #270

john referenced this issue 2026-03-08 21:07:54 +00:00

[Bugfix] UIGridPoint::parent_grid is a dangling raw pointer #271

john referenced this issue 2026-03-08 21:08:00 +00:00

[Bugfix] GridChunk::parent_grid raw pointer can dangle #277

john referenced this issue 2026-03-15 00:30:04 +00:00

Entity cell_pos — integer logical position decoupled from render position #295

john referenced this issue 2026-03-15 00:32:02 +00:00

Pathfinding with entity collision labels #302

john commented 2026-03-15 20:17:53 +00:00

Author

Owner

Copy link

Roadmap context

This issue is part of the Grid & Entity Overhaul Roadmap (docs/GRID_ENTITY_OVERHAUL_ROADMAP.md), placed in Phase 4 (Grid Architecture).

Phase 4 is sequenced after the behavior system (Phases 2-3) so the data model is validated before the architecture shift. The behavior system (grid.step(), cell_pos, etc.) lives on the data Grid, not the renderer, so it survives the split with minimal rework.

Dangling pointer fixes #270, #271, #277 are folded into this issue — converting raw UIGrid* to weak_ptr<Grid> during the split.

Related new issues: #295 (cell_pos), #296 (labels), #297-#303 (behavior system). See roadmap for full dependency graph."

## Roadmap context This issue is part of the Grid & Entity Overhaul Roadmap (`docs/GRID_ENTITY_OVERHAUL_ROADMAP.md`), placed in **Phase 4** (Grid Architecture). Phase 4 is sequenced after the behavior system (Phases 2-3) so the data model is validated before the architecture shift. The behavior system (`grid.step()`, `cell_pos`, etc.) lives on the data Grid, not the renderer, so it survives the split with minimal rework. **Dangling pointer fixes** #270, #271, #277 are folded into this issue — converting raw `UIGrid*` to `weak_ptr<Grid>` during the split. **Related new issues**: #295 (cell_pos), #296 (labels), #297-#303 (behavior system). See roadmap for full dependency graph."

john referenced this issue 2026-03-15 20:17:56 +00:00

Entity.gridstate as DiscreteMap reference #294

john referenced this issue 2026-03-15 20:18:09 +00:00

[Bugfix] GridLayer::parent_grid is a dangling raw pointer when grid is destroyed #270

john referenced this issue 2026-03-15 20:18:10 +00:00

[Bugfix] UIGridPoint::parent_grid is a dangling raw pointer #271

john referenced this issue 2026-03-15 20:18:12 +00:00

[Bugfix] GridChunk::parent_grid raw pointer can dangle #277

john referenced this issue 2026-03-15 20:18:45 +00:00

Pathfinding with entity collision labels #302

john referenced this issue from a commit 2026-03-16 22:23:55 +00:00

Phase 4.1: Extract GridData base class from UIGrid (#252, #270, #271, #277)

john referenced this issue from a commit 2026-04-01 15:27:59 +00:00

Phase 4.3: Grid auto-creates GridView with rendering property sync

john referenced this issue from a commit 2026-04-04 08:34:18 +00:00

Grid/GridView API unification: mcrfpy.Grid now returns GridView, closes #252

john closed this issue 2026-04-04 08:34:18 +00:00

john commented 2026-04-04 08:34:50 +00:00

Author

Owner

Copy link

Grid/GridView API Unification — Implemented

Commit 109bc21 implements the core unification. mcrfpy.Grid() now returns a GridView that internally owns a GridData (UIGrid). The old UIGrid type is renamed to _GridData (internal).

What changed

10 files, +612/-142 lines:

• UIGridView.h/cpp: Two-mode init (factory vs explicit view), tp_getattro/tp_setattro delegation to underlying UIGrid, PythonObjectCache registration, animation property support (center, zoom, shader.*)
• UIGrid.h: tp_name changed to mcrfpy._GridData (internal)
• GridData.h: Added owning_view weak_ptr back-reference for Entity.grid reverse lookup
• McRFPy_API.cpp: Grid moved from exported_types to internal_types; GridView alias added
• UIEntity.cpp: grid getter returns GridView wrapper; setter accepts GridView; find_path uses internal type directly
• UIDrawable.cpp: removeFromParent() handles UIGRIDVIEW (accesses children via grid_data, not invalid static_cast); ~12 switch cases added for UIGRIDVIEW
• GridLayers.cpp: ColorLayer_set_grid and TileLayer_set_grid extract UIGrid from GridView correctly
• UIFrame.cpp: children init accepts GridView objects
• PyAnimation.cpp: UIGRIDVIEW case for animation targets

Test results

263/263 tests pass (100%). Several bugs were found and fixed during testing:

• removeFromParent() was doing static_pointer_cast<UIGrid> on a GridView (segfault)
• find_path was fetching mcrfpy.Grid (now GridView) but casting to PyUIGridObject (segfault)
• Layer set_grid was casting Grid (now GridView) to PyUIGridObject (garbage grid dimensions)
• GridView needed center as Vector2f animation property, and shader.* property support

What's NOT in this commit

• Multi-view support (future: multiple GridViews sharing one GridData)
• Dangling pointer fixes (#270, #271, #277) — separate issues

## Grid/GridView API Unification — Implemented Commit `109bc21` implements the core unification. `mcrfpy.Grid()` now returns a `GridView` that internally owns a `GridData` (UIGrid). The old `UIGrid` type is renamed to `_GridData` (internal). ### What changed **10 files, +612/-142 lines:** - **UIGridView.h/cpp**: Two-mode init (factory vs explicit view), `tp_getattro`/`tp_setattro` delegation to underlying UIGrid, PythonObjectCache registration, animation property support (center, zoom, shader.*) - **UIGrid.h**: `tp_name` changed to `mcrfpy._GridData` (internal) - **GridData.h**: Added `owning_view` weak_ptr back-reference for Entity.grid reverse lookup - **McRFPy_API.cpp**: Grid moved from exported_types to internal_types; GridView alias added - **UIEntity.cpp**: `grid` getter returns GridView wrapper; setter accepts GridView; `find_path` uses internal type directly - **UIDrawable.cpp**: `removeFromParent()` handles UIGRIDVIEW (accesses children via grid_data, not invalid static_cast); ~12 switch cases added for UIGRIDVIEW - **GridLayers.cpp**: `ColorLayer_set_grid` and `TileLayer_set_grid` extract UIGrid from GridView correctly - **UIFrame.cpp**: children init accepts GridView objects - **PyAnimation.cpp**: UIGRIDVIEW case for animation targets ### Test results 263/263 tests pass (100%). Several bugs were found and fixed during testing: - `removeFromParent()` was doing `static_pointer_cast<UIGrid>` on a GridView (segfault) - `find_path` was fetching `mcrfpy.Grid` (now GridView) but casting to `PyUIGridObject` (segfault) - Layer `set_grid` was casting Grid (now GridView) to `PyUIGridObject` (garbage grid dimensions) - GridView needed `center` as Vector2f animation property, and `shader.*` property support ### What's NOT in this commit - Multi-view support (future: multiple GridViews sharing one GridData) - Dangling pointer fixes (#270, #271, #277) — separate issues

john referenced this issue 2026-04-17 03:25:40 +00:00

[Refactoring] Migrate UIEntity::grid from shared_ptr&lt;UIGrid&gt; to shared_ptr&lt;GridData&gt; #313

john referenced this issue from a commit 2026-04-18 11:34:00 +00:00

Phase 5.3: documentation regeneration + introspection-based stub generator

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

master

threedee

emscripten-mcrogueface

rogueliketutorial25

alpha_presentable

alpha_streamline_2

alpha_streamline_1

interpreter_mode

grid_entity_integration

reprs_and_member_names

break_up_ui_h

standardize_font_handling

standardize_vector_handling

standardize_color_handling

standardize_texture_handling

raii_pyobjects

rebase-for-7DRL_Feb-2024

engjam_uicollection

engjam_ui_overhaul

0.2.3-prerelease-7drl2026-emscripten

Labels

Clear labels   

Alpha Release Requirement

This issue is a blocker to 0.1 Alpha release of McRogueFace.

  

Bugfix

  

Demo Target

Functionality to demonstrate; Depends on new features, but this issue isn't for writing them.

  

Documentation

  

Major Feature

Significant time and effort required.

  

Minor Feature

Some effort required to create or overhaul functionality.

  

priority:tier1-active

Current development focus - critical path to v1.0

  

priority:tier2-foundation

Important foundation work - not blocking v1.0

  

priority:tier3-future

Future features - deferred until after v1.0

  

priority:tier4-deferred

  

Refactoring & Cleanup

no new functionality, just improving the codebase.

  

system:animation

Animation and property interpolation

  

system:documentation

Documentation infrastructure

  

system:grid

Grid system and spatial containers

  

system:input

Input handling and events

  

system:performance

Performance optimization and profiling

  

system:procgen

Procedural Generation (Noise, BSP, Heightmap)

  

system:python-binding

Python/C++ binding layer

  

system:rendering

Rendering pipeline and visuals

  

system:ui-hierarchy

UI component hierarchy and composition

  

Tiny Feature

quick and easy - a few lines or with little interconnection around the codebase

  

workflow:blocked

Blocked by other work - waiting on dependencies

  

workflow:needs-benchmark

Needs performance testing and benchmarks

  

workflow:needs-documentation

Needs documentation before or after implementation

No labels

Alpha Release Requirement

Bugfix

Demo Target

Documentation

Major Feature

Minor Feature

priority:tier1-active

priority:tier2-foundation

priority:tier3-future

priority:tier4-deferred

Refactoring & Cleanup

system:animation

system:documentation

system:grid

system:input

system:performance

system:procgen

system:python-binding

system:rendering

system:ui-hierarchy

Tiny Feature

workflow:blocked

workflow:needs-benchmark

workflow:needs-documentation

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks

#270 [Bugfix] GridLayer::parent_grid is a dangling raw pointer when grid is destroyed

john/McRogueFace

#271 [Bugfix] UIGridPoint::parent_grid is a dangling raw pointer

john/McRogueFace

#277 [Bugfix] GridChunk::parent_grid raw pointer can dangle

john/McRogueFace

Reference

john/McRogueFace#252

No description provided.