GridData SoA refactor: dense uint8 planes for walkable/transparent #332

New issue

Closed

opened 2026-07-02 12:29:39 +00:00 by john · 0 comments

john commented

2026-07-02 12:29:39 +00:00

Owner

From the 2026-07-02 memory-model review. Internal refactor -- Python only sees cells through grid.at(x,y) proxies, so the API is unchanged. This is the single highest-leverage change for both memory and numpy/AI integration.

Current storage

UIGridPoint is array-of-structs, 24 bytes/cell for 2 bytes of payload (src/UIGridPoint.h:31-54):

bool walkable, transparent (the actual data)
int grid_x, grid_y (redundant -- derivable from index)
GridData* parent_grid (back-pointer used only for TCOD sync on write)

Grids >64 in either dimension switch to chunked storage (CHUNK_THRESHOLD=64, src/GridData.h:50-59, src/GridChunk.h) -- non-contiguous, so no whole-grid zero-copy view is possible at all today.

Proposal

Store logic data as two dense row-major std::vector<uint8_t> planes (walkable, transparent) on GridData -- the same shape DiscreteMap already uses (src/DiscreteMap.h). Drop chunking for logic data entirely: a 1024x1024 grid is 2 MB as two planes; chunking only pays for render caches, which GridLayers already handles independently per-layer (src/GridLayers.h:42-45).

GridData::at(x,y) returns a proxy (the Python GridPoint wrapper already goes through getattro/setattro, src/UIGridPoint.cpp:167-256) that reads/writes the planes.
grid_x/grid_y/parent_grid disappear; the wrapper carries coordinates + grid ref itself.

Benefits

12x memory reduction on the logic layer; cache-friendly iteration for step/collision code.
Whole-grid contiguity at any size -> zero-copy numpy views become possible (blocked today by AoS + chunking).
Enables TCOD map dedup (#333, depends on this).

Risks / checks

Anything holding UIGridPoint& across calls (grep UIGridPoint&); FOV/path code reads via TCOD map so mostly unaffected.
Benchmark: apply_threshold/apply_ranges loops, full syncTCODMap, step-heavy scenes.

Related: #152 (sparse layers -- orthogonal; sparse applies to render layers, logic planes stay dense), #124.

From the 2026-07-02 memory-model review. Internal refactor -- Python only sees cells through `grid.at(x,y)` proxies, so the API is unchanged. This is the single highest-leverage change for both memory and numpy/AI integration. ## Current storage `UIGridPoint` is array-of-structs, **24 bytes/cell for 2 bytes of payload** (`src/UIGridPoint.h:31-54`): - `bool walkable, transparent` (the actual data) - `int grid_x, grid_y` (redundant -- derivable from index) - `GridData* parent_grid` (back-pointer used only for TCOD sync on write) Grids >64 in either dimension switch to chunked storage (`CHUNK_THRESHOLD=64`, `src/GridData.h:50-59`, `src/GridChunk.h`) -- **non-contiguous**, so no whole-grid zero-copy view is possible at all today. ## Proposal Store logic data as two dense row-major `std::vector<uint8_t>` planes (`walkable`, `transparent`) on GridData -- the same shape `DiscreteMap` already uses (`src/DiscreteMap.h`). Drop chunking for **logic** data entirely: a 1024x1024 grid is 2 MB as two planes; chunking only pays for render caches, which `GridLayers` already handles independently per-layer (`src/GridLayers.h:42-45`). - `GridData::at(x,y)` returns a proxy (the Python `GridPoint` wrapper already goes through getattro/setattro, `src/UIGridPoint.cpp:167-256`) that reads/writes the planes. - `grid_x/grid_y/parent_grid` disappear; the wrapper carries coordinates + grid ref itself. ## Benefits - 12x memory reduction on the logic layer; cache-friendly iteration for step/collision code. - **Whole-grid contiguity at any size** -> zero-copy numpy views become possible (blocked today by AoS + chunking). - Enables TCOD map dedup (#333, depends on this). ## Risks / checks - Anything holding `UIGridPoint&` across calls (grep `UIGridPoint&`); FOV/path code reads via TCOD map so mostly unaffected. - Benchmark: `apply_threshold`/`apply_ranges` loops, full `syncTCODMap`, step-heavy scenes. Related: #152 (sparse layers -- orthogonal; sparse applies to render layers, logic planes stay dense), #124.