[1.0 Decision] Bulk-edit convention for future array/buffer APIs (edit() context manager vs explicit sync) #328

New issue

Closed

opened 2026-07-02 12:29:37 +00:00 by john · 1 comment

john commented

2026-07-02 12:29:37 +00:00

Owner

From the 2026-07-02 pre-1.0 memory-model review. The buffer-protocol APIs themselves are additive (post-1.0-safe), but the idiom should be chosen once, before apply_threshold/fill_rect/future numpy views accumulate three inconsistent conventions.

Problem

Zero-copy views (memoryview / numpy) let Python write cell data while bypassing the setters that maintain invariants:

per-cell markDirty(x,y) -> chunk render-texture invalidation (src/GridLayers.h:42-79)
walkable/transparent -> single-cell TCOD sync + transparency_generation bump + fov_dirty (src/GridData.cpp:133-140, src/UIGridPoint.cpp:90)

The one existing zero-copy export, DiscreteMap.mask() (src/PyDiscreteMap.cpp:1315-1317), dodges this only because DiscreteMap has no dirty flags.

Options

Context manager: with layer.edit() as view: -- yields a writable view, marks affected state dirty (conservatively: whole layer / full TCOD resync) on __exit__. Scoped, explicit, hard to misuse.
Explicit sync: raw views are always available (layer.view / grid.walkable_view); after writing, the user must call layer.mark_dirty() / grid.sync(). More flexible, easier to forget.

Per the project's fail-early principle, no automatic change detection -- pick an explicit contract.

Blocks

Buffer protocol for ColorLayer/TileLayer (#335).
Any future grid.walkable array view (depends on GridData SoA refactor, #332).

From the 2026-07-02 pre-1.0 memory-model review. The buffer-protocol APIs themselves are additive (post-1.0-safe), but the **idiom should be chosen once**, before `apply_threshold`/`fill_rect`/future numpy views accumulate three inconsistent conventions. ## Problem Zero-copy views (memoryview / numpy) let Python write cell data while bypassing the setters that maintain invariants: - per-cell `markDirty(x,y)` -> chunk render-texture invalidation (`src/GridLayers.h:42-79`) - walkable/transparent -> single-cell TCOD sync + `transparency_generation` bump + `fov_dirty` (`src/GridData.cpp:133-140`, `src/UIGridPoint.cpp:90`) The one existing zero-copy export, `DiscreteMap.mask()` (`src/PyDiscreteMap.cpp:1315-1317`), dodges this only because DiscreteMap has no dirty flags. ## Options 1. **Context manager**: `with layer.edit() as view:` -- yields a writable view, marks affected state dirty (conservatively: whole layer / full TCOD resync) on `__exit__`. Scoped, explicit, hard to misuse. 2. **Explicit sync**: raw views are always available (`layer.view` / `grid.walkable_view`); after writing, the user must call `layer.mark_dirty()` / `grid.sync()`. More flexible, easier to forget. Per the project's fail-early principle, **no automatic change detection** -- pick an explicit contract. ## Blocks - Buffer protocol for ColorLayer/TileLayer (#335). - Any future `grid.walkable` array view (depends on GridData SoA refactor, #332).

john added the

Minor Feature

system:python-binding

system:grid

priority:tier1-active

labels

2026-07-02 12:29:37 +00:00

john referenced this issue

2026-07-02 12:30:42 +00:00

Buffer protocol for DiscreteMap (np.asarray zero-copy) #334

john referenced this issue

2026-07-02 12:30:43 +00:00

Buffer protocol / numpy views for ColorLayer and TileLayer #335

john added a new dependency

2026-07-02 12:30:44 +00:00

#335 Buffer protocol / numpy views for ColorLayer and TileLayer

john commented

2026-07-03 00:04:08 +00:00

Author

Owner

DECIDED 2026-07-02: Option 1 — edit() context manager is THE bulk-edit convention.

with layer.edit() as view: yields the writable view; __exit__ conservatively invalidates (whole-layer markDirty / full TCOD resync + generation bump as applicable). Forgetting to sync is impossible by construction, which is the fail-early principle applied to the API surface. Read-only zero-copy views (like DiscreteMap.mask()) may still be exposed without ceremony.

No always-writable raw view + mark_dirty() primitives will be shipped; if a real need for persistent writable views emerges post-1.0, primitives can be added additively without breaking the edit() idiom.

This unblocks #335 (ColorLayer/TileLayer buffers) and sets the convention #334 and any future grid.walkable view (#332) must follow. Convention will be recorded in the compatibility/docs pass (#327/#330/#326), which closes this issue.

**DECIDED 2026-07-02: Option 1 — `edit()` context manager is THE bulk-edit convention.** `with layer.edit() as view:` yields the writable view; `__exit__` conservatively invalidates (whole-layer markDirty / full TCOD resync + generation bump as applicable). Forgetting to sync is impossible by construction, which is the fail-early principle applied to the API surface. Read-only zero-copy views (like `DiscreteMap.mask()`) may still be exposed without ceremony. No always-writable raw view + `mark_dirty()` primitives will be shipped; if a real need for persistent writable views emerges post-1.0, primitives can be added additively without breaking the `edit()` idiom. This unblocks #335 (ColorLayer/TileLayer buffers) and sets the convention #334 and any future `grid.walkable` view (#332) must follow. Convention will be recorded in the compatibility/docs pass (#327/#330/#326), which closes this issue.

john referenced this issue from a commit

2026-07-03 03:45:58 +00:00

Freeze value semantics + record 1.0 compatibility policy

john closed this issue

2026-07-03 03:45:58 +00:00

john referenced this issue from a commit

2026-07-11 14:42:07 +00:00

feat(grid): numpy buffer views for ColorLayer/TileLayer via layer.edit(); closes #335