numpy availability strategy: bundle, side-venv, or scientific package variant #337

New issue

Open

opened 2026-07-02 12:29:42 +00:00 by john · 1 comment

john commented

2026-07-02 12:29:42 +00:00

Owner

From the 2026-07-02 memory-model review. The buffer-protocol issues (#334, #335) make mcrfpy numpy-ready; this issue decides how users actually get numpy.

Current state

Embedded 3.14 ships stdlib only: __lib/Python/Lib/site-packages/ contains just a README; __lib/Python/lib.linux-x86_64-3.14/ has no _multiarray_umath.
src/scripts/mcrf_venv.py:146-233 already pip-installs into a separate venv and wires it up -- numpy is literally in its example strings.

Options

Document the side-venv as the supported path (cheapest; works today; desktop-only).
Bundle numpy in the full package variant: ~+20 MB on a ~26 MB package; must match the embedded interpreter ABI (cp314, plus cp314t if free-threaded builds ship -- matrix doubles).
New package-*-scientific preset: full stdlib + numpy, keeping light/full unchanged. Fits the existing tools/stdlib_modules.yaml packaging structure.
#70 route (mcrfpy against system Python): makes pip install numpy trivially work, but is its own large project.

WASM: numpy-in-pyodide is a separate project, explicitly out of scope here.

Recommendation to evaluate first: (1) now + (3) when #334/#335 land, so 'McRogueFace for agents/RL' (#55, #154, #156) is a one-download story.

Note: python-tcod's ecosystem is numpy-native; matching that convention makes McRogueFace legible to the existing roguelike-dev community.

From the 2026-07-02 memory-model review. The buffer-protocol issues (#334, #335) make mcrfpy *numpy-ready*; this issue decides how users actually get numpy. ## Current state - Embedded 3.14 ships stdlib only: `__lib/Python/Lib/site-packages/` contains just a README; `__lib/Python/lib.linux-x86_64-3.14/` has no `_multiarray_umath`. - `src/scripts/mcrf_venv.py:146-233` already pip-installs into a **separate venv** and wires it up -- numpy is literally in its example strings. ## Options 1. **Document the side-venv as the supported path** (cheapest; works today; desktop-only). 2. **Bundle numpy** in the full package variant: ~+20 MB on a ~26 MB package; must match the embedded interpreter ABI (cp314, plus cp314t if free-threaded builds ship -- matrix doubles). 3. **New `package-*-scientific` preset**: full stdlib + numpy, keeping light/full unchanged. Fits the existing `tools/stdlib_modules.yaml` packaging structure. 4. **#70 route** (mcrfpy against system Python): makes `pip install numpy` trivially work, but is its own large project. WASM: numpy-in-pyodide is a separate project, explicitly out of scope here. Recommendation to evaluate first: (1) now + (3) when #334/#335 land, so 'McRogueFace for agents/RL' (#55, #154, #156) is a one-download story. Note: python-tcod's ecosystem is numpy-native; matching that convention makes McRogueFace legible to the existing roguelike-dev community.

john added the

priority:tier3-future

system:python-binding

Minor Feature

labels

2026-07-02 12:29:42 +00:00

john referenced this issue

2026-07-02 12:30:43 +00:00

Free-threading (3.14t) hardening: Py_mod_gil, UIEntity::pyobject race, unlocked registries #336

john added

priority:tier2-foundation

and removed

priority:tier3-future

labels

2026-07-11 15:17:33 +00:00

john commented

2026-07-11 15:18:34 +00:00

Author

Owner

Refinement (2026-07-11): unblocked, bumped to tier2, framed as agent-env enabler

Unblocked. The buffer-protocol prerequisite shipped 2026-07-11: #334 (DiscreteMap buffer protocol) and #335 (ColorLayer/TileLayer zero-copy numpy views via with layer.edit() as view:). mcrfpy is now buffer-protocol-ready; this issue is the only thing standing between that and users actually consuming the views with real numpy. Bumped tier3 → tier2.

Why it matters now (destination). This is enabling background work for the agent environment: #54 (Jupyter), #55 (Wumpus sim), and #154/#156 (multi-agent LLM testbed). VLM/RL perception pipelines and Jupyter users expect numpy-native arrays, and matching python-tcod's numpy-native convention makes the environment legible to the existing roguelike-dev community.

New task — numpy round-trip validation. The sprint validated the buffer views through memoryview only (numpy isn't in the build). Before we claim "numpy-ready," add a test/demo under the side-venv (src/scripts/mcrf_venv.py) proving np.asarray/np.frombuffer round-trips (a) a ColorLayer.edit() view and (b) a DiscreteMap — write through numpy, read back the mutation in-engine.

Delivery recommendation (reaffirmed). (1) Document the side-venv as the supported path now + (3) a package-*-scientific preset later. Keep numpy out of the core package — reinforced by the free-threading coupling: a bundled numpy must ship cp314t wheels if 3.14t ever lands (#336), doubling the interpreter matrix. #70 (mcrfpy as wheels against system Python) is the eventual cleanest delivery and makes pip install numpy trivial.

## Refinement (2026-07-11): unblocked, bumped to tier2, framed as agent-env enabler **Unblocked.** The buffer-protocol prerequisite shipped 2026-07-11: #334 (DiscreteMap buffer protocol) and #335 (ColorLayer/TileLayer zero-copy numpy views via `with layer.edit() as view:`). mcrfpy is now buffer-protocol-ready; this issue is the only thing standing between that and users actually consuming the views with real numpy. Bumped tier3 → **tier2**. **Why it matters now (destination).** This is enabling background work for the **agent environment**: #54 (Jupyter), #55 (Wumpus sim), and #154/#156 (multi-agent LLM testbed). VLM/RL perception pipelines and Jupyter users expect numpy-native arrays, and matching python-tcod's numpy-native convention makes the environment legible to the existing roguelike-dev community. **New task — numpy round-trip validation.** The sprint validated the buffer views through `memoryview` only (numpy isn't in the build). Before we claim "numpy-ready," add a test/demo under the side-venv (`src/scripts/mcrf_venv.py`) proving `np.asarray`/`np.frombuffer` round-trips (a) a `ColorLayer.edit()` view and (b) a `DiscreteMap` — write through numpy, read back the mutation in-engine. **Delivery recommendation (reaffirmed).** (1) Document the side-venv as the supported path **now** + (3) a `package-*-scientific` preset later. Keep numpy **out** of the core package — reinforced by the free-threading coupling: a bundled numpy must ship cp314t wheels if 3.14t ever lands (#336), doubling the interpreter matrix. #70 (mcrfpy as wheels against system Python) is the eventual cleanest delivery and makes `pip install numpy` trivial.

john referenced this issue

2026-07-11 15:18:49 +00:00

get_metrics() draw_calls / ui_elements / render counters read 0 in windowed builds #341

john referenced this issue

2026-07-11 15:18:55 +00:00

Jupyter Notebook Interface #54

john referenced this issue

2026-07-16 16:47:18 +00:00

[Epic] Web deployment parity & divergence-testing sprint #387

john referenced this issue

2026-07-16 16:48:06 +00:00

[Docs] Promote the Web Build Constraints table into an audited parity matrix with test-coverage column #391