Compare commits

..

4 commits

Author SHA1 Message Date
a7ada7d65b distribution packaging 2026-01-09 12:00:59 -05:00
e6fa62f35d set version string for 7DRL2026 prerelease 2026-01-09 07:01:29 -05:00
ed85ccdf33 update to cpython 3.14.2 2026-01-09 07:00:15 -05:00
08c7c797a3 asset cleanup 2026-01-08 22:52:34 -05:00
31 changed files with 1017 additions and 82 deletions

3
.gitignore vendored
View file

@ -17,6 +17,8 @@ Makefile
*.zip
__lib/
__lib_windows/
build-windows/
build_windows/
_oldscripts/
assets/
cellular_automata_fire/
@ -28,6 +30,7 @@ scripts/
tcod_reference
.archive
.mcp.json
dist/
# Keep important documentation and tests
!CLAUDE.md

141
CLAUDE.md
View file

@ -175,26 +175,121 @@ The project uses a structured label system to organize issues:
20=blocked, 21=needs-benchmark, 22=needs-documentation
```
## Build Commands
## Build System
McRogueFace uses a unified Makefile for both Linux native builds and Windows cross-compilation.
**IMPORTANT**: All `make` commands must be run from the **project root directory** (`/home/john/Development/McRogueFace/`), not from `build/` or any subdirectory.
### Quick Reference
```bash
# Build the project (compiles to ./build directory)
make
# Linux builds
make # Build for Linux (default target)
make linux # Same as above
make run # Build and run
make clean # Remove Linux build artifacts
# Or use the build script directly
./build.sh
# Windows cross-compilation (requires MinGW-w64)
make windows # Release build for Windows
make windows-debug # Debug build with console output
make clean-windows # Remove Windows build artifacts
# Run the game
make run
# Distribution packages
make package-linux-light # Linux with minimal stdlib (~25 MB)
make package-linux-full # Linux with full stdlib (~26 MB)
make package-windows-light # Windows with minimal stdlib
make package-windows-full # Windows with full stdlib
make package-all # All platform/preset combinations
# Clean build artifacts
make clean
# The executable and all assets are in ./build/
cd build
./mcrogueface
# Cleanup
make clean-all # Remove all builds and packages
make clean-dist # Remove only distribution packages
```
### Build Outputs
| Command | Output Directory | Executable |
|---------|------------------|------------|
| `make` / `make linux` | `build/` | `build/mcrogueface` |
| `make windows` | `build-windows/` | `build-windows/mcrogueface.exe` |
| `make windows-debug` | `build-windows-debug/` | `build-windows-debug/mcrogueface.exe` |
| `make package-*` | `dist/` | `.tar.gz` or `.zip` archives |
### Prerequisites
**Linux build:**
- CMake 3.14+
- GCC/G++ with C++17 support
- SFML 2.6 development libraries
- Libraries in `__lib/` directory (libpython3.14, libtcod, etc.)
**Windows cross-compilation:**
- MinGW-w64 (`x86_64-w64-mingw32-g++-posix`)
- Libraries in `__lib_windows/` directory
- Toolchain file: `cmake/toolchains/mingw-w64-x86_64.cmake`
### Library Dependencies
The build expects pre-built libraries in:
- `__lib/` - Linux shared libraries (libpython3.14.so, libsfml-*.so, libtcod.so)
- `__lib/Python/Lib/` - Python standard library source
- `__lib/Python/lib.linux-x86_64-3.14/` - Python extension modules (.so)
- `__lib_windows/` - Windows DLLs and libraries
### Manual CMake Build
If you need more control over the build:
```bash
# Linux
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
# Windows cross-compile
mkdir build-windows && cd build-windows
cmake .. -DCMAKE_TOOLCHAIN_FILE=../cmake/toolchains/mingw-w64-x86_64.cmake \
-DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
# Windows debug with console
cmake .. -DCMAKE_TOOLCHAIN_FILE=../cmake/toolchains/mingw-w64-x86_64.cmake \
-DCMAKE_BUILD_TYPE=Debug \
-DMCRF_WINDOWS_CONSOLE=ON
```
### Distribution Packaging
The packaging system creates self-contained archives with:
- Executable
- Required shared libraries
- Assets (sprites, fonts, audio)
- Python scripts
- Filtered Python stdlib (light or full variant)
**Light variant** (~25 MB): Core + gamedev + utility modules only
**Full variant** (~26 MB): Includes networking, async, debugging modules
Packaging tools:
- `tools/package.sh` - Main packaging orchestrator
- `tools/package_stdlib.py` - Creates filtered stdlib archives
- `tools/stdlib_modules.yaml` - Module categorization config
### Troubleshooting
**"No rule to make target 'linux'"**: You're in the wrong directory. Run `make` from project root.
**Library linking errors**: Ensure `__lib/` contains all required .so files. Check `CMakeLists.txt` for `link_directories(${CMAKE_SOURCE_DIR}/__lib)`.
**Windows build fails**: Verify MinGW-w64 is installed with posix thread model: `x86_64-w64-mingw32-g++-posix --version`
### Legacy Build Scripts
The following are deprecated but kept for reference:
- `build.sh` - Original Linux build script (use `make` instead)
- `GNUmakefile.legacy` - Old wrapper makefile (renamed to avoid conflicts)
## Project Architecture
McRogueFace is a C++ game engine with Python scripting support, designed for creating roguelike games. The architecture consists of:
@ -311,20 +406,14 @@ cd build
## Common Development Tasks
### Compiling McRogueFace
See the [Build System](#build-system) section above for comprehensive build instructions.
```bash
# Standard build (to ./build directory)
make
# Full rebuild
make clean && make
# Manual CMake build
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
# The library path issue: if linking fails, check that libraries are in __lib/
# CMakeLists.txt expects: link_directories(${CMAKE_SOURCE_DIR}/__lib)
# Quick reference (run from project root!)
make # Linux build
make windows # Windows cross-compile
make clean && make # Full rebuild
```
### Running and Capturing Output

View file

@ -1,54 +0,0 @@
# Convenience Makefile wrapper for McRogueFace
# This delegates to CMake build in the build directory
.PHONY: all build clean run test dist help
# Default target
all: build
# Build the project
build:
@./build.sh
# Clean build artifacts
clean:
@./clean.sh
# Run the game
run: build
@cd build && ./mcrogueface
# Run in Python mode
python: build
@cd build && ./mcrogueface -i
# Test basic functionality
test: build
@echo "Testing McRogueFace..."
@cd build && ./mcrogueface -V
@cd build && ./mcrogueface -c "print('Test passed')"
@cd build && ./mcrogueface --headless -c "import mcrfpy; print('mcrfpy imported successfully')"
# Create distribution archive
dist: build
@echo "Creating distribution archive..."
@cd build && zip -r ../McRogueFace-$$(date +%Y%m%d).zip . -x "*.o" "CMakeFiles/*" "Makefile" "*.cmake"
@echo "Distribution archive created: McRogueFace-$$(date +%Y%m%d).zip"
# Show help
help:
@echo "McRogueFace Build System"
@echo "======================="
@echo ""
@echo "Available targets:"
@echo " make - Build the project (default)"
@echo " make build - Build the project"
@echo " make clean - Remove all build artifacts"
@echo " make run - Build and run the game"
@echo " make python - Build and run in Python interactive mode"
@echo " make test - Run basic tests"
@echo " make dist - Create distribution archive"
@echo " make help - Show this help message"
@echo ""
@echo "Build output goes to: ./build/"
@echo "Distribution archives are created in project root"

Binary file not shown.

Before

Width:  |  Height:  |  Size: 181 KiB

Binary file not shown.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.8 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 202 KiB

Binary file not shown.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 33 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 674 KiB

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 KiB

@ -1 +1 @@
Subproject commit ebf955df7a89ed0c7968f79faec1de49f61ed7cb
Subproject commit df793163d5821791d4e7caf88885a2c11a107986

View file

@ -1,4 +1,4 @@
#pragma once
// McRogueFace version string (#164)
#define MCRFPY_VERSION "1.0.0"
#define MCRFPY_VERSION "0.2.0-prerelease-7drl2026"

354
tools/package.sh Executable file
View file

@ -0,0 +1,354 @@
#!/bin/bash
#
# McRogueFace Distribution Packager
#
# Creates clean distribution packages for Windows and Linux
# Supports light (minimal stdlib) and full (complete stdlib) variants
#
# Usage:
# ./tools/package.sh windows light # Windows light build
# ./tools/package.sh windows full # Windows full build
# ./tools/package.sh linux light # Linux light build
# ./tools/package.sh linux full # Linux full build
# ./tools/package.sh all # All variants
#
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
DIST_DIR="$PROJECT_ROOT/dist"
# Version from git or default
VERSION=$(cd "$PROJECT_ROOT" && git describe --tags --always 2>/dev/null || echo "dev")
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
log_info() {
echo -e "${GREEN}[INFO]${NC} $1"
}
log_warn() {
echo -e "${YELLOW}[WARN]${NC} $1"
}
log_error() {
echo -e "${RED}[ERROR]${NC} $1"
}
# Files and directories to exclude from distribution
EXCLUDE_PATTERNS=(
"CMakeFiles"
"CMakeCache.txt"
"cmake_install.cmake"
"Makefile"
"*.o"
"*.obj"
".git"
".gitignore"
"__pycache__"
"*.pyc"
"*.pyo"
)
# Platform-specific excludes
WINDOWS_EXCLUDES=(
"*.so"
"*.so.*"
"lib.linux-*"
)
LINUX_EXCLUDES=(
"*.dll"
"*.exe"
"*.pdb"
)
check_build_exists() {
local platform=$1
local build_dir
if [ "$platform" = "windows" ]; then
build_dir="$PROJECT_ROOT/build-windows"
if [ ! -f "$build_dir/mcrogueface.exe" ]; then
log_error "Windows build not found. Run 'make windows' first."
return 1
fi
else
build_dir="$PROJECT_ROOT/build"
if [ ! -f "$build_dir/mcrogueface" ]; then
log_error "Linux build not found. Run 'make' first."
return 1
fi
fi
return 0
}
create_stdlib_zip() {
local platform=$1
local preset=$2
local output_dir=$3
log_info "Creating stdlib zip: platform=$platform preset=$preset"
# Use Python 3 to run our stdlib packager
python3 "$SCRIPT_DIR/package_stdlib.py" \
--platform "$platform" \
--preset "$preset" \
--output "$output_dir"
}
package_windows() {
local preset=$1
local build_dir="$PROJECT_ROOT/build-windows"
local package_name="McRogueFace-${VERSION}-Windows-${preset}"
local package_dir="$DIST_DIR/$package_name"
log_info "Packaging Windows ($preset): $package_name"
# Check build exists
check_build_exists windows || return 1
# Clean and create package directory
rm -rf "$package_dir"
mkdir -p "$package_dir"
# Copy executable
cp "$build_dir/mcrogueface.exe" "$package_dir/"
# Copy DLLs (excluding build artifacts)
for dll in "$build_dir"/*.dll; do
[ -f "$dll" ] && cp "$dll" "$package_dir/"
done
# Copy assets
if [ -d "$build_dir/assets" ]; then
cp -r "$build_dir/assets" "$package_dir/"
fi
# Copy scripts
if [ -d "$build_dir/scripts" ]; then
cp -r "$build_dir/scripts" "$package_dir/"
fi
# Copy Python stdlib directory structure (same as Linux)
# Python home is set to <exe>/lib/Python, so stdlib must be there
mkdir -p "$package_dir/lib/Python"
# Copy the Lib directory from build (matches Linux structure)
if [ -d "$build_dir/lib/Python/Lib" ]; then
log_info "Copying Python stdlib (preset: $preset)"
cp -r "$build_dir/lib/Python/Lib" "$package_dir/lib/Python/"
# Remove test directories and other excludes to save space
rm -rf "$package_dir/lib/Python/Lib/test"
rm -rf "$package_dir/lib/Python/Lib/tests"
rm -rf "$package_dir/lib/Python/Lib/idlelib"
rm -rf "$package_dir/lib/Python/Lib/tkinter"
rm -rf "$package_dir/lib/Python/Lib/turtledemo"
rm -rf "$package_dir/lib/Python/Lib/pydoc_data"
rm -rf "$package_dir/lib/Python/Lib/lib2to3"
rm -rf "$package_dir/lib/Python/Lib/ensurepip"
rm -rf "$package_dir/lib/Python/Lib/_pyrepl"
find "$package_dir/lib/Python/Lib" -name "__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true
find "$package_dir/lib/Python/Lib" -name "*.pyc" -delete 2>/dev/null || true
find "$package_dir/lib/Python/Lib" -name "test_*.py" -delete 2>/dev/null || true
find "$package_dir/lib/Python/Lib" -name "*_test.py" -delete 2>/dev/null || true
fi
# Also copy python314.zip for backwards compatibility (some deployments use it)
if [ -f "$build_dir/python314.zip" ]; then
cp "$build_dir/python314.zip" "$package_dir/"
fi
# Create the distribution archive
log_info "Creating archive: ${package_name}.zip"
(cd "$DIST_DIR" && zip -r "${package_name}.zip" "$package_name")
# Report size
local size=$(du -h "$DIST_DIR/${package_name}.zip" | cut -f1)
log_info "Created: $DIST_DIR/${package_name}.zip ($size)"
# Cleanup uncompressed directory
rm -rf "$package_dir"
}
package_linux() {
local preset=$1
local build_dir="$PROJECT_ROOT/build"
local package_name="McRogueFace-${VERSION}-Linux-${preset}"
local package_dir="$DIST_DIR/$package_name"
log_info "Packaging Linux ($preset): $package_name"
# Check build exists
check_build_exists linux || return 1
# Clean and create package directory
rm -rf "$package_dir"
mkdir -p "$package_dir"
mkdir -p "$package_dir/lib"
# Copy executable
cp "$build_dir/mcrogueface" "$package_dir/"
# Copy shared libraries from __lib (not from build dir which has artifacts)
if [ -d "$PROJECT_ROOT/__lib" ]; then
# Copy only essential runtime libraries (not test modules)
# Core libraries: libpython, libsfml-*, libtcod
for lib in libpython3.14.so.1.0 libsfml-graphics.so.2.6.1 libsfml-window.so.2.6.1 \
libsfml-system.so.2.6.1 libsfml-audio.so.2.6.1 libtcod.so; do
[ -f "$PROJECT_ROOT/__lib/$lib" ] && cp "$PROJECT_ROOT/__lib/$lib" "$package_dir/lib/"
done
# Create necessary symlinks
(cd "$package_dir/lib" && \
ln -sf libpython3.14.so.1.0 libpython3.14.so && \
ln -sf libsfml-graphics.so.2.6.1 libsfml-graphics.so.2.6 && \
ln -sf libsfml-graphics.so.2.6.1 libsfml-graphics.so && \
ln -sf libsfml-window.so.2.6.1 libsfml-window.so.2.6 && \
ln -sf libsfml-window.so.2.6.1 libsfml-window.so && \
ln -sf libsfml-system.so.2.6.1 libsfml-system.so.2.6 && \
ln -sf libsfml-system.so.2.6.1 libsfml-system.so && \
ln -sf libsfml-audio.so.2.6.1 libsfml-audio.so.2.6 && \
ln -sf libsfml-audio.so.2.6.1 libsfml-audio.so)
# Copy Python extension modules to correct location (excluding test modules)
# Must match structure: lib/Python/lib.linux-x86_64-3.14/
local pylib_dir="$PROJECT_ROOT/__lib/Python/lib.linux-x86_64-3.14"
if [ -d "$pylib_dir" ]; then
mkdir -p "$package_dir/lib/Python/lib.linux-x86_64-3.14"
for so in "$pylib_dir"/*.so; do
local basename=$(basename "$so")
# Skip test modules
case "$basename" in
*test*|xxlimited*|_ctypes_test*|_xxtestfuzz*)
continue
;;
*)
cp "$so" "$package_dir/lib/Python/lib.linux-x86_64-3.14/"
;;
esac
done
fi
fi
# Copy assets
if [ -d "$build_dir/assets" ]; then
cp -r "$build_dir/assets" "$package_dir/"
fi
# Copy scripts
if [ -d "$build_dir/scripts" ]; then
cp -r "$build_dir/scripts" "$package_dir/"
fi
# Copy Python stdlib directory
# Python home is set to <exe>/lib/Python, stdlib must be in lib/Python/Lib/
mkdir -p "$package_dir/lib/Python"
# Copy the Lib directory from __lib/Python/Lib (filtered by preset)
if [ -d "$PROJECT_ROOT/__lib/Python/Lib" ]; then
log_info "Copying Python stdlib (preset: $preset)"
# For now, copy entire Lib - filtering happens via package_stdlib.py for zip
# TODO: Implement directory-based filtering for light preset
cp -r "$PROJECT_ROOT/__lib/Python/Lib" "$package_dir/lib/Python/"
# Remove test directories and other excludes to save space
rm -rf "$package_dir/lib/Python/Lib/test"
rm -rf "$package_dir/lib/Python/Lib/tests"
rm -rf "$package_dir/lib/Python/Lib/idlelib"
rm -rf "$package_dir/lib/Python/Lib/tkinter"
rm -rf "$package_dir/lib/Python/Lib/turtledemo"
rm -rf "$package_dir/lib/Python/Lib/pydoc_data"
rm -rf "$package_dir/lib/Python/Lib/lib2to3"
rm -rf "$package_dir/lib/Python/Lib/ensurepip"
rm -rf "$package_dir/lib/Python/Lib/_pyrepl"
find "$package_dir/lib/Python/Lib" -name "__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true
find "$package_dir/lib/Python/Lib" -name "*.pyc" -delete 2>/dev/null || true
find "$package_dir/lib/Python/Lib" -name "test_*.py" -delete 2>/dev/null || true
find "$package_dir/lib/Python/Lib" -name "*_test.py" -delete 2>/dev/null || true
fi
# Create run script
cat > "$package_dir/run.sh" << 'EOF'
#!/bin/bash
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
export LD_LIBRARY_PATH="$SCRIPT_DIR/lib:$LD_LIBRARY_PATH"
exec "$SCRIPT_DIR/mcrogueface" "$@"
EOF
chmod +x "$package_dir/run.sh"
# Create the distribution archive
log_info "Creating archive: ${package_name}.tar.gz"
(cd "$DIST_DIR" && tar -czf "${package_name}.tar.gz" "$package_name")
# Report size
local size=$(du -h "$DIST_DIR/${package_name}.tar.gz" | cut -f1)
log_info "Created: $DIST_DIR/${package_name}.tar.gz ($size)"
# Cleanup uncompressed directory
rm -rf "$package_dir"
}
show_usage() {
echo "McRogueFace Distribution Packager"
echo ""
echo "Usage: $0 <platform> <preset>"
echo ""
echo "Platforms:"
echo " windows - Windows build (requires 'make windows' first)"
echo " linux - Linux build (requires 'make' first)"
echo " all - Build all variants"
echo ""
echo "Presets:"
echo " light - Minimal stdlib (~2-3 MB)"
echo " full - Complete stdlib (~8-10 MB)"
echo ""
echo "Examples:"
echo " $0 windows light # Small Windows package"
echo " $0 linux full # Full Linux package"
echo " $0 all # All platform/preset combinations"
}
main() {
local platform=${1:-}
local preset=${2:-full}
# Create dist directory
mkdir -p "$DIST_DIR"
case "$platform" in
windows)
package_windows "$preset"
;;
linux)
package_linux "$preset"
;;
all)
log_info "Building all distribution variants..."
package_windows light || true
package_windows full || true
package_linux light || true
package_linux full || true
log_info "All packages created in $DIST_DIR"
ls -lh "$DIST_DIR"/*.zip "$DIST_DIR"/*.tar.gz 2>/dev/null || true
;;
-h|--help|"")
show_usage
exit 0
;;
*)
log_error "Unknown platform: $platform"
show_usage
exit 1
;;
esac
}
main "$@"

304
tools/package_stdlib.py Executable file
View file

@ -0,0 +1,304 @@
#!/usr/bin/env python3
"""
McRogueFace Standard Library Packager
Creates light/full stdlib variants from Python source or existing stdlib.
Compiles to .pyc bytecode and creates platform-appropriate zip archives.
Usage:
python3 package_stdlib.py --preset light --platform windows --output dist/
python3 package_stdlib.py --preset full --platform linux --output dist/
"""
import argparse
import compileall
import fnmatch
import os
import py_compile
import shutil
import sys
import tempfile
import zipfile
from pathlib import Path
# Try to import yaml, fall back to simple parser if not available
try:
import yaml
HAS_YAML = True
except ImportError:
HAS_YAML = False
SCRIPT_DIR = Path(__file__).parent
PROJECT_ROOT = SCRIPT_DIR.parent
CONFIG_FILE = SCRIPT_DIR / "stdlib_modules.yaml"
# Default module lists if YAML not available or for fallback
DEFAULT_CORE = [
'abc', 'codecs', 'encodings', 'enum', 'genericpath', 'io', 'os',
'posixpath', 'ntpath', 'stat', '_collections_abc', '_sitebuiltins',
'site', 'types', 'warnings', 'reprlib', 'keyword', 'operator',
'linecache', 'tokenize', 'token'
]
DEFAULT_GAMEDEV = [
'random', 'json', 'collections', 'dataclasses', 'pathlib', 're',
'functools', 'itertools', 'bisect', 'heapq', 'copy', 'weakref', 'colorsys'
]
DEFAULT_UTILITY = [
'contextlib', 'datetime', 'time', 'calendar', 'string', 'textwrap',
'shutil', 'tempfile', 'glob', 'fnmatch', 'hashlib', 'hmac', 'base64',
'binascii', 'struct', 'array', 'queue', 'threading', '_threading_local'
]
DEFAULT_TYPING = ['typing', 'annotationlib']
DEFAULT_DATA = [
'pickle', 'csv', 'configparser', 'zipfile', 'tarfile',
'gzip', 'bz2', 'lzma'
]
DEFAULT_EXCLUDE = [
'test', 'tests', 'idlelib', 'idle', 'ensurepip', 'tkinter', 'turtle',
'turtledemo', 'pydoc', 'pydoc_data', 'lib2to3', 'distutils', 'venv',
'__phello__', '_pyrepl'
]
EXCLUDE_PATTERNS = [
'**/test_*.py', '**/tests/**', '**/*_test.py', '**/__pycache__/**',
'**/*.pyc', '**/*.pyo'
]
def parse_yaml_config():
"""Parse the YAML configuration file."""
if not HAS_YAML:
return None
if not CONFIG_FILE.exists():
return None
with open(CONFIG_FILE) as f:
return yaml.safe_load(f)
def get_module_list(preset: str, config: dict = None) -> tuple:
"""Get the list of modules to include and patterns to exclude."""
if config and 'presets' in config and preset in config['presets']:
preset_config = config['presets'][preset]
include_categories = preset_config.get('include', [])
exclude_patterns = preset_config.get('exclude_patterns', EXCLUDE_PATTERNS)
modules = []
for category in include_categories:
if category in config:
modules.extend(config[category])
# Always add exclude list
exclude_modules = config.get('exclude', DEFAULT_EXCLUDE)
return modules, exclude_modules, exclude_patterns
# Fallback to defaults
if preset == 'light':
modules = DEFAULT_CORE + DEFAULT_GAMEDEV + DEFAULT_UTILITY + DEFAULT_TYPING + DEFAULT_DATA
else: # full
modules = DEFAULT_CORE + DEFAULT_GAMEDEV + DEFAULT_UTILITY + DEFAULT_TYPING + DEFAULT_DATA
# Add more for full build (text, debug, network, async, system would be added here)
return modules, DEFAULT_EXCLUDE, EXCLUDE_PATTERNS
def should_include_file(filepath: Path, include_modules: list, exclude_modules: list,
exclude_patterns: list) -> bool:
"""Determine if a file should be included in the stdlib."""
rel_path = str(filepath)
# Check exclude patterns first
for pattern in exclude_patterns:
if fnmatch.fnmatch(rel_path, pattern):
return False
# Get the top-level module name
parts = filepath.parts
if not parts:
return False
# Remove file extension properly (handle .pyc before .py to avoid partial match)
top_module = parts[0]
if top_module.endswith('.pyc'):
top_module = top_module[:-4]
elif top_module.endswith('.py'):
top_module = top_module[:-3]
# Check if explicitly excluded
if top_module in exclude_modules:
return False
# For preset-based filtering, check if module is in include list
# But be permissive - include if it's a submodule of an included module
# or if it's a standalone .py file that matches
for mod in include_modules:
if top_module == mod or top_module.startswith(mod + '.'):
return True
# Check for directory modules
if mod in parts:
return True
return False
def compile_to_pyc(src_dir: Path, dest_dir: Path, include_modules: list,
exclude_modules: list, exclude_patterns: list) -> int:
"""Compile Python source files to .pyc bytecode."""
count = 0
for src_file in src_dir.rglob('*.py'):
rel_path = src_file.relative_to(src_dir)
if not should_include_file(rel_path, include_modules, exclude_modules, exclude_patterns):
continue
# Determine destination path (replace .py with .pyc)
dest_file = dest_dir / rel_path.with_suffix('.pyc')
dest_file.parent.mkdir(parents=True, exist_ok=True)
try:
# Compile to bytecode
py_compile.compile(str(src_file), str(dest_file), doraise=True)
count += 1
except py_compile.PyCompileError as e:
print(f"Warning: Failed to compile {src_file}: {e}", file=sys.stderr)
return count
def repackage_existing_zip(src_zip: Path, dest_zip: Path, include_modules: list,
exclude_modules: list, exclude_patterns: list) -> int:
"""Repackage an existing stdlib zip with filtering."""
count = 0
with zipfile.ZipFile(src_zip, 'r') as src:
with zipfile.ZipFile(dest_zip, 'w', zipfile.ZIP_DEFLATED) as dest:
for info in src.infolist():
rel_path = Path(info.filename)
if not should_include_file(rel_path, include_modules, exclude_modules, exclude_patterns):
continue
# Copy the file
data = src.read(info.filename)
dest.writestr(info, data)
count += 1
return count
def create_stdlib_zip(source: Path, output: Path, preset: str,
platform: str, config: dict = None) -> Path:
"""Create a stdlib zip file from source directory or existing zip."""
include_modules, exclude_modules, exclude_patterns = get_module_list(preset, config)
# Determine output filename
output_name = f"python314-{preset}.zip"
output_path = output / output_name
output.mkdir(parents=True, exist_ok=True)
if source.suffix == '.zip':
# Repackage existing zip
print(f"Repackaging {source} -> {output_path}")
count = repackage_existing_zip(source, output_path, include_modules,
exclude_modules, exclude_patterns)
else:
# Compile from source directory
print(f"Compiling {source} -> {output_path}")
with tempfile.TemporaryDirectory() as tmpdir:
tmp_path = Path(tmpdir)
count = compile_to_pyc(source, tmp_path, include_modules,
exclude_modules, exclude_patterns)
# Create zip from compiled files
with zipfile.ZipFile(output_path, 'w', zipfile.ZIP_DEFLATED) as zf:
for pyc_file in tmp_path.rglob('*.pyc'):
arc_name = pyc_file.relative_to(tmp_path)
zf.write(pyc_file, arc_name)
size_mb = output_path.stat().st_size / (1024 * 1024)
print(f"Created {output_path} ({count} files, {size_mb:.2f} MB)")
return output_path
def find_stdlib_source(platform: str) -> Path:
"""Find the stdlib source for the given platform."""
if platform == 'windows':
# Check for existing Windows stdlib zip
win_stdlib = PROJECT_ROOT / '__lib_windows' / 'python314.zip'
if win_stdlib.exists():
return win_stdlib
# Fall back to cpython source
cpython_lib = PROJECT_ROOT / 'modules' / 'cpython' / 'Lib'
if cpython_lib.exists():
return cpython_lib
else: # linux
# Check for existing Linux stdlib
linux_stdlib = PROJECT_ROOT / '__lib' / 'Python' / 'Lib'
if linux_stdlib.exists():
return linux_stdlib
# Fall back to cpython source
cpython_lib = PROJECT_ROOT / 'modules' / 'cpython' / 'Lib'
if cpython_lib.exists():
return cpython_lib
raise FileNotFoundError(f"Could not find stdlib source for {platform}")
def main():
parser = argparse.ArgumentParser(description='Package McRogueFace Python stdlib')
parser.add_argument('--preset', choices=['light', 'full'], default='full',
help='Stdlib preset (default: full)')
parser.add_argument('--platform', choices=['windows', 'linux'], required=True,
help='Target platform')
parser.add_argument('--output', type=Path, default=Path('dist'),
help='Output directory (default: dist)')
parser.add_argument('--source', type=Path, default=None,
help='Override stdlib source (zip or directory)')
parser.add_argument('--list-modules', action='store_true',
help='List modules for preset and exit')
args = parser.parse_args()
# Parse config
config = parse_yaml_config()
if args.list_modules:
include, exclude, patterns = get_module_list(args.preset, config)
print(f"Preset: {args.preset}")
print(f"Include modules ({len(include)}):")
for mod in sorted(include):
print(f" {mod}")
print(f"\nExclude modules ({len(exclude)}):")
for mod in sorted(exclude):
print(f" {mod}")
return 0
# Find source
if args.source:
source = args.source
else:
source = find_stdlib_source(args.platform)
print(f"Source: {source}")
print(f"Preset: {args.preset}")
print(f"Platform: {args.platform}")
# Create stdlib zip
create_stdlib_zip(source, args.output, args.preset, args.platform, config)
return 0
if __name__ == '__main__':
sys.exit(main())

239
tools/stdlib_modules.yaml Normal file
View file

@ -0,0 +1,239 @@
# McRogueFace Python Standard Library Module Configuration
# Used by package_stdlib.py to create light/full distribution variants
#
# Categories:
# core - Required for Python interpreter to function
# gamedev - Commonly needed for game development
# utility - Generally useful modules
# typing - Type hints and annotations
# data - Data structures and serialization
# text - Text processing and parsing
# debug - Debugging and development tools
# network - Networking (excluded from light)
# async - Async programming (excluded from light)
# system - System/OS interaction (selective)
# exclude - Always excluded (test suites, IDE, etc.)
# Modules required for Python to start
core:
- abc
- codecs
- encodings # Directory - all encodings
- enum
- genericpath
- io
- os
- posixpath
- ntpath
- stat
- _collections_abc
- _sitebuiltins
- site
- types
- warnings
- reprlib
- keyword
- operator
- linecache
- tokenize
- token
# Game development essentials
gamedev:
- random
- json
- math # Note: mostly builtin, but module exists
- collections
- dataclasses
- pathlib
- re
- functools
- itertools
- bisect
- heapq
- copy
- weakref
- colorsys # Color conversion utilities
# Generally useful utilities
utility:
- contextlib
- datetime
- time
- calendar
- string
- textwrap
- shutil
- tempfile
- glob
- fnmatch
- hashlib
- hmac
- base64
- binascii
- struct
- array
- queue
- threading
- _threading_local
# Type system support
typing:
- typing
- typing_extensions # If present
- annotationlib
# Data handling
data:
- pickle
- shelve
- dbm
- csv
- configparser
- tomllib
- zipfile
- tarfile
- gzip
- bz2
- lzma
- zlib
# Text processing
text:
- html
- html.parser
- html.entities
- xml
- xml.etree
- xml.etree.ElementTree
- xml.dom
- xml.sax
- difflib
- pprint
- gettext
- locale
# Debugging/development (include in full, optional for light)
debug:
- traceback
- logging
- pdb
- dis
- inspect
- ast
- code
- codeop
- profile
- cProfile
- pstats
- timeit
# Networking (excluded from light builds)
network:
- socket
- ssl
- http
- http.client
- http.server
- http.cookies
- http.cookiejar
- urllib
- urllib.request
- urllib.parse
- urllib.error
- ftplib
- imaplib
- poplib
- smtplib
- email
- mailbox
- mimetypes
- webbrowser
# Async support (excluded from light builds)
async:
- asyncio
- concurrent
- concurrent.futures
- selectors
- select
# System interaction (selective inclusion)
system:
- subprocess
- multiprocessing
- signal
- platform
- sysconfig
- importlib
- pkgutil
- runpy
- zipimport
- ctypes
- getopt
- argparse
- getpass
- grp # Unix only
- pwd # Unix only
- pty # Unix only
- tty # Unix only
- termios # Unix only
- fcntl # Unix only
- resource # Unix only
- nis # Unix only
- spwd # Unix only
- crypt # Unix only
- winreg # Windows only
- msvcrt # Windows only
- winsound # Windows only
- _winapi # Windows only
- ensurepip # pip installer
- venv # Virtual environments
# Always excluded - never include these
exclude:
- test # Python test suite (34MB!)
- tests # Any tests directories
- idlelib # IDLE editor
- idle # IDLE
- tkinter # No Tcl/Tk runtime
- turtle # Requires tkinter
- turtledemo # Turtle demos
- pydoc # Documentation generator
- pydoc_data # Documentation strings (571KB)
- lib2to3 # Python 2 to 3 converter
- distutils # Deprecated
- __phello__ # Test package
- _pyrepl # REPL (we have our own)
# Build presets
presets:
light:
include:
- core
- gamedev
- utility
- typing
- data
- system
exclude_patterns:
- "**/test_*.py"
- "**/tests/**"
- "**/*_test.py"
full:
include:
- core
- gamedev
- utility
- typing
- data
- text
- debug
- network
- async
- system
exclude_patterns:
- "**/test_*.py"
- "**/tests/**"
- "**/*_test.py"