Status: Partially Implemented (Phase 1 complete, Phase 4 complete, Phase 2 partial)
Complexity: Major architectural overhaul
Impact: Grid System, Entity Management, Performance

Related Pages:

Grid-System - Current grid architecture
Entity-Management - Current entity usage
Grid-Rendering-Pipeline - Current rendering architecture (includes Dynamic Layer API)

Source Documents:

NEXT_GEN_GRIDS_ENTITIES_SHORTCOMINGS.md - Analysis of current limitations
NEXT_GEN_GRIDS_ENTITIES_PROPOSAL.md - Detailed technical proposal
NEXT_GEN_GRIDS_ENTITIES_IDEATION.md - Use cases and ideation

Related Issues:

#115 - SpatialHash for 10,000+ entities - CLOSED (completed)
#116 - Dirty flag system - CLOSED (completed)
#113 - Batch operations - CLOSED (completed)
#117 - Memory pool - Open (tier3-future)
#123 - Subgrid system - CLOSED (completed) (Grid now has children collection; see also #132)
#124 - Grid Point Animation - Open (tier4-deferred)
#122 - Parent-Child UI System - CLOSED (completed)
#237 - Multi-tile entity support - Open (future)
#147, #148, #150 - Dynamic Layer System - CLOSED (completed)

Implementation Progress

Phase 1: Performance Foundation - COMPLETE

All three Phase 1 items have been implemented and are in production:

SpatialHash (#115): O(1) entity lookup by grid position. Entities are indexed in a spatial hash on the grid, enabling fast queries for 10,000+ entities.
Dirty flag system (#116): Grids track dirty state to avoid unnecessary re-rendering. Only modified regions trigger render updates.
Batch operations (#113): Bulk entity operations implemented for efficient mass updates.

Phase 2: Multi-Tile Support - PARTIAL

Subgrid system (#123): CLOSED. Grid now supports a children collection (see also #132), enabling nested UI elements within grids.
Full multi-tile entity support (#237): Still open (future). Entity dimensions, occupied tile tracking, and multi-tile pathfinding remain unimplemented.

Phase 3: Flexible Content - NOT STARTED

Grid children (#132) enables attaching UIDrawables to grids, which partially addresses this use case. However, the proposed entity content flexibility (replacing hardcoded UISprite with arbitrary UIDrawable content) has not been implemented.

Phase 4: Layer System - COMPLETE (implemented differently than proposed)

The Dynamic Layer System (#147, #148, #150) has been fully implemented, but with a different API than originally proposed. Instead of a generic GridLayer class, the system uses standalone TileLayer and ColorLayer objects that are created independently and added to grids:

# Current API (implemented):
layer = mcrfpy.TileLayer(name="terrain", z_index=-1, texture=texture)
grid.add_layer(layer)

color_layer = mcrfpy.ColorLayer(name="fog", z_index=5)
grid.add_layer(color_layer)

# Access layers:
grid.layers       # returns tuple of all layers
grid.layer("fog") # lookup by name
grid.remove_layer(layer)
layer.visible = False

See Grid-Rendering-Pipeline for the complete current layer API documentation.

Executive Summary

The current UIEntity/UIGrid system has fundamental architectural limitations preventing implementation of modern roguelike features. This proposal outlines a comprehensive redesign supporting:

Flexible entity content - Entities containing any UIDrawable (Frame, Caption, Grid, Sprite)
Multi-tile entities - 2x2, 3x3, or arbitrary-sized creatures and structures
Custom layer system - Weather effects, particle layers, UI overlays
Spatial optimization - O(1) entity queries via spatial hashing
Memory efficiency - Optional gridstate, chunk-based loading

Key Insight: Maintain entity as grid-specific container (no inheritance from UIDrawable), but allow flexible content (any UIDrawable).

Current Limitations

1. Entity Type Rigidity

Problem:

UIEntity hardcoded to contain only UISprite
Cannot place Frames, Captions, or Grids on grids
Blocks speech bubbles, nested grids, complex UI

Current Code:

class UIEntity {
    UISprite sprite;  // Hardcoded!
    // Should be: std::shared_ptr<UIDrawable> content;
}

2. Single-Tile Limitation

Problem:

Entity position is single point
No concept of dimensions or occupied tiles
Blocks large enemies (2x2 dragons), multi-tile structures (castle doors)

Missing:

width/height properties
Spatial occupancy tracking
Collision detection for multi-tile entities

3. Fixed Layer System (RESOLVED)

Original Problem:

Grid had three hardcoded layers: tiles, entities, visibility
No custom layers
Blocked cloud layer, particle effects, weather overlays

Resolution: The Dynamic Layer System (#147, #148, #150) was implemented with TileLayer and ColorLayer objects. Grids now support arbitrary numbers of named layers with z-ordering and visibility control. See Grid-Rendering-Pipeline for details.

4. Performance Issues (RESOLVED)

Original Problem:

Linear O(n) iteration through all entities
No spatial indexing
Full grid re-render every frame

Resolution: SpatialHash (#115) provides O(1) entity lookup. Dirty flag system (#116) prevents unnecessary re-renders. These optimizations are now in production.

5. Memory Inefficiency

Problem:

Every entity maintains full gridstate vector (width x height)
Decorative entities (clouds) waste memory on visibility data
Cannot unload distant chunks

Status: Memory pool (#117) remains open as tier3-future work.

Proposed Architecture

Core Change 1: Flexible Entity Content

class UIEntity {  // No inheritance - grid-specific container
private:
    std::shared_ptr<UIDrawable> content;      // Any drawable!
    sf::Vector2f gridPosition;                // Position in grid coords
    sf::Vector2i dimensions;                  // Size in tiles (default 1x1)
    std::set<sf::Vector2i> occupiedTiles;     // Cached occupied positions
    std::vector<UIGridPointState> gridstate;  // Optional perspective data
    
public:
    void setContent(std::shared_ptr<UIDrawable> drawable);
    void renderAt(sf::RenderTarget& target, sf::Vector2f pixelPos);
    bool occupies(int x, int y) const;
};

Python API:

# Entity with sprite (backward compatible)
enemy = mcrfpy.Entity(grid_pos=(10, 10), sprite_index=5)

# Entity with frame (NEW - speech bubble)
speech_frame = mcrfpy.Frame(size=(100, 50))
speech_caption = mcrfpy.Caption(text="Hello!")
speech_frame.append(speech_caption)
speech_entity = mcrfpy.Entity(grid_pos=(player.x, player.y - 2))
speech_entity.content = speech_frame

# Entity with nested grid (NEW - mini-map)
minimap_grid = mcrfpy.Grid(grid_size=(20, 20), pos=(0, 0), size=(100, 100))
minimap_entity = mcrfpy.Entity(grid_pos=(5, 5))
minimap_entity.content = minimap_grid

Core Change 2: Multi-Tile Entities

class GridOccupancyMap {
private:
    std::unordered_map<int, std::set<std::shared_ptr<UIEntity>>> spatialHash;
    int cellSize = 16;
    
public:
    void addEntity(std::shared_ptr<UIEntity> entity);
    void removeEntity(std::shared_ptr<UIEntity> entity);
    std::vector<std::shared_ptr<UIEntity>> getEntitiesAt(int x, int y);  // O(1)
    std::vector<std::shared_ptr<UIEntity>> getEntitiesInRect(sf::IntRect rect);
};

Python API:

# Large enemy (2x2 tiles)
dragon = mcrfpy.Entity(
    grid_pos=(20, 20),
    sprite_index=10,
    dimensions=(2, 2)  # NEW: multi-tile support
)

# Check what tiles dragon occupies
occupied = dragon.occupied_tiles  # [(20, 20), (21, 20), (20, 21), (21, 21)]

# Collision detection accounts for size
if grid.can_move_to(dragon, new_x, new_y):
    dragon.x = new_x
    dragon.y = new_y

Core Change 3: Flexible Layer System (IMPLEMENTED - see above)

The layer system has been implemented as standalone TileLayer/ColorLayer objects rather than the generic GridLayer class originally proposed. The current implementation provides named layers, z-ordering, visibility control, and per-cell access. See the Implementation Progress section above and Grid-Rendering-Pipeline for the actual API.

Core Change 4: Spatial Optimization (IMPLEMENTED)

SpatialHash is now in production (#115). Entity queries are O(1) average case. The dirty flag system (#116) prevents unnecessary re-renders.

Migration Path

Phase 1: Performance Foundation (Issues #115, #116, #113) - COMPLETE

Backward compatible improvements:

~~Add SpatialHash to existing UIGrid~~ DONE
~~Implement dirty flag system~~ DONE
~~Add batch operations for entities~~ DONE

No breaking changes to Python API.

Phase 2: Multi-Tile Support (Issues #123, #237) - PARTIAL

The subgrid system (#123) is complete - Grid now has a children collection. Full multi-tile entity support (#237) remains future work:

// Still proposed (not yet implemented):
// Add to UIEntity class
sf::Vector2i dimensions = {1, 1};  // Default 1x1 (backward compatible)
std::set<sf::Vector2i> occupiedTiles;

void updateOccupiedTiles();
bool occupies(int x, int y) const;

Python API additions:

# Backward compatible - existing code works unchanged
enemy = mcrfpy.Entity(grid_pos=(10, 10), sprite_index=5)

# New code can specify dimensions
dragon = mcrfpy.Entity(grid_pos=(20, 20), sprite_index=10, dimensions=(2, 2))

Phase 3: Flexible Content (Issue #124) - NOT STARTED

Replace UIEntity::sprite with content:

// Deprecate: UISprite sprite;
// Add: std::shared_ptr<UIDrawable> content;

// Backward compatibility shim:
PyObject* get_sprite() {
    auto sprite = std::dynamic_pointer_cast<UISprite>(content);
    if (!sprite) {
        // Legacy: entity still has sprite member
        return legacy_sprite_accessor();
    }
    return RET_PY_INSTANCE(sprite);
}

Migration period: 1-2 releases with deprecation warnings.

Phase 4: Layer System - COMPLETE

Implemented as TileLayer/ColorLayer standalone objects (#147, #148, #150). See Grid-Rendering-Pipeline for the current API. The implementation differs from the original proposal (generic GridLayer class) but achieves the same goals: named layers, z-ordering, visibility, and per-cell control.

Use Cases Enabled

Speech Bubbles (requires Phase 3)

speech = mcrfpy.Frame(size=(100, 40))
speech.append(mcrfpy.Caption(text="Hello adventurer!"))
bubble = mcrfpy.Entity(grid_pos=(npc.x, npc.y - 1))
bubble.content = speech
grid.entities.append(bubble)

Large Enemies (requires Phase 2)

dragon = mcrfpy.Entity(grid_pos=(25, 25), sprite_index=DRAGON, dimensions=(3, 3))

# Pathfinding accounts for size
if grid.can_large_entity_move_to(dragon, new_x, new_y):
    dragon.move_to(new_x, new_y)

Weather Effects (POSSIBLE NOW with Dynamic Layers)

# Using current TileLayer API:
rain_layer = mcrfpy.TileLayer(name="rain", z_index=200, texture=rain_texture)
grid.add_layer(rain_layer)

# Set rain tile indices on the layer
for i in range(100):
    x, y = random.randint(0, 49), random.randint(0, 49)
    rain_layer.set((x, y), RAINDROP_TILE_INDEX)

Nested Mini-Map (requires Phase 3)

minimap = mcrfpy.Grid(grid_size=(20, 20), pos=(0, 0), size=(100, 100))
# ... populate minimap ...

minimap_entity = mcrfpy.Entity(grid_pos=(0, 0))
minimap_entity.content = minimap
hud_layer.entities.append(minimap_entity)

Performance Expectations

Before Phase 1 (Historical - Pre-Implementation)

1,000 entities: 60 FPS
10,000 entities: 15 FPS (unacceptable)
Entity query: O(n) = slow

After Phase 1 (Current - SpatialHash + Dirty Flags in Production)

1,000 entities: 60 FPS
10,000 entities: 60 FPS (with spatial hash + culling)
Entity query: O(1) average case

Note: The SpatialHash and dirty flag systems are now in production. The "Before" numbers above reflect historical performance prior to these optimizations.

Memory Impact (Projected - for future phases)

Per-entity overhead: +24 bytes (dimensions, occupied tiles set)
Spatial hash: ~8KB for 1000 entities (negligible)
Optional gridstate: Save width x height x sizeof(UIGridPointState) per decorative entity

Open Questions

Backward Compatibility Timeline
- How many releases should deprecation period last?
- Support for legacy entity.sprite accessor?
~~Layer API Design~~ (RESOLVED - TileLayer/ColorLayer implemented)
Multi-Tile Pathfinding (relevant if Phase 2 proceeds)
- Should large entities use separate TCOD maps?
- How to handle partially-blocked paths?
Content Delegation (relevant if Phase 3 proceeds)
- Should entity forward all UIDrawable methods to content?
- Or keep explicit entity.content.method() pattern?

Implementation Complexity

Estimated Effort:

~~Phase 1 (SpatialHash, dirty flags, batch ops): 40-60 hours~~ COMPLETE
Phase 2 (Multi-tile): 20-30 hours - partially done (subgrid complete, multi-tile entities remain)
Phase 3 (Flexible content): 30-40 hours - not started
~~Phase 4 (Layers): 40-50 hours~~ COMPLETE (implemented as TileLayer/ColorLayer)

Remaining: ~40-60 hours for Phase 2 completion + Phase 3 (if pursued)

Risk Areas:

Backward compatibility testing
Python binding complexity for flexible content
Performance regression testing
Documentation updates

Remaining Decisions

Phase 1 (Performance Foundation) and Phase 4 (Layer System) are complete and in production. The remaining question is whether to pursue:

Phase 2 completion (#237): Full multi-tile entity support (dimensions, occupied tiles, multi-tile pathfinding). This enables large enemies, multi-tile structures, and size-aware collision.
Phase 3 (#124): Flexible entity content (replacing hardcoded UISprite with arbitrary UIDrawable). This enables speech bubbles, nested grids in entities, and complex entity visuals.

Both remain as future/deferred priorities. The incremental approach has proven effective - the highest-impact items (performance and layers) were completed first.

Navigation:

Home - Documentation hub
Grid-System - Current architecture
Design-Proposals - All design proposals