2D & UI primer #786

Merged
SakulFlee merged 73 commits from feat/primer_2d_ui into main 2026-09-20 21:39:06 +00:00
Owner
No description provided.
- Add Parent(Entity) and Children(Vec<Entity>) components for entity hierarchy
- Add hierarchy utilities: add_child, remove_child, find_root, ancestors, descendants
- Add clean_hierarchy() to remove stale parent/child references
- Add generic Events<T> resource for inter-system communication
- All 93 tests pass with no warnings
- Comprehensive plan for 2D rendering, layered rendering, and UI system
- 6 phases: ECS extensions, 2D core, text rendering, render layers, UI, integration
- Design decisions documented (layout, camera, batching, fonts, events, layers)
- Camera2D: orthographic and perspective projections
- ShapeDescriptor: edge-count based shape generation (3=triangle, 4=quad, 8+=polygon)
- Vertex2D: vertex format with position, color, texcoord, shape_params
- Batch2D: batched rendering for efficient draw calls
- 29 tests passing
- vertex_2d_input: 2D vertex input layout
- vertex_2d_output: 2D vertex output structure
- vertex_2d_transform: camera-based vertex transform
- fragment_2d_color: solid color fragment shader
- fragment_2d_texture: textured fragment shader with sampler
- FontData: font loading via fontdue with glyph caching
- SdfAtlas: texture atlas with shelf packing for glyph storage
- Text mesh generation for 2D screen-space and 3D billboard text
- TextConfig: font size, color, word wrapping, line height
- measure_text: text dimension calculation
- SDF shader node for resolution-independent text rendering
- 13 tests passing
- Default Inter font included in assets
- RenderLayer enum: Skybox, Scene3D, Scene3DTransparent, World2D, Overlay3D, UI, Debug
- LayerConfig: enable/disable layers, query layer properties
- LayerRenderer trait: layer-aware rendering with automatic sort by layer order
- Updated RenderOverlayContext with screen_size field
- RenderOverlayResource: separate overlays and layer_renderers
- ModuleRuntime: renders layer renderers sorted by layer order
- 9 tests passing
- UI components: UiButton, UiText, UiTextBox, UiCheckbox, UiImage, UiBackground
- Widget helpers: create_button, create_text, create_textbox, create_checkbox
- Layout system: absolute, horizontal, vertical layouts with alignment
- Hit testing: topmost element selection, z-order aware
- Events: ButtonPressed, ButtonReleased, TextBoxChanged, CheckboxToggled
- 32 tests passing
- New 2D template with Camera2D, ShapeDescriptor, minimal 2D game setup
- Register '2d' template in CLI (available alongside minimal, all-in-one)
- Add ui_demo example: Button, Text, TextBox, Checkbox, event handling
- Add scene_2d example: 2D shapes, camera, minimal 2D game
- Re-export orbital_2d, orbital_text, orbital_ui from orbital facade
- Add fontdue dependency to workspace
- 176 tests passing across all crates
- Add SdfAtlas::build_atlas() to generate atlas from FontData
- Make generate_sdf() public for use by atlas module
- Update generate_text_mesh() to accept Option<&SdfAtlas>
- When atlas is provided, UV coordinates are looked up per-glyph
- Backward compatible: None uses placeholder UVs
- 14 tests passing
- DefaultFont struct with bytes() and load() methods
- Embeds Inter font (SIL Open Font License)
- No runtime file I/O needed for default font
- 14 tests passing
- Renderer2D struct with GPU pipeline, vertex buffer, bind group
- Camera2DUniform for orthographic projection
- WGSL shader for solid-color 2D with per-vertex alpha blending
- Vertex layout matches orbital_2d::Vertex2D (40 bytes stride)
- Ready to be wired into main renderer
- UiRenderer struct with GPU pipeline and vertex buffer
- WGSL shader for SDF rounded corners with anti-aliasing
- Uses fwidth() for screen-space anti-aliasing
- Alpha blending for UI transparency
- Ready to be wired into main renderer
- TextRenderer struct with GPU pipeline and vertex buffer
- WGSL shader for SDF text with anti-aliasing and outline support
- upload_atlas() to send SdfAtlas data to GPU
- create_atlas_bind_group() for texture sampling
- Supports both screen-space and world-space text
- Added orbital_text dependency to orbital_renderer
- render_2d(): renders solid-color 2D shapes with alpha blending
- render_ui(): renders SDF rounded rectangles for UI backgrounds
- render_text(): renders SDF text with atlas texture sampling
- All methods create their own render passes with LoadOp::Load
- Added orbital_2d dependency to orbital_renderer
- Create ShapeOverlay implementing RenderOverlay
- Generate vertices for square, circle, triangle, quad
- Use orthographic projection for screen-space rendering
- Register overlay via RenderOverlayResource
- Create UiBackgroundOverlay implementing RenderOverlay
- Generate rounded rectangle vertices for UI elements
- Use SDF rendering for smooth corners
- Register overlay via RenderOverlayResource
All phases complete, plan document no longer needed.
- Use generate_rect(x, y, w, h, color) for quads (top-left corner position)
- Use generate_circle(center, radius, segments, color) for circles
- Offset triangle and hexagon vertices after generation
- Added cyan hexagon as additional shape
- All shapes now positioned correctly on screen
- Divide by (min(w,h) / 2.0) instead of min(w,h)
- P coordinate ranges 0-1 where 1 = min(w,h)/2 pixels from center
- Corner radius now renders at correct pixel size
- Add TextOverlay with SDF atlas and font loading
- Render labels: title, button text, placeholder, checkbox label
- Build 1024x1024 atlas for UI character set
- Upload atlas to GPU on first render
- Text rendered with anti-aliasing via SDF smoothstep
- WGSL vec4<f32> requires 16-byte alignment
- Add _padding: [f32; 2] after outline_width
- Struct now 32 bytes matching shader expectation
- Fixes BindGroup size mismatch error in ui_demo text rendering
- Fix SDF sign convention: inside glyph → positive, outside → negative
- Fix UV Y-coordinates: y0 maps to top of atlas, y1 maps to bottom
- Fix atlas GlyphInfo to use actual metrics (offset, advance)
- Update pack_glyph to accept fontdue::Metrics for correct values
- All 14 orbital_text tests passing
- Change glyph_y calculation from 'cursor_y + ymin' to 'cursor_y - ymin - height'
- Properly converts fontdue's Y-up baseline-relative coords to screen Y-down coords
- All glyphs on same line now share consistent top edge
- Characters with descenders (g, p, y) correctly extend below baseline
- All 14 orbital_text tests passing
- Add orbital-iced as git submodule (iced fork with wgpu v30 support)
- Apply wgpu v30 changes from iced PR #3420
- Fix cosmic-text version conflict via [patch.crates-io]
- Create orbital_iced crate for UI integration layer
- IcedState, IcedEventBridge, IcedLayerRenderer stubs
- Exclude orbital-iced from workspace members
- Add AdapterResource to ECS for iced Engine creation
- Implement IcedLayerRenderer with real iced wgpu::Renderer
- IcedState holds view() returning iced Element tree
- Build UserInterface, process events, draw widgets via iced
- Create iced_demo example registering IcedLayerRenderer
- Each change committed individually
- Fix workspace-level iced paths (crates/ -> Crates/)
- Add IcedWindowEvent owned enum to orbital_ecs_bridge
- Add IcedEventQueue ECS resource with cursor + modifiers tracking
- Populate IcedEventQueue from module_runtime.rs window_event()
- IcedLayerRenderer::render() now drains events, converts to iced events
- Cursor position forwarded from CursorMoved events
- Mouse button events forwarded to iced UserInterface
- ModifiersChanged events forwarded to iced
- Remove dead engine field from RendererInner
- All packages compile with zero warnings
- Enable fira-sans feature on iced_graphics so FiraSans-Regular.ttf
  is bundled as the sans-serif fallback font (fixes cosmic_text
  'No default font match for SansSerif' warnings)
- Store iced Cache in RendererInner instead of creating new one
  each frame (avoids full widget tree rebuild every frame)
- Forward keyboard events to iced with proper key/physical_key/
  modifiers/text/repeat conversion (using winit types directly
  to avoid version mismatch between workspace winit and iced
  fork winit)
Dark background (rgba 0.05, 0.05, 0.1, 0.75) with rounded corners
on the container holding the title and button.
- New FloatingPanel widget with title bar, content, and close button
- Drag-to-move via title bar with 3px deadband
- Close button publishes on_close message
- Grab/Grabbing cursor hints
- Stores position in widget::Tree state (persists across frames)
- State includes panel_visible flag for show/hide
- iced_demo updated to show draggable floating panel
When UserInterface::build() uses a cached tree from Cache::new(), it
calls diff() directly without Tree::new(). The tree has State::None,
so layout() panics on downcast_mut::<State>(). Fix: check for State::None
in diff() and initialize tag/state if needed.
Tree::new() never calls diff(), only sets tag and state. On first
frame, UserInterface::build() creates a fresh tree via Tree::new()
when tags don't match, leaving children empty. Then layout() tries
to access tree.children[0] and panics with 'index out of bounds'.

Fix: call self.diff(tree) at the start of layout() if children
aren't populated yet (len < 2).
Tree::new() creates a tree with correct tag/state but empty children
and never calls Widget::diff(). This means the Column child's tree
had 0 children when layout() was called, causing flex::resolve to
panic on 'index out of bounds'.

Fix: after creating child trees via Tree::new(), call Widget::diff()
on each to recursively populate sub-trees (e.g. Column's button).
- Draw dark semi-transparent panel background with rounded corners
- Draw title bar background with top-rounded corners
- Draw close button quad (red, hover highlight)
- Fix SDF shader: add missing min(max(q.x, q.y), 0.0) term for
  correct signed distance inside the rectangle
The root node was at (0,0) and only children were offset by
state.position. This meant the background quad (drawn from
layout.bounds()) was always at (0,0) while only the title and
content moved during drag.

Fix: move the root node to state.position and position children
relative to (0,0) within it. Now the entire panel frame moves
as one unit.
Tree::new() creates fresh state on every frame, destroying widget state
like Button.is_pressed. ButtonPressed and ButtonReleased almost never
land in the same frame, so is_pressed was always false on release and
shell.publish() was never reached.

Replace manual clear/recreate with tree.diff_children(), which
reconciles old and new children — preserving existing tree state for
unchanged widgets.
Add RenderOverlayResource::ensure() and IcedLayerRenderer::register()
to eliminate the repeated check-then-insert boilerplate in Module::setup().

Before (every module):
  if ecs.get_resource::<RenderOverlayResource>().is_none() {
      ecs.insert_resource(RenderOverlayResource::new());
  }
  if let Some(res) = ecs.get_resource_mut::<RenderOverlayResource>() {
      res.add_layer_renderer(Box::new(overlay));
  }

After:
  IcedLayerRenderer::new(state).register(ecs);
Modules now declare UI intent by inserting IcedUiState into the ECS.
IcedBridgeModule detects it and creates the renderer automatically.

Before (module creates and registers renderer directly):
  let overlay = IcedLayerRenderer::new(state);
  ecs.insert_resource(RenderOverlayResource::new());
  ecs.get_resource_mut::<RenderOverlayResource>()
      .unwrap()
      .add_layer_renderer(Box::new(overlay));

After (module just inserts state, bridge handles the rest):
  ecs.insert_resource(IcedUiState(state));

App::new()
    .add_module(MyModule)
    .add_module(IcedBridgeModule)
    .liftoff(event_loop, settings)
- Add IcedBridge, OrbitalUI, OrbitalUIModule aliases for IcedBridgeModule
- Add iced UI to all-in-one template (IcedUiState + IcedBridgeModule)
- Update 2d template with working 2D shape rendering (from 2d_scene example)
- Delete iced_demo example (content now in all-in-one template)
- Delete 2d_scene example (moved to 2d template)
- IcedUiState: HashMap<String, IcedState> with push() API
- IcedState: titled(), with_hud(), with_view(&World), with_position()
- view(&World): view functions read ECS directly, no message queues needed
- Bridge iterates collection, creates renderer per entry
- Template: HUD + Inventory examples demonstrating both patterns
- ViewFn uses Arc<dyn Fn> for Clone support
- orbital/Cargo.toml: add orbital_iced dependency
- orbital/lib.rs: re-export orbital_iced as orbital::iced
- orbital_iced/lib.rs: re-export iced_widget for template view closures
- Template: fix imports to use orbital::iced paths
- .gitmodules: change orbital-iced submodule URLs from SSH to HTTPS
- prompt.rs: validate engine-repo is a remote URL, not a local path
- Remove orbital-iced workspace boundary (exclude + own [workspace])
- Add all iced sub-crates as Orbital workspace members
- Merge iced workspace dependencies, lints, and patches into root
- Use iced fork's winit consistently across all crates
- Fixes cryoglyph using crates.io cosmic-text while iced uses git fork
cryoglyph (git dep of iced_wgpu) pulls cosmic-text 0.19.0 from
crates.io while the iced fork uses a hecrj-patched fork. Without
this patch, cargo builds two incompatible cosmic-text versions and
iced_wgpu fails to compile. Added [patch.crates-io] with a comment
explaining why to all three templates (minimal, all-in-one, 2d).
Two issues when building generated projects from git dependency:

1. Template Cargo.toml files with {name} placeholders were being parsed
   as workspace members during git checkout resolution. Added workspace
   exclude for template directories.

2. Generated project depended on winit 0.30.0 from crates.io, but
   orbital re-exports the iced fork's winit. Two versions = type
   mismatch. Removed direct winit dep; use orbital::winit instead.
Template Cargo.toml files with {name} placeholders were being parsed
by cargo when resolving the Orbital workspace from a git dependency.
Removed the files entirely; all three templates now share the same
generate_cargo_toml() function.

Also fixed make_desktop_main/make_android_main macros to use
::orbital::winit instead of ::winit, eliminating the need for
generated projects to depend on winit directly.
Engine changes:
- New CursorToggle system: toggles cursor grab on Escape press with
  edge detection (only fires once per press, not held). Pushes
  EngineEvent::CursorGrabbed/EngineEvents via existing event plumbing.

Template changes (all-in-one):
- StatsAnimator: decrements Health/Mana every 0.5s, resets to 100
  when reaching 0. HUD now shows live values instead of '--'.
- ShapeOverlay: renders 2D shapes (red rect, green circle, blue
  triangle, yellow quad) on top of the 3D scene via RenderOverlay.
- Escape key now releases cursor for UI interaction.
Camera controller:
- Added CursorGrabState ECS resource to track live grab state
- sys_camera_controller now reads CursorGrabState and skips mouse
  rotation when cursor is visible (ungrabbed). WASD movement still works.
- Added 5-parameter IntoSystem variant (3 Res + 2 &mut) to ecs crate.

Cursor toggle:
- Changed toggle key from Escape to AltLeft
- CursorToggle now updates CursorGrabState resource on toggle

2D shapes overlay:
- Shapes now dynamically positioned on first render using screen_size
- Centered at top of screen: red rect, green circle, blue triangle,
  yellow quad spaced evenly around screen center-top
- Small scale (~25-90px) to avoid overlapping HUD text
Cursor toggle:
- CursorVisible event now uses !self.grabbed so cursor is shown
  when ungrabbed and hidden when grabbed (was inverted)

2D shape overlay:
- Reduced from 4 shapes to 3: green circle, blue triangle, red rectangle
- Centered on top of screen with even spacing
- Removed yellow rectangle and extra shapes that caused duplication
- Replaced generate_rect with generate_shape_vertices + solid_quad
- Moved from cx+30 to cx+55 (was cx+30 + 80 width = cx+110, now cx+55 + 70 width = cx+90)
- Shapes are now: green circle, blue triangle, red quad — centered on top
Shape overlay:
- Changed quad dimensions from 70x25 to 40x40 for a proper square

Iced UI interaction (PRIMARY FIX):
- Added scale_factor field to IcedEventQueue
- Runtime now queries window.scale_factor() and stores it in the queue
- Iced renderer converts physical cursor coordinates to logical coordinates
- Screen size now converted from physical to logical for UserInterface::build()
- CursorMoved events now converted from physical to logical coordinates
- Viewport scale factor now uses actual window scale instead of hardcoded 1.0

This fixes the issue where iced UI was non-interactive on HiDPI displays
because physical coordinates were larger than logical widget bounds, causing
cursor.is_over() to always return false.
Shape overlay:
- Moved square from cx+55 to cx+25 (closer to center)

Iced renderer:
- Reverted viewport Scale from {scale_factor, scale_factor} to {1.0, 1.0}
- This matches original rendering behavior and prevents UI from growing
- Cursor/event coordinate conversion (physical→logical) remains correct
- Layout size conversion (physical→logical) remains correct
- Log event count, cursor position (physical + logical), scale factor
- Log interface.update() state and event statuses
- Log published messages from widgets
- Add title() accessor to IcedState
- Forward WindowEvent::Focused to IcedEventQueue
- On focus loss (Focused=false): release cursor grab, show cursor,
  update CursorGrabState, publish engine events
- This fixes the issue where iced UI was unresponsive after toggling
  cursor visible via AltLeft — the Wayland compositor keeps pointer
  constraints active until focus loss resets them
- Remove diagnostic logging added during investigation
Scale factor conversion caused growing positional offset because layout
was in logical space while viewport was in physical space (Scale {1.0}).
Revert cursor, CursorMoved events, and layout size to physical pixels
to match the viewport — consistent with the working iced_demo.
Each IcedLayerRenderer::render() was calling q.drain() on the shared
IcedEventQueue. With multiple panels (HUD + Inventory), only the first
renderer received events. Since HashMap iteration order is
non-deterministic, the inventory button would only very rarely react.

Fix: each renderer now clones (not drains) the queue so all panels see
the same events. The main runtime drains once after all renderers have
processed.
CursorToggle maintained its own grabbed field which could desync from
CursorGrabState when Focused(false) released the cursor externally. On
Wayland, some runs start unfocused causing Focused(false) to fire,
setting CursorGrabState(false) while CursorToggle.grabbed stayed true.
The next Alt press would toggle to false (no-op), and the press after
that would re-grab the cursor — making the iced UI unresponsive with an
invisible cursor.

Fix by reading CursorGrabState at the start of each CursorToggle run
so the toggle always reflects the actual cursor state. Also remove
redundant EngineEvent pushes from the Focused(false) handler since the
direct window API calls and CursorGrabState update are sufficient.
Winit reports cursor/events/window size in physical pixels while iced
layout and hit-testing work in logical pixels (physical / scale). The
previous revert to physical + Scale{1,1} is only correct at 100% display
scaling; at 150% every widget is offset proportionally to its distance
from the top-left.

Mirror the canonical iced integration example: convert cursor,
CursorMoved events and layout size to logical, build the viewport with
the real window scale. Conversion happens once at the iced boundary;
engine resources stay physical.

Also add temporary INFO-level [iced-trace] logging (push/update/panel
messages) to verify event flow on real runs. To be removed once
verified.
The DPI scale fix is verified working at 150% display scaling
(scale=1.50 flowing through, physical->logical conversion correct).
Remove temporary [iced-trace] INFO logging from renderer, floating
panel and runtime.

Also stop forwarding WindowEvent::RedrawRequested into IcedEventQueue:
it is a winit-level signal, not iced input, and was forcing a useless
interface.update() with a Window event on every frame per panel.
Point Crates/orbital-iced at fix/android-clipboard-gating (arboard +
modifier_supplement gated out on Android; arboard has no Android backend).

Gate iced_winit x11/wayland features to Linux-only in orbital_iced so the
wayland feature (arboard/wayland-data-control) is not enabled on Android.
Apply the same modifier_supplement fallback in orbital_iced's key conversion.
Bare PresentMode::Immediate fails surface validation on devices that only
advertise Mailbox/Fifo (typical Android), killing the app at startup.
AutoNoVsync resolves inside wgpu (Immediate -> Mailbox -> Fifo), so the
intent 'vsync off' survives on every surface.

Also silence unused screen_h warning in all-in-one template.
Android emits only WindowEvent::Touch (no synthetic mouse events), and the iced bridge forwarded only mouse events, so no iced UI was interactive on touch. Forward touches through IcedEventQueue (mapping them to iced touch events via iced_winit::conversion::touch_event) and update the tracked cursor position from them, mirroring iced_winit's own window state - otherwise the cursor stays Unavailable and all hit-testing fails. FloatingPanel now handles finger press/drag/lift beside mouse, so panels can be dragged and closed by touch.

The all-in-one template's ShapeOverlay baked its vertex positions for the screen size of the first frame, so after rotating an Android screen the shapes stayed anchored to the old width while the projection followed the new one. Track the laid-out size and re-layout on change; skip zero-sized frames (WindowSize starts at 0x0, which previously produced a NaN projection).
Touches consumed by an iced UI overlay (buttons, FloatingPanel title-bar
drag/close, panel body) were also fed into the game-input path, driving the
virtual joystick / drag-to-look camera with fingers resting on the UI.

- Add IcedCapturedTouches ECS resource (set of winit touch ids the UI owns)
- IcedLayerRenderer now uses the per-event Status returned by
  interface.update(): Captured on a touch event registers the finger;
  lifted/lost fingers are always released
- FloatingPanel captures any touch press/move inside its bounds (after
  children, touch-only) so the panel body owns its finger too
- module_runtime drops Started/Moved for captured fingers from the
  game-input path (iced queue keeps receiving everything); a synthesized
  Cancelled releases a finger that slipped through in the 1-frame
  capture-latency window
- Diagnostics: log on newly-captured fingers and on game-input release
Raise the red square 10px so its vertical position matches the circle and
triangle in the ShapeOverlay HUD row.
Root cause: IcedCapturedTouches was populated during the render pass
(after game systems ran), so game touch input always saw stale capture
state from the previous frame. This caused touches to leak through to
the game on the first frame, then be blocked on subsequent frames,
breaking the virtual joystick and drag-to-look.

Changes:
- Add LayerRenderer::process_events() trait method (default no-op)
- Implement process_events() in IcedLayerRenderer: clones event queue,
  runs UserInterface::update(), populates IcedCapturedTouches, saves
  cache — all without GPU rendering
- Defer game touch input in window_event() to a Vec<Touch>, process
  them in update() after process_events() has current capture state
- Move overlay renderers out of RenderOverlayResource (ECS world) into
  ModuleRuntime fields to avoid borrow-checker conflicts (get_resource
  holds a read lock that prevents &mut World)
- Add Module::register_overlays() trait method for modules to register
  renderers with the runtime instead of the ECS resource
- Update IcedBridgeModule, DebugModule, TouchUiModule, and templates
  to use register_overlays()
- Store device/queue in IcedLayerRenderer for process_events()
  initialization (Engine::new takes owned Device/Queue)
Templates use orbital::logging::warn!, engine crates use
orbital_core::logging::warn!. Both re-export log::*, so log macros
work without adding log as a direct dependency.

Added orbital_core dependency to orbital_debug_render and
orbital_touch_ui for access to logging.
CombinedModule wraps all modules but was using the default no-op
register_overlays(), so no child module's overlays were ever registered
with the runtime. This caused all UI and 2D overlays to disappear.
fix: snapshot input state after deferred touch processing
Some checks failed
PR / lint (pull_request) Failing after 14s
5e9831435a
InputSnapshot was inserted before deferred touches were processed,
so game systems (camera controller) always read stale touch data.
This caused right-side drag-to-look to not work because touch deltas
weren't visible to sys_camera_controller until the next frame.

Move InputSnapshot insertion to after deferred touch processing so
game systems see the current frame's touch positions and deltas.
Format
Some checks failed
PR / lint (pull_request) Failing after 16s
fdb543dd5d
Clippy fixes: sort_by_key, too_many_arguments allow
Some checks failed
PR / lint (pull_request) Failing after 32s
a063b746d0
ci: initialize git submodules in checkout step
All checks were successful
PR / lint (pull_request) Successful in 8m40s
522cf297ef
cargo metadata fails because Crates/orbital-iced submodule isn't
checked out, causing 'No such file or directory' for iced_core's
Cargo.toml.
Sign in to join this conversation.
No description provided.