perf(reporting): record capture environment and timing distributions in performance CSV #1078

Closed
opened 2026-11-07 03:28:22 +00:00 by SakulFlee · 3 comments
Owner

Summary

The performance CSV is missing the context needed to compare captures across machines:

  • adapter name, backend, device type
  • surface/window resolution and DPI scale
  • monitor refresh rate
  • vsync state, present mode, fps limit
  • build profile, package version, git revision
  • AA/SMAA mode, deferred, volumetrics, bloom/glare
  • debug UI visibility / which panels are open
  • optionally scene/camera state

Without these, Windows and Linux captures differ by 1.5-10x and cannot be normalized. The Windows capture also has no information about resolution or refresh rate.

Timing fidelity

The CSV currently writes per-second averages only. cdt_ms is the last frame's delta, not an average, and short hitches are hidden. Add at least:

  • mean, min, max, p95 per stage
  • average CDT or true frame period
  • a sample count and dropped/stale timestamp indicator

Suggested shape

  • One metadata block at the top of the CSV, or a sidecar JSON with the same basename.
  • Keep the existing columns backward compatible and add new ones.
  • Document the schema in PERFORMANCE_REPORTING.md.

Acceptance criteria

  • A capture from any platform can be compared without external notes.
  • Resolution, refresh, present mode, build profile, and active rendering features are recorded.
  • Stage distributions are available, not only averages.
## Summary The performance CSV is missing the context needed to compare captures across machines: - adapter name, backend, device type - surface/window resolution and DPI scale - monitor refresh rate - vsync state, present mode, fps limit - build profile, package version, git revision - AA/SMAA mode, deferred, volumetrics, bloom/glare - debug UI visibility / which panels are open - optionally scene/camera state Without these, Windows and Linux captures differ by 1.5-10x and cannot be normalized. The Windows capture also has no information about resolution or refresh rate. ## Timing fidelity The CSV currently writes per-second averages only. `cdt_ms` is the last frame's delta, not an average, and short hitches are hidden. Add at least: - mean, min, max, p95 per stage - average CDT or true frame period - a sample count and dropped/stale timestamp indicator ## Suggested shape - One metadata block at the top of the CSV, or a sidecar JSON with the same basename. - Keep the existing columns backward compatible and add new ones. - Document the schema in `PERFORMANCE_REPORTING.md`. ## Acceptance criteria - A capture from any platform can be compared without external notes. - Resolution, refresh, present mode, build profile, and active rendering features are recorded. - Stage distributions are available, not only averages.
Author
Owner

The tablet capture makes this higher priority: phone present is ~89 ms, tablet present is ~342 ms, and pre-hybrid FPS was ~130 vs ~60-80. Without resolution, render scale, refresh, and present mode, we cannot tell whether the 3.8x present difference is pixel count, GPU capability, or a swapchain problem.

Please record at least resolution, render scale, refresh rate, present mode, and vsync/fps-limit state with each Android capture.

The tablet capture makes this higher priority: phone `present` is ~89 ms, tablet `present` is ~342 ms, and pre-hybrid FPS was ~130 vs ~60-80. Without resolution, render scale, refresh, and present mode, we cannot tell whether the 3.8x present difference is pixel count, GPU capability, or a swapchain problem. Please record at least resolution, render scale, refresh rate, present mode, and vsync/fps-limit state with each Android capture.
Author
Owner

The minimal Android capture shows why debug-UI state must be recorded:

  • Debug UI hidden: ~60 FPS, iced_draw ~0.05 ms, overlay_layers ~1.05 ms.
  • Debug UI open: ~40 FPS, iced_draw ~10.4 ms, overlay_layers ~15.3 ms.

Record at least:

  • overlay visible/hidden
  • visible panel count and panel names
  • active tab
  • graph series/window
  • whether touch UI is visible

Otherwise two captures from the same device and scene are not comparable.

The minimal Android capture shows why debug-UI state must be recorded: - Debug UI hidden: ~60 FPS, `iced_draw` ~0.05 ms, `overlay_layers` ~1.05 ms. - Debug UI open: ~40 FPS, `iced_draw` ~10.4 ms, `overlay_layers` ~15.3 ms. Record at least: - overlay visible/hidden - visible panel count and panel names - active tab - graph series/window - whether touch UI is visible Otherwise two captures from the same device and scene are not comparable.
Author
Owner

Expanded metadata list after the minimal-run analysis:

Startup / sidecar:

  • build: package version, debug/release, OS/arch
  • CPU: model, logical/physical cores, total/available RAM
  • GPU: adapter name, backend, device type, vendor/device, driver/driver info, relevant features (timestamp queries, SHADER_F16), key limits
  • window/surface: inner size, scale factor, monitor size/name/refresh, fullscreen variant (windowed / borderless / exclusive)
  • surface: format, color space, alpha mode, supported and selected present modes, vsync, fps limit, desired_maximum_frame_latency
  • renderer: forward/hybrid/deferred, AA/SMAA, volumetrics and quality, bloom/glare, render scale if present
  • scene: entity/model/light counts, shadow casters
  • debug UI: overlay visible, visible panel names/count, active tab, graph series/window
  • Android: ADPF target, power-save state, sustained-performance state, game mode

Per frame / per second:

  • visible debug-panel count (it changes during a capture)
  • surface status counters: success, suboptimal, timeout, outdated, lost
  • resize/reconfigure counters
  • frame_gap / event-loop wait and frame-limiter wait
  • optional CPU usage and process RAM at 1 Hz

This is not meant to be per-frame allocations or per-entity logging; startup + one row per second is enough unless a detail mode is enabled.

Expanded metadata list after the minimal-run analysis: Startup / sidecar: - build: package version, debug/release, OS/arch - CPU: model, logical/physical cores, total/available RAM - GPU: adapter name, backend, device type, vendor/device, driver/driver info, relevant features (timestamp queries, `SHADER_F16`), key limits - window/surface: inner size, scale factor, monitor size/name/refresh, fullscreen variant (windowed / borderless / exclusive) - surface: format, color space, alpha mode, supported and selected present modes, vsync, fps limit, `desired_maximum_frame_latency` - renderer: forward/hybrid/deferred, AA/SMAA, volumetrics and quality, bloom/glare, render scale if present - scene: entity/model/light counts, shadow casters - debug UI: overlay visible, visible panel names/count, active tab, graph series/window - Android: ADPF target, power-save state, sustained-performance state, game mode Per frame / per second: - visible debug-panel count (it changes during a capture) - surface status counters: success, suboptimal, timeout, outdated, lost - resize/reconfigure counters - `frame_gap` / event-loop wait and frame-limiter wait - optional CPU usage and process RAM at 1 Hz This is not meant to be per-frame allocations or per-entity logging; startup + one row per second is enough unless a detail mode is enabled.
Sign in to join this conversation.
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
SakulFlee/Orbital#1078
No description provided.