gpu_vram_bytes() now chains per vendor/platform:
- NVIDIA everywhere: nvidia-smi (shipped by the NVIDIA driver on every
OS) — the NVDEC path's primary device.
- AMD/Intel on Linux: the DRM mem_info_vram_total/used sysfs attributes
(amdgpu, i915, Xe). Free = total - used; the first non-zero card wins
(an iGPU without dedicated vram reports 0 and is skipped). The walk is
now testable via an injectable sysfs root; a card missing the attrs is
skipped, never aborts the walk.
- Apple Silicon: unified memory — no separate vram exists; the RAM/4
budget IS the correct bound for decode surfaces and render targets, so
no query (an explicit vram budget would double-count the same pool).
- Windows AMD/Intel: no portable CLI; DXGI QueryVideoMemoryInfo is the
real API but wgpu 25 does not expose it. Falling back to the RAM
policy is safe-side (under-sized pool loses throughput, never OOMs the
device).
Tests: the sysfs walk (fixture with a missing-attr card, a 0-total
iGPU and a discrete winner) and the per-worker budget scaling (1080p
baseline, 4K ~4x, 60 fps over-provision).
- open_hw_accel marks the device unavailable when the decoder OPEN fails
(cuvidCreateDecoder OOM at 4K) too, not just device-context creation:
without it every subsequent decoder session retried CUDA and flooded
the log per open.
- Decoder gains hardware_decoding(); the oak-render decode-session LRU
evicts hardware sessions first (each pins a GPU surface pool — ~100 MB
at 4K), so a full cache cannot exhaust video memory before the next
open.
- Worker pool count now factors GPU vram: per-worker budget = 1 GiB
(1080p peak) scaled by pixel ratio + 256 MiB idle floor, 10% reserve
of free vram; applied when hardware decoding is on (nvidia-smi query,
None otherwise falls back to the RAM/CPU policy).
- Dynamic pool resize: ProcessDispatcher::set_target_workers grows or
retires workers; retiring ones stop claiming, drain their in-flight
batch (future playback frames included), then exit naturally on the
shutdown signal — no mid-work kill (30 s deadline only as a hung-
decoder last resort). A retiring worker that dies re-queues its frames
to surviving workers. Resizes are throttled to 2 s (a resolution burst
merges; only the latest target applies) so 1080p<->4K flaps cannot
thrash process spawns.
- RenderManager::set_workspace_size announces the sequence resolution;
RealEngine calls it from refresh_sequence_info.
- Integration test: shrink 3->1 mid-wave (all frames complete, retired
workers exit naturally) then regrow 1->3 and render a fresh wave.
Overlapping clips sum linearly in mix_audio_montage and can exceed full
scale (two hot clips reach +/-2; gain > 1 would too); the cpal sink
forwarded samples unclamped, so overlaps clipped at the DAC. Clamp the
accumulator after the montage mix (both the heap and shm-slot paths
share mix_audio_montage) and document it on render_audio_samples.
Preview now follows the project output colorspace end to end: the
display chain derives its content space from the project's OutputColorSpec
instead of a hardcoded sRGB name, self-managed ICC transforms go through
an XYZ D65 interchange stage (OCIO cie_xyz_d65_interchange) for non-sRGB
targets, and the platform layer declares the content colorspace (gpui
submodule bump). macOS defaults to OS-managed (fixes wide-gamut UI
oversaturation); Windows ACM warns once on non-sRGB targets.
Multi-monitor: the display ICC is looked up per the window's current
screen (macOS display id, Windows per-monitor DC, X11 RandR output
profile) with a throttled poll that invalidates frame caches on moves.
Pipeline precision: 10-bit+ sources fall back to YUV444P16LE + a Rust
matrix conversion when swscale lacks F32 output (no more 8-bit
truncation); BT.709/2020 SDR decodes with BT.1886 gamma 2.4 instead of
the sRGB EOTF; working-space compositing no longer clamps RGB to [0,1]
(alpha still clamped); the output node clamps to the target gamut;
frames without colorimetry metadata convert with BT.709 defaults
(warned once) instead of passing through; scopes read the
output-colorspace signal on both F32 paths.
Also: only emit rerun-if-changed for .env when it exists (a missing file
made every build fully dirty).
- oak-worker gains an LRU frame cache (default 64 MiB) keyed by the
render-deterministic spec subset; repeat frames (paused frames,
scrubs over rendered ranges, re-renders after an effect change) are
memcpy-cheap, which is what made adding an OFX plugin -- and the
in-flight batches after removing one -- stall the UI
- audio dispatch on the Processes backend now mixes inline on the UI
tick instead of queueing behind video batches in the worker pool, so
video stalls no longer starve the ~100ms cpal output buffer into
silence
Preferences gains a display-bit-depth combo stored in the config store;
at startup the app forwards it to gpui_wgpu via OAK_DISPLAY_BIT_DEPTH,
which prefers Rgb10a2Unorm for 10-bit presentation. Takes effect after
restart (noted in the dialog); i18n in all eight packs.
Match the timeline UI (V_max drawn topmost): composite tracks from
V1 up to V_max so the highest-numbered track is composited last, in
both the montage path and direct graph evaluation.
Root causes found for the 4K stalls and the second-footage memory
blowup (audit + code review):
- ticket bookkeeping leaked unbounded: the procpool ticket table and
the arena slot map only ever grew (50-100 tickets/sec during
playback, each pinning montage params and shm region views).
Completed/cancelled/superseded/crashed entries are now removed, and
the arena reaps fire-and-forget tickets once finished; the sync poll
path reaps via a terminal result() read. InFlight duplicate submits
now answer State immediately instead of sitting in the map forever.
- decode ran a full-resolution swscale to F32 RGBA (~132 MB at 4K)
plus a second full-res copy before downscaling to the 480px proxy:
RetrieveVideoParams.target_size lets swscale convert AND resize in
one pass (bilinear, matching the old Rust resampler), so a 4K
preview frame costs ~1 MB instead of ~260 MB of churn. This applies
to proxy AND full-res requests alike.
- per-process decoder cache was unbounded (each session pins an FFmpeg
context + 2 native decoded frames): LRU-capped at 16, eviction drops
the map entry (in-flight renders keep their Arc; Drop releases
FFmpeg).
- playback window completions were not generation-gated: a stale
render from before an edit landed in the rebuilt window (wrong frame
displayed, fresh request blocked). Stale completions now return
their shm slot credit instead.
- async audio prefetch used the polling ticket submit without ever
polling: switched to the fire-and-forget submit so entries reap.
Adding an effect to a clip did nothing: the sequence render is
flattened into a montage (decode + composite), and MontageClip carried
no effect data at all.
- MontageClip gains an ordered effect stack (type id / enabled /
effect input / parameter values); protocol v2 carries it as an
additive wire field (older peers default to an empty stack).
- renderops::video_montage fills the stack from the effect chain
(the footage source node — the chain end without an effect input —
is dropped; the montage decodes the footage itself). Export
(oak-task) and the multicam single-track montage fill it too.
- The worker applies the stack between decode and composite: built-in
Opacity gets a CPU evaluator (C++ opacity.frag parity — whole vec4,
alpha included, unity pass-through); everything else dispatches as an
OFX plugin job through a new instance-factory slot (oak-plugin
lazily creates + caches one instance per identifier per render
process) with the montage's parameters injected. Disabled effects
bypass (the C++ traverser pushes the effect input through). Unknown
types warn once per type id and pass through — no silent no-ops.
Not covered (explicitly): Transform/Crop and the other ~30 built-in
effects have no CPU evaluator in oak-render (they pass through with a
warning), keyframed parameter animation, audio effect chains, and the
CLI's simplified montage.
Acceptance: a real 50% Opacity on real media quarters the rendered
pixels both in-process (renderops test) and through a real worker
process over IPC + shared memory (procpool_integration test);
disabling restores the plain render byte-for-byte.
The menu item was a placeholder print; it now opens a real dialog (the
C++ ProjectPropertiesDialog):
- Per-project OCIO config override with a 浏览… picker: validated on OK
(an invalid config keeps the dialog open with the error shown, like
the C++ accept()), persisted in the project settings, applied to the
display color pipeline on accept and on project open, and reverted to
the app default when the project closes. oak-render gains
set_up_default_config_from for the explicit-path load.
- Disk-cache location (default / alongside the project / custom path):
persisted through the OVE serializer (cachesetting/customcachepath
round-trip the settings map, clamped on load) and honored by the
thumbnail writer — the first live consumer of Project::cache_path.
- PathField gains an enabled state (the custom path field follows the
combo selection).
The C++ color tab's Default Input Color Space and Reference Space
combos are intentionally absent: the Rust render pipeline has no
consumer for them today (decode performs no input transfer conversion),
so showing them would be dead settings.
Tests: dialog opens, OK applies the cache location, an invalid OCIO
config keeps the dialog open with the error row. i18n keys for all
eight packs.
convert_bgra8 applied packed u8 pixels to the default (F32-finalized)
OCIO CPU processor, which rejects them with a bit-depth mismatch; the
callers swallow the error, so the display-ICC transform was silently
inert on the shm preview path. ocio-rs exposes no Uint8-finalized CPU
processor, so the conversion now detours through F32. Verified against
the machine's actual display profile with the new
display_icc_bgra8_never_outputs_black test (OAK_DISPLAY_ICC).
Also:
- procpool_integration: audio tickets are Seek priority and claimable
by any worker now, so the shard-spread assertion goes (rendering on a
live worker is what matters).
- OAK_DEBUG_VIEWER=1: the program viewer logs frame pushes and dumps
the displayed frame to /tmp/oak_viewer_frame.ppm (the black-screen
investigation tooling).
Three compounding bugs froze the UI when dragging the playhead after
playback:
1. Self-deadlock on preview_windows: supply_preview_window /
cancel_preview_windows / cancel_preview_window called
cancel_preview_sequence / cancel_preview_frame while HOLDING the
preview_windows mutex; those calls fire completions synchronously and
the completion locks preview_windows again. Caught by sampling the
hung process: UI thread in cancel_preview_sequence -> TicketSlot::
finish -> completion -> Mutex::lock. Cancels/releases are now
collected under the lock and fired after it is dropped.
2. Seek starvation by shard pinning: a Seek request's scheduler frame
is its ticket id, pinning it to worker (id mod W). The playback
window fills every worker's slots (window slots are only released by
UI-thread consumption), so the seek's pinned worker could have zero
free slots while the UI thread blocked on the seek — permanent
starvation. Seeks (interactive frame / real-time audio) are now
claimable by ANY worker; the no-stealing shard rule stays for
Playback frames (adjacent frames finish together).
3. No per-worker reserve: the global preview_window_capacity reserve is
pool-wide accounting, but exhaustion happens per worker. Playback /
Background claims now leave one credit unused per worker; Seek
claims may use the last slot (they complete on the worker without
UI involvement).
Also: RealEngine::drop cancels the preview windows — ShmFrameRef has no
self-release, so every dropped engine leaked its window's slots from
the shared pool, starving later windows (surfaced as the full-suite
playback_window_supplies_playhead_frames failure once the new probe
test shifted the test schedule). new_sequence_has_default_two_video_
two_audio_tracks now takes the engine test lock (it asserts on the
global undo stack; running lock-free raced parallel undo histories).
New regression probe interactive_seek_renders_without_hanging: play 30
ticks (window fills and holds shm slots), pause, seek, synchronously
render — must not hang. Scheduler tests updated for the reserve and
seek-any-worker contract. OAK_DEBUG_DISPATCH=1 enables the dispatcher
starvation/pool diagnostics used to track this down.
All crates take the oak-* kebab-case naming (oak-audio, oak-codec,
oak-common, oak-core, oak-ffmpeg-link, oak-node, oak-otio, oak-plugin,
oak-render, oak-storage, oak-task, oak-timeline, oak-undo), with the
lib identifiers rewritten (oakrender:: -> oak_render::, oakcore_rs:: ->
oak_core::, ...) across all 226 referencing files.
The GUI application moves from the workspace root into
crates/oak-app/: src/, build.rs (paths fixed for the new location) and
tests/ travel with it, the root Cargo.toml becomes workspace-only
([workspace] + workspace.package + profiles), and the app package
inherits the workspace version. The screenshots example becomes a
standalone crate examples/simple_player/ with its own Cargo.toml.
Every crate now inherits the single workspace version
(version.workspace = true), and the workflows' crate paths and the
build docs follow the renames.
Validated with a clean cargo check --workspace.
Mike-Solar