The transform shader sampled in a top-left-origin pixel space while
Olive's transform semantics (and every other node) are center-origin:
rotation swung the image around the top-left corner, pushing it partly
off-frame - reading exactly like an unwanted zoom. Match the C++
transform.vert projection: position (0,0) is the frame center and
rotation/scale pivot around the anchor, so rotation and scale stay
independent user controls. GPU tests pin the 90-degree landing spot
(no smearing) and the 2x scale centroid (stays centered).
Preview renders at proxy size while a paused frame renders full-res, so
anchoring resolution_in to the render target made every sequence-pixel
effect (shape size/pos, transform offsets, corner pin points, drop
shadow distance) change apparent size whenever the transport stopped.
Pre-fill resolution_in from the sequence's video params (C++ inserts
the NodeGlobals square resolution at job-build time), covering the
nested generator job inside a merge as well; a node that inserted its
own resolution_in keeps it.
naga's WGSL emitter rejects fall-through-capable GLSL switch blocks, so
every shape and despill job failed to compile and silently fell back to
the effect input - both effects were no-ops. Rewrite the type/method
dispatch as if/else chains (same semantics as the C++ shaders) and
cover all three shape types plus green-screen despill with GPU pixel
tests. Also drop the now-stale nested-payload/merge-binding TODO notes.
- process_shader_job: bind all texture params by name, recurse into nested
shader payloads (depth cap 8), fall back to frame size without inputs
- run_effect: take iterative_input so dropshadow previous_iteration_in works
- merge: actually composite inputs; keyer mask, opacity modulation, math
texture ops and mrg generator layers now bind their textures
- transform distort: real fragment-side inverse-matrix sampling
- time offset / time remap: wire NodeBehavior time adjustment hooks
- plugin: fix first-node identity colliding with unbound sentinel
oak-common is gone; its modules (configstore, xmlutils, ocioutils,
oiioutils, colormath, colortransform, videoparams, ffmpegutils, ...)
now live in oak-core alongside the value types. The render value/GPU
types moved too: backend (wgpu context + DisplayRenderer), color
(ColorProcessor over ocio-rs), texture, frame, and the commonutil
config helpers.
Fix-ups to make the merged tree build and pass tests:
- oak-core Cargo.toml: wgpu back to 25 (the moved backend code is
written against that API generation); add the toml/quick-xml/image
deps oak-common carried.
- lib.rs: drop the duplicate 'pub mod error;'.
- error.rs: unified OAKCORE_* codes; restore Error::new() and
From<OcioError> from oak-common's error type.
- backend.rs/color.rs: oak_core::/oak_render:: self-references
rewritten to crate::; the shaderfx-dependent GPU effect test moved
to oak-render's shaderfx tests (shaderfx depends on oak-node and
cannot live in oak-core).
- oak-render's error module re-exports oak_core::error::{Error,
Result}; the OAKRENDER_* codes stay as the public-code contract.
- oak-node jobs.rs: ColorProcessor imported from oak_core::color.
- Integration tests repointed at oak_core::{texture, frame, backend,
color, colormath}.
- the display-ICC regression test treats an empty OAK_DISPLAY_ICC as
unset, matching displayicc::env_override_icc.
The OCIO grading nodes previously pushed null texture handles; they now
push real ShaderJobPayloads whose GLSL is the OCIO-generated dynamic
grading-primary GPU shader — the exact code the C++ path applies, so no
approximation:
- color: grading_primary_function_shader(style) builds a dynamic
GradingPrimaryTransform (LIN/LOG) on the default config, extracts the
GLSL via GpuShaderDesc (function 'ove_grading_primary', resource
prefix ocio_, no LUT textures) and caches it per style + config id.
- eval: OCIO_GRADING_STUBS maps the two node type ids to the grading
style; process_shader_job resolves the stub and splices it into the
node's %1 marker (same wiring as the chromakey OCIO stub); the
pipeline cache key folds the stub text so a config change recompiles.
- nodes: value() pushes a ShaderJobPayload with the C++ value()
rewrite applied to the row — vec4 (RGBM x=master) grading inputs to
the vec3 GPU uniform form (lin: contrast RGB=c*m, offset RGB=c+m,
exposure RGB=2^(c+m); log: lift RGB=c+m, gain c*m, gamma c*m), plus
pivot/saturation floats, the log pivotBlack/pivotWhite normalization
range (0/1), clamp sentinels (NoClampBlack -1 / NoClampWhite 2),
white>black enforcement per frame, and localBypass=false. Generated
uniform names bind by name (the log node's OCIO_NAMESPACE_ id text
normalizes to the ocio_ resource prefix).
- Tests: grading stub generation (analytic GLSL, cache) in color,
end-to-end GPU exposure doubling for lin (+1 stop on 0.2 gray -> 0.4)
and lift for log, node payload rewrite assertions, and the
all-shaders sweep now retries grading stubs. oak-render 181,
oak-node 441, oak-app 272 lib tests pass.
Polygon and mask previously pushed null texture handles ('fake'
implementations). They now generate real ShaderJobPayloads and render
through the existing GPU shader pipeline:
- shaderfx: std140 uniform array support (Vec4Array(N)) — the parser
accepts 'uniform <type> <name>[N];' declarations, translate()
re-emits them as vec4[N] block members with per-element std140
offsets, and pack_uniforms writes array items from the new
NodeValue::Vec4Array value, padding short arrays to the declared N.
- polygon: value() collects the inherited points array (row element
keys 'points_in[i]', else the node's own per-element values — an
unconnected array resolves to the default pentagon via
GetValueAtTime parity) and pushes a ShaderJobPayload; the 'rgb'
fragment shader rasterizes the closed point loop with an odd-even
fill in screen space (center-translated, y-flipped to match the C++
point convention) and outputs color_in inside / transparent outside.
The CPU QPainterPath generate_frame stays a documented no-op.
- mask: value() pushes a single ShaderJobPayload whose new 'mask'
fragment shader folds the whole C++ chain into one GPU pass — base
texture multiplied by the polygon matte, optional invert, and the
optional feather gaussian softens the matte during sampling (the
separable blur.frag h/v iterations as a one-pass product,
density-normalized, radius capped at 16 px).
- Tests: translate/pack array coverage in shaderfx, GPU end-to-end
rasterization of the pentagon (center white, corner transparent),
mask multiply/invert/feather on real frames, and updated oak-node
payload assertions. oak-render 178, oak-node 440, oak-app 272 lib
tests pass.
Tests run in parallel inside one process; each GraphSnapshotStore::new()
used the same 'oakrender-snapshots-<pid>' root, so one test's cleanup()
deleted another test's live snapshot — the
acquire_rewrite_forces_file_rewrite_on_same_key flake on the 16-core
Windows runner (file written, then exists() == false). Append a per-store
sequence number to the directory.
- multicamnode: type_id() assertions called the method on dyn
NodeBehavior, which method resolution routed to std::any::Any's
TypeId::of — qualify via NodeBehavior::type_id so the trait method
(the &str node type id) is compared
- procpool: the per-worker GPU budget grew a security headroom (×2)
for the CUDA-OOM flood; the two budget tests now assert against the
real formula (2 GiB + 256 MiB at 1080p24, 2 workers at 4K/24 GiB)
- cli info fixture: the fixture runs 29.97 fps; the assertion expected
30/1 (stale from the older fixture)
- MulticamPanel: playback angle refresh runs one cycle per 3 ticks
instead of re-requesting every source every tick — each angle decode
is a keyframe-scanning FFmpeg seek that stole worker capacity from
the main viewer
- PreRender frames default 120 -> 12: a window larger than what the
pool can render in real time queues far ahead of the playhead, so
the painted frame lags seconds behind (playback frozen); a smaller
window keeps the backlog bounded
- render_graph_frame: OAK_PERF clip-level timing
The per-process 'hardware session == 1, evict before every new open'
guard was turning every frame's open into a decoder re-open: after
inserting the fresh session, the NEXT frame's pre-open eviction dropped
it again, so no request ever hit the cache ([open] (request) on every
single frame, zero CACHED hits). Every decode then cost a full FFmpeg
session open (~0.5 s) + a keyframe-scanning seek — playback could never
keep up (the 'main viewer barely moves' report; [perf] showed 1.6-2.2 s
per frame).
Hardware VRAM is bounded by the LRU cap (hardware-first eviction at
MAX_CACHED_DECODERS) and the gpu-vram worker-count policy; the eager
pre-open eviction was the regression.
gpu_vram_bytes() now chains per vendor/platform:
- NVIDIA everywhere: nvidia-smi (shipped by the NVIDIA driver on every
OS) — the NVDEC path's primary device.
- AMD/Intel on Linux: the DRM mem_info_vram_total/used sysfs attributes
(amdgpu, i915, Xe). Free = total - used; the first non-zero card wins
(an iGPU without dedicated vram reports 0 and is skipped). The walk is
now testable via an injectable sysfs root; a card missing the attrs is
skipped, never aborts the walk.
- Apple Silicon: unified memory — no separate vram exists; the RAM/4
budget IS the correct bound for decode surfaces and render targets, so
no query (an explicit vram budget would double-count the same pool).
- Windows AMD/Intel: no portable CLI; DXGI QueryVideoMemoryInfo is the
real API but wgpu 25 does not expose it. Falling back to the RAM
policy is safe-side (under-sized pool loses throughput, never OOMs the
device).
Tests: the sysfs walk (fixture with a missing-attr card, a 0-total
iGPU and a discrete winner) and the per-worker budget scaling (1080p
baseline, 4K ~4x, 60 fps over-provision).
- open_hw_accel marks the device unavailable when the decoder OPEN fails
(cuvidCreateDecoder OOM at 4K) too, not just device-context creation:
without it every subsequent decoder session retried CUDA and flooded
the log per open.
- Decoder gains hardware_decoding(); the oak-render decode-session LRU
evicts hardware sessions first (each pins a GPU surface pool — ~100 MB
at 4K), so a full cache cannot exhaust video memory before the next
open.
- Worker pool count now factors GPU vram: per-worker budget = 1 GiB
(1080p peak) scaled by pixel ratio + 256 MiB idle floor, 10% reserve
of free vram; applied when hardware decoding is on (nvidia-smi query,
None otherwise falls back to the RAM/CPU policy).
- Dynamic pool resize: ProcessDispatcher::set_target_workers grows or
retires workers; retiring ones stop claiming, drain their in-flight
batch (future playback frames included), then exit naturally on the
shutdown signal — no mid-work kill (30 s deadline only as a hung-
decoder last resort). A retiring worker that dies re-queues its frames
to surviving workers. Resizes are throttled to 2 s (a resolution burst
merges; only the latest target applies) so 1080p<->4K flaps cannot
thrash process spawns.
- RenderManager::set_workspace_size announces the sequence resolution;
RealEngine calls it from refresh_sequence_info.
- Integration test: shrink 3->1 mid-wave (all frames complete, retired
workers exit naturally) then regrow 1->3 and render a fresh wave.
- Project explorer header gains a 新建序列 button (opens the existing
new-sequence dialog, seeded like the menu action).
- Sequence presets gain 4K UHD (3840x2160@25) and 4K DCI (4096x2160@24);
the sequence-properties dialog re-selects them on reopen. Format fields
are also seedable from a probed footage format (the drop flow).
- Chroma Key (and Color Difference Key) value() now box a ShaderJobPayload
like Despill: the old OCIO-processor gate pushed nothing (the processor
is never populated without the render bridge), so the traverser handed
the clip NodeValue::None and the rendered frame lost the clip. The
renderer resolves the OCIO stub at compile time from OCIO_SHADER_STUBS.
End-to-end graph test: green key on a green frame keys out, red key
keeps it.
- The OFX color picker's SV palette / hue bar / preview / swatch canvases
get size_full(): the bare canvases collapsed to zero height in the
block layout, so the palette painted nothing (the reported 色板没显示).
Regression test clicks the palette center and expects mid s/v.
retrieve_audio_to decodes whole codec frames but copies only the part
inside the requested chunk; the tail of the frame crossing the chunk end
(up to 1023 samples for AAC) was consumed by the decoder and lost, so
the next chunk started with a hole. On the playback grid (1920 samples
at 25fps/48kHz) the hole cycles 128..896 samples and hits 7 of 8 chunk
starts -- the heavy stutter/noise heard during playback.
Keep the overflow (decode and resampler-flush tails) in a per-session
carry buffer and serve it at the start of the next contiguous chunk;
clear it on seek, format change and chunk failure. Also make seek()
actually drop the cached resampler as its comment claimed.
Verified sample-exact: chunked decode of a 440Hz tone now matches a
one-shot decode bit for bit, and chunked renders of real media line up
with the ffmpeg CLI reference at correlation 1.0 / drift 0.
Adds a regression test (playback_sized_chunks_match_oneshot_sample_exact)
with a new tone fixture -- demo.mp4's audio is -90dB digital silence and
cannot expose the holes -- plus a render_audio_wav example used to
diagnose chunk-boundary artifacts offline.
Overlapping clips sum linearly in mix_audio_montage and can exceed full
scale (two hot clips reach +/-2; gain > 1 would too); the cpal sink
forwarded samples unclamped, so overlaps clipped at the DAC. Clamp the
accumulator after the montage mix (both the heap and shm-slot paths
share mix_audio_montage) and document it on render_audio_samples.
Preview now follows the project output colorspace end to end: the
display chain derives its content space from the project's OutputColorSpec
instead of a hardcoded sRGB name, self-managed ICC transforms go through
an XYZ D65 interchange stage (OCIO cie_xyz_d65_interchange) for non-sRGB
targets, and the platform layer declares the content colorspace (gpui
submodule bump). macOS defaults to OS-managed (fixes wide-gamut UI
oversaturation); Windows ACM warns once on non-sRGB targets.
Multi-monitor: the display ICC is looked up per the window's current
screen (macOS display id, Windows per-monitor DC, X11 RandR output
profile) with a throttled poll that invalidates frame caches on moves.
Pipeline precision: 10-bit+ sources fall back to YUV444P16LE + a Rust
matrix conversion when swscale lacks F32 output (no more 8-bit
truncation); BT.709/2020 SDR decodes with BT.1886 gamma 2.4 instead of
the sRGB EOTF; working-space compositing no longer clamps RGB to [0,1]
(alpha still clamped); the output node clamps to the target gamut;
frames without colorimetry metadata convert with BT.709 defaults
(warned once) instead of passing through; scopes read the
output-colorspace signal on both F32 paths.
Also: only emit rerun-if-changed for .env when it exists (a missing file
made every build fully dirty).
- oak-worker gains an LRU frame cache (default 64 MiB) keyed by the
render-deterministic spec subset; repeat frames (paused frames,
scrubs over rendered ranges, re-renders after an effect change) are
memcpy-cheap, which is what made adding an OFX plugin -- and the
in-flight batches after removing one -- stall the UI
- audio dispatch on the Processes backend now mixes inline on the UI
tick instead of queueing behind video batches in the worker pool, so
video stalls no longer starve the ~100ms cpal output buffer into
silence
Preferences gains a display-bit-depth combo stored in the config store;
at startup the app forwards it to gpui_wgpu via OAK_DISPLAY_BIT_DEPTH,
which prefers Rgb10a2Unorm for 10-bit presentation. Takes effect after
restart (noted in the dialog); i18n in all eight packs.
Match the timeline UI (V_max drawn topmost): composite tracks from
V1 up to V_max so the highest-numbered track is composited last, in
both the montage path and direct graph evaluation.
Root causes found for the 4K stalls and the second-footage memory
blowup (audit + code review):
- ticket bookkeeping leaked unbounded: the procpool ticket table and
the arena slot map only ever grew (50-100 tickets/sec during
playback, each pinning montage params and shm region views).
Completed/cancelled/superseded/crashed entries are now removed, and
the arena reaps fire-and-forget tickets once finished; the sync poll
path reaps via a terminal result() read. InFlight duplicate submits
now answer State immediately instead of sitting in the map forever.
- decode ran a full-resolution swscale to F32 RGBA (~132 MB at 4K)
plus a second full-res copy before downscaling to the 480px proxy:
RetrieveVideoParams.target_size lets swscale convert AND resize in
one pass (bilinear, matching the old Rust resampler), so a 4K
preview frame costs ~1 MB instead of ~260 MB of churn. This applies
to proxy AND full-res requests alike.
- per-process decoder cache was unbounded (each session pins an FFmpeg
context + 2 native decoded frames): LRU-capped at 16, eviction drops
the map entry (in-flight renders keep their Arc; Drop releases
FFmpeg).
- playback window completions were not generation-gated: a stale
render from before an edit landed in the rebuilt window (wrong frame
displayed, fresh request blocked). Stale completions now return
their shm slot credit instead.
- async audio prefetch used the polling ticket submit without ever
polling: switched to the fire-and-forget submit so entries reap.
Adding an effect to a clip did nothing: the sequence render is
flattened into a montage (decode + composite), and MontageClip carried
no effect data at all.
- MontageClip gains an ordered effect stack (type id / enabled /
effect input / parameter values); protocol v2 carries it as an
additive wire field (older peers default to an empty stack).
- renderops::video_montage fills the stack from the effect chain
(the footage source node — the chain end without an effect input —
is dropped; the montage decodes the footage itself). Export
(oak-task) and the multicam single-track montage fill it too.
- The worker applies the stack between decode and composite: built-in
Opacity gets a CPU evaluator (C++ opacity.frag parity — whole vec4,
alpha included, unity pass-through); everything else dispatches as an
OFX plugin job through a new instance-factory slot (oak-plugin
lazily creates + caches one instance per identifier per render
process) with the montage's parameters injected. Disabled effects
bypass (the C++ traverser pushes the effect input through). Unknown
types warn once per type id and pass through — no silent no-ops.
Not covered (explicitly): Transform/Crop and the other ~30 built-in
effects have no CPU evaluator in oak-render (they pass through with a
warning), keyframed parameter animation, audio effect chains, and the
CLI's simplified montage.
Acceptance: a real 50% Opacity on real media quarters the rendered
pixels both in-process (renderops test) and through a real worker
process over IPC + shared memory (procpool_integration test);
disabling restores the plain render byte-for-byte.
The menu item was a placeholder print; it now opens a real dialog (the
C++ ProjectPropertiesDialog):
- Per-project OCIO config override with a 浏览… picker: validated on OK
(an invalid config keeps the dialog open with the error shown, like
the C++ accept()), persisted in the project settings, applied to the
display color pipeline on accept and on project open, and reverted to
the app default when the project closes. oak-render gains
set_up_default_config_from for the explicit-path load.
- Disk-cache location (default / alongside the project / custom path):
persisted through the OVE serializer (cachesetting/customcachepath
round-trip the settings map, clamped on load) and honored by the
thumbnail writer — the first live consumer of Project::cache_path.
- PathField gains an enabled state (the custom path field follows the
combo selection).
The C++ color tab's Default Input Color Space and Reference Space
combos are intentionally absent: the Rust render pipeline has no
consumer for them today (decode performs no input transfer conversion),
so showing them would be dead settings.
Tests: dialog opens, OK applies the cache location, an invalid OCIO
config keeps the dialog open with the error row. i18n keys for all
eight packs.
convert_bgra8 applied packed u8 pixels to the default (F32-finalized)
OCIO CPU processor, which rejects them with a bit-depth mismatch; the
callers swallow the error, so the display-ICC transform was silently
inert on the shm preview path. ocio-rs exposes no Uint8-finalized CPU
processor, so the conversion now detours through F32. Verified against
the machine's actual display profile with the new
display_icc_bgra8_never_outputs_black test (OAK_DISPLAY_ICC).
Also:
- procpool_integration: audio tickets are Seek priority and claimable
by any worker now, so the shard-spread assertion goes (rendering on a
live worker is what matters).
- OAK_DEBUG_VIEWER=1: the program viewer logs frame pushes and dumps
the displayed frame to /tmp/oak_viewer_frame.ppm (the black-screen
investigation tooling).
Three compounding bugs froze the UI when dragging the playhead after
playback:
1. Self-deadlock on preview_windows: supply_preview_window /
cancel_preview_windows / cancel_preview_window called
cancel_preview_sequence / cancel_preview_frame while HOLDING the
preview_windows mutex; those calls fire completions synchronously and
the completion locks preview_windows again. Caught by sampling the
hung process: UI thread in cancel_preview_sequence -> TicketSlot::
finish -> completion -> Mutex::lock. Cancels/releases are now
collected under the lock and fired after it is dropped.
2. Seek starvation by shard pinning: a Seek request's scheduler frame
is its ticket id, pinning it to worker (id mod W). The playback
window fills every worker's slots (window slots are only released by
UI-thread consumption), so the seek's pinned worker could have zero
free slots while the UI thread blocked on the seek — permanent
starvation. Seeks (interactive frame / real-time audio) are now
claimable by ANY worker; the no-stealing shard rule stays for
Playback frames (adjacent frames finish together).
3. No per-worker reserve: the global preview_window_capacity reserve is
pool-wide accounting, but exhaustion happens per worker. Playback /
Background claims now leave one credit unused per worker; Seek
claims may use the last slot (they complete on the worker without
UI involvement).
Also: RealEngine::drop cancels the preview windows — ShmFrameRef has no
self-release, so every dropped engine leaked its window's slots from
the shared pool, starving later windows (surfaced as the full-suite
playback_window_supplies_playhead_frames failure once the new probe
test shifted the test schedule). new_sequence_has_default_two_video_
two_audio_tracks now takes the engine test lock (it asserts on the
global undo stack; running lock-free raced parallel undo histories).
New regression probe interactive_seek_renders_without_hanging: play 30
ticks (window fills and holds shm slots), pause, seek, synchronously
render — must not hang. Scheduler tests updated for the reserve and
seek-any-worker contract. OAK_DEBUG_DISPATCH=1 enables the dispatcher
starvation/pool diagnostics used to track this down.
All crates take the oak-* kebab-case naming (oak-audio, oak-codec,
oak-common, oak-core, oak-ffmpeg-link, oak-node, oak-otio, oak-plugin,
oak-render, oak-storage, oak-task, oak-timeline, oak-undo), with the
lib identifiers rewritten (oakrender:: -> oak_render::, oakcore_rs:: ->
oak_core::, ...) across all 226 referencing files.
The GUI application moves from the workspace root into
crates/oak-app/: src/, build.rs (paths fixed for the new location) and
tests/ travel with it, the root Cargo.toml becomes workspace-only
([workspace] + workspace.package + profiles), and the app package
inherits the workspace version. The screenshots example becomes a
standalone crate examples/simple_player/ with its own Cargo.toml.
Every crate now inherits the single workspace version
(version.workspace = true), and the workflows' crate paths and the
build docs follow the renames.
Validated with a clean cargo check --workspace.
Mike-Solar