Four independent CI failures the first real cross-platform run surfaced:
- macOS SIGSEGV: the real-GL unit tests in gl_bridge ran wherever CGL is
available, including the headless CI runner. Gate them with the same
OAK_GPU_TESTS switch the integration GL tests already use (skip on CI,
opt in on a real Mac).
- Windows build: examples compile under `cargo test`, and bench_playback
used libc::getrusage unconditionally. Keep the Unix CPU accounting
behind #[cfg(unix)] and report zero CPU seconds elsewhere.
- openKylin arm64: engine_without_a_project_hits_the_guard_paths assumed
the library backend was unconfigured while a parallel config test
transiently set Storage/Backend=sqlite. Take the shared config lock and
pin the key off for the test's duration.
- openKylin x64: the prefetch smoke test asserted an exact decode count,
but the hand-off LRU holds only DECODE_LRU_CAP (2) frames, so a request
can miss the prefetched copy under scheduling pressure and re-run the
producer (the eval cache still serves the pixels). Bound the count
instead of pinning it; the deterministic sibling test pins read-ahead
usage.
The macOS job finally reached the build (after the vcpkg manifest fix) and
hit a macOS-only compile error in the VideoToolbox import: `*ptr as
*const T` parses as `(*ptr) as *const T`, so `sw_format` was read off a
pointer instead of the AVHWFramesContext. Bind the frames pointer first.
Also fix the warnings the cross-check surfaced: the redundant
MTLPixelFormat import, and doc comments on an extern block and a
thread_local! (rustdoc does not document those).
Verified locally with a host-cc wrapper:
`cargo check -p oak-core -p oak-codec -p oak-node -p oak-render
-p oak-task -p oak-plugin --target aarch64-apple-darwin` is clean.
(oak-app itself needs a real Apple toolchain for ring.)
- ForceParams: hand-written Default with force_format = -1 (was 0 = U8,
which pushed the F32 pipeline into the U8 scale path).
- Plugin clip output: write CPU pixels back into the target texture
instead of the deep clone returned by texture_get_frame.
- Display ICC: probe the Debian/Ubuntu icc-profiles-free path.
- RippleInfo: public constructor and accessors so the ripple command is
reachable from integration tests.
- MockEngine: record effect-parameter and push-button attempts so the
params-view routing tests are falsifiable.
- OFX params: log rejected parameter writes instead of discarding them.
- Manager docs: state the synchronous codec-submission contract.
Adds VAAPI DMA-BUF, D3D11VA shared-handle and VideoToolbox IOSurface
imports behind a tri-state outcome (imported / unsupported / failed),
planar textures with bounded residency and a CPU staging fallback, the
staged montage decode path, reference-counted decoder frames, VAAPI-first
device selection on Linux, and the host-GPU context plumbing used by the
app and worker. See docs/zh/plans/render-pipeline-threads.md (M5).
- Mark the raw-pointer interop entry points unsafe with # Safety docs
(oak-core upload/download/frame-from-pixels, oak-audio convert) and
satisfy the existing callers (tests).
- mut_from_ref: allow with the ABI contract documented (the handle
get_mut helpers in oak-timeline/oak-render/oak-task take the shared
reference the C ABI passes; exclusivity is the caller's unsafe
contract).
- Fix the eq_op in the white-balance normalization (green / green).
- Apply cargo clippy --fix across the workspace (redundant closures and
field names, field reassignment, items after test modules, ...).
- Revert the replace_box fix in image_effect's clip_define: a
redefinition must allocate a new box, otherwise the old clip handle
stays valid and the HS-map replace contract (clip != clip2) breaks.
- 283 warnings remain; they are all non-machine-applicable
(chunks_exact -> as_chunks needs a manual iter_mut, too_many_arguments,
complex types, missing Safety docs, ...) and are tracked as the
follow-up.
docs/zh/plans/render-pipeline-threads.md M3 (design 3.2): OpenFX crash
isolation moves from "every worker hosts plugins" to a single dedicated
host process, served over NDJSON + shared memory.
- oak-worker --ofx-host mode (src/ofx_host.rs): loads every plugin once,
resolves jobs by the cross-process-stable OFX identifier, and renders
through the same in-process executor the workers used to install.
- oak-render/ofxhost.rs: the single-host client. The render manager
creates and installs it for the Pipeline backend (lazy spawn on the
first plugin job); eval::process_plugin_job prefers it and falls back
to the in-process executor otherwise, so the process backend keeps its
current behavior until M4.
- Data plane: input/output FrameSlotPool pairs (the handshake's input_*
fields are used for the first time). Named clips and the source frame
are written to input slots after the explicit CPU readback; the plugin
output returns through an output slot. Pool size/capacity grow by a
host restart when a job needs more (safe: submissions are serialized
and one job is in flight).
- Crash loop: reader EOF fails the in-flight submit, which respawns the
host and re-posts the same job (frames are read back once); after three
consecutive crashes the client is permanently dead and the evaluator
falls back to a purple frame. The dead child is reaped immediately, and
a submit mutex enforces the one-job-in-flight contract.
- Progress/cancel: the host flushes plugin_progress immediately (live
progress), and reads stdin on its own thread so plugin_cancel takes
effect mid-render at the plugin's next progressUpdate; the sticky flag
resets at progressStart and request_plugin_cancel_all broadcasts to
both the worker pool and the host.
- JobSpec::Plugin / PluginJobPayload carry the plugin type_id (stable
across processes); `--ofx-crash-once` / `--ofx-crash-always` are the
deterministic crash hooks, matching the worker's env hooks.
- Tests: wire round-trips; host unit tests (crash budget, cancel-flag
reset through the factory, source mapping); oak-worker integration
tests against the real host + bundled test plugin (render + progress,
crash respawn and re-post, three-crash give-up, mid-render cancel on
the new slow variant, concurrent submits); eval's purple fallback.
254 warnings (320 counting replayed-cache re-emitters) cleaned:
unused mut/imports/variables, irrefutable if-lets and unreachable
patterns, dead code removed or annotated #[allow(dead_code)] with
the reason (C++ parity value sets, cfg(test) helpers, public API
reservations), drop(&ref) no-ops removed, fn-pointer identity via
std::ptr::fn_addr_eq, the test-stubs feature declared in
oak-node's manifest, missing docs filled. Every unused-Result site
was judged individually: meaningful errors propagate, intentional
ignores are let _ = with a note.
Two pre-existing latent bugs are documented in place, behavior
preserved: app.rs's timeline-tool observer and dialogs.rs's format
subscription both drop the returned Subscription immediately, so
they never fire.
- process_shader_job: bind all texture params by name, recurse into nested
shader payloads (depth cap 8), fall back to frame size without inputs
- run_effect: take iterative_input so dropshadow previous_iteration_in works
- merge: actually composite inputs; keyer mask, opacity modulation, math
texture ops and mrg generator layers now bind their textures
- transform distort: real fragment-side inverse-matrix sampling
- time offset / time remap: wire NodeBehavior time adjustment hooks
- plugin: fix first-node identity colliding with unbound sentinel
oak-common is gone; its modules (configstore, xmlutils, ocioutils,
oiioutils, colormath, colortransform, videoparams, ffmpegutils, ...)
now live in oak-core alongside the value types. The render value/GPU
types moved too: backend (wgpu context + DisplayRenderer), color
(ColorProcessor over ocio-rs), texture, frame, and the commonutil
config helpers.
Fix-ups to make the merged tree build and pass tests:
- oak-core Cargo.toml: wgpu back to 25 (the moved backend code is
written against that API generation); add the toml/quick-xml/image
deps oak-common carried.
- lib.rs: drop the duplicate 'pub mod error;'.
- error.rs: unified OAKCORE_* codes; restore Error::new() and
From<OcioError> from oak-common's error type.
- backend.rs/color.rs: oak_core::/oak_render:: self-references
rewritten to crate::; the shaderfx-dependent GPU effect test moved
to oak-render's shaderfx tests (shaderfx depends on oak-node and
cannot live in oak-core).
- oak-render's error module re-exports oak_core::error::{Error,
Result}; the OAKRENDER_* codes stay as the public-code contract.
- oak-node jobs.rs: ColorProcessor imported from oak_core::color.
- Integration tests repointed at oak_core::{texture, frame, backend,
color, colormath}.
- the display-ICC regression test treats an empty OAK_DISPLAY_ICC as
unset, matching displayicc::env_override_icc.
Preview now follows the project output colorspace end to end: the
display chain derives its content space from the project's OutputColorSpec
instead of a hardcoded sRGB name, self-managed ICC transforms go through
an XYZ D65 interchange stage (OCIO cie_xyz_d65_interchange) for non-sRGB
targets, and the platform layer declares the content colorspace (gpui
submodule bump). macOS defaults to OS-managed (fixes wide-gamut UI
oversaturation); Windows ACM warns once on non-sRGB targets.
Multi-monitor: the display ICC is looked up per the window's current
screen (macOS display id, Windows per-monitor DC, X11 RandR output
profile) with a throttled poll that invalidates frame caches on moves.
Pipeline precision: 10-bit+ sources fall back to YUV444P16LE + a Rust
matrix conversion when swscale lacks F32 output (no more 8-bit
truncation); BT.709/2020 SDR decodes with BT.1886 gamma 2.4 instead of
the sRGB EOTF; working-space compositing no longer clamps RGB to [0,1]
(alpha still clamped); the output node clamps to the target gamut;
frames without colorimetry metadata convert with BT.709 defaults
(warned once) instead of passing through; scopes read the
output-colorspace signal on both F32 paths.
Also: only emit rerun-if-changed for .env when it exists (a missing file
made every build fully dirty).
The pipeline is ACEScg + F32 end to end by design; a plugin that does
not support F32 must not fail the render — its inputs convert down to
the negotiated depth and its output converts back to F32 (the previous
"Phase 2 F32 only" error path purple-framed those plugins).
- render_driver maps getClipPreferences' output bit depth to
Byte/Short/Half/Float, sets it on the input/output clips, allocates
the output image at the negotiated depth and converts the result
back to F32 for frame assembly (both the CPU path and the
GL-failure CPU fallback)
- the GL path keeps F32 clip params: GL textures are created in the
pipeline format and kOfxOpenGLPropPixelDepth is negotiated
separately, so clip props must match what the plugin actually sees
- fetch_image converts the decoded input down to the clip's
negotiated depth (default F32)
- new Image::convert_depth ([0,1]-normalized conversion between all
depth pairs) plus round-trip and f16 edge tests; f16 helpers moved
into image.rs and shared with clip.rs
Three independent host-side gaps kept real plugins from instantiating:
- clips now define OfxImageClipPropFieldOrder (default OfxFieldNone):
ofxs Clip::getFieldOrder() is a strong read with no default, so
field-aware plugins (Mirror) threw PropertyUnknownToHost ->
MissingHostFeature from createInstance
- OfxParamPropParametricUIColour is no longer predefined as an empty
array: the OFX implicit-create semantics let the plugin's first
propSetDouble create the property and grow it index by index, while
the empty predefined array rejected every write at index >= 1 with
BadIndex (ColorLookup, HueCorrect describeInContext)
- Host::create_instance_preferred: context selection with fallback
(filter -> general -> tracker -> paint -> rest). TrackerPM advertises
the filter context but its createInstance fetches a Mask clip that
its describe only defines for tracker/general/paint; Natron simply
instantiates it in the tracker context, and now so do we. Both the
node-registration scan and the shared instance factory use it.
Still unsupported, by design: Premult/Unpremult (this openfx-misc
build hard-requires the Nuke multi-plane suite + dynamic choices) and
the stereo view plugins (Switch/anaglyph/joinViews/etc. need Natron's
isNatron multi-clip folding or view rendering).
Verified: ColorLookup, HueCorrect, Mirror, TrackerPM now instantiate;
full oak-plugin suite green.
The OFX abort contract is "0 = keep rendering, anything else = stop"
and the ofxs support library literally checks abort(...) != 0. Our
host answered kOfxStatReplyNo (13) for "not cancelled", so every
openfx-misc processor aborted its pixel loop before writing a single
row: the render action returned OK with the output buffer untouched —
a solid black frame everywhere an effect was applied (the previous
smoke test only asserted "not the purple failure frame", which let
this slip through as a fake pass).
- image-effect abort: return 0 when not cancelled, 1 when cancelled
(instance cancel flag or progress cancellation), BadHandle for
descriptor handles as before
- strengthen the real-plugin smoke tests to also reject an all-zero
output frame, so a silent no-write can no longer pass
- more trace-gated [ofx] diagnostics (value dumps for propGetN /
propGetPointer / clipGetImage fetches, multiThread entry, render
output buffer address) — the tooling that pinned this down
Verified with lldb watchpoints: the AddOFX/ChromaKeyerOFX processors
now write the real pixels through the executor path.
Real plugins (CImg ChromaKeyerOFX, AddOFX) failed the render action with
kOfxStatFailed / MissingHostFeature and painted the magenta failure frame:
- images lacked the mandatory ImageBase properties (OfxPropType,
PixelAspectRatio, PreMultiplication, Field, RenderScale); the ofxs
ImageBase constructor throws on the missing/invalid strong reads
- PreMultiplication used the made-up string "OfxImagePreMultiplied";
kOfxImagePreMultiplied is actually "OfxImageAlphaPremultiplied", the
only value mapStrToPreMultiplicationEnum accepts (lldb __cxa_throw
backtrace pinpointed this)
- RenderWindow is Int x4 per ofxsPropertyValidation, not Double x4
- field strings use the real constant "OfxFieldNone"
- clips define OfxImageClipPropConnected (isConnected is a strong read;
optional mask clips blew up without it)
- choice params predefine empty ChoiceEnum / ChoiceLabelOption arrays
- isIdentity failure is no longer fatal (the C++ plugin renderer never
calls it; plugins that error on it simply render normally)
- property suite coerces Int <-> Double on reads (the CImg framework
reads the render window with propGetIntN against a Double store)
- in-args carry NatronOfxPropNativeOverlays=0 for the Natron framework
- plugin jobs pass a GL-kind marker so GL-only plugins take the real
gl_bridge offscreen path instead of the CPU MissingHostFeature path
- trace-gated [ofx] diagnostics for property misses and suite calls
Verified with new smoke tests that render the real AddOFX and
ChromaKeyerOFX plugins through the executor and assert the output is
not the purple failure frame.
Adding an effect to a clip did nothing: the sequence render is
flattened into a montage (decode + composite), and MontageClip carried
no effect data at all.
- MontageClip gains an ordered effect stack (type id / enabled /
effect input / parameter values); protocol v2 carries it as an
additive wire field (older peers default to an empty stack).
- renderops::video_montage fills the stack from the effect chain
(the footage source node — the chain end without an effect input —
is dropped; the montage decodes the footage itself). Export
(oak-task) and the multicam single-track montage fill it too.
- The worker applies the stack between decode and composite: built-in
Opacity gets a CPU evaluator (C++ opacity.frag parity — whole vec4,
alpha included, unity pass-through); everything else dispatches as an
OFX plugin job through a new instance-factory slot (oak-plugin
lazily creates + caches one instance per identifier per render
process) with the montage's parameters injected. Disabled effects
bypass (the C++ traverser pushes the effect input through). Unknown
types warn once per type id and pass through — no silent no-ops.
Not covered (explicitly): Transform/Crop and the other ~30 built-in
effects have no CPU evaluator in oak-render (they pass through with a
warning), keyframed parameter animation, audio effect chains, and the
CLI's simplified montage.
Acceptance: a real 50% Opacity on real media quarters the rendered
pixels both in-process (renderops test) and through a real worker
process over IPC + shared memory (procpool_integration test);
disabling restores the plain render byte-for-byte.
Every test_project() node packs the same NodeId::identity (a fresh
graph's first node), so two tests running in parallel collide in the
process-wide registry and one resolves the other's live entry —
identity_project_dropped's dangling-upgrade assert then fails
intermittently on CI. Both identity tests now share a mutex.
All crates take the oak-* kebab-case naming (oak-audio, oak-codec,
oak-common, oak-core, oak-ffmpeg-link, oak-node, oak-otio, oak-plugin,
oak-render, oak-storage, oak-task, oak-timeline, oak-undo), with the
lib identifiers rewritten (oakrender:: -> oak_render::, oakcore_rs:: ->
oak_core::, ...) across all 226 referencing files.
The GUI application moves from the workspace root into
crates/oak-app/: src/, build.rs (paths fixed for the new location) and
tests/ travel with it, the root Cargo.toml becomes workspace-only
([workspace] + workspace.package + profiles), and the app package
inherits the workspace version. The screenshots example becomes a
standalone crate examples/simple_player/ with its own Cargo.toml.
Every crate now inherits the single workspace version
(version.workspace = true), and the workflows' crate paths and the
build docs follow the renames.
Validated with a clean cargo check --workspace.
Mike-Solar