render: apply clip effect stacks in the montage path
Adding an effect to a clip did nothing: the sequence render is flattened into a montage (decode + composite), and MontageClip carried no effect data at all. - MontageClip gains an ordered effect stack (type id / enabled / effect input / parameter values); protocol v2 carries it as an additive wire field (older peers default to an empty stack). - renderops::video_montage fills the stack from the effect chain (the footage source node — the chain end without an effect input — is dropped; the montage decodes the footage itself). Export (oak-task) and the multicam single-track montage fill it too. - The worker applies the stack between decode and composite: built-in Opacity gets a CPU evaluator (C++ opacity.frag parity — whole vec4, alpha included, unity pass-through); everything else dispatches as an OFX plugin job through a new instance-factory slot (oak-plugin lazily creates + caches one instance per identifier per render process) with the montage's parameters injected. Disabled effects bypass (the C++ traverser pushes the effect input through). Unknown types warn once per type id and pass through — no silent no-ops. Not covered (explicitly): Transform/Crop and the other ~30 built-in effects have no CPU evaluator in oak-render (they pass through with a warning), keyframed parameter animation, audio effect chains, and the CLI's simplified montage. Acceptance: a real 50% Opacity on real media quarters the rendered pixels both in-process (renderops test) and through a real worker process over IPC + shared memory (procpool_integration test); disabling restores the plain render byte-for-byte.
This commit is contained in:
@@ -142,6 +142,38 @@ fn clip_preview_media(
|
||||
Some(preview_footage_media(f, is_video))
|
||||
}
|
||||
|
||||
/// The clip block's effect stack as montage effect descriptors
|
||||
/// (source-first order — the chain walk's signal order, so the renderer
|
||||
/// applies them media-side first). The chain walk runs all the way to
|
||||
/// the media source, so its first element is the footage node feeding
|
||||
/// the clip; the montage decodes that footage itself, so the source
|
||||
/// node (the one WITHOUT an effect input) is dropped here. Disabled
|
||||
/// effects are carried with `enabled = false`; the renderer bypasses
|
||||
/// them (the C++ traverser's bypass pushes the effect input through
|
||||
/// unchanged). Parameters are the inspector's parameter set (non-hidden,
|
||||
/// non-connection inputs at their standard values; keyframed values are
|
||||
/// not time-resolved on this path).
|
||||
fn clip_effects(g: &oak_node::graph::Graph, block_id: NodeId) -> Vec<oak_render::ticket::MontageEffect> {
|
||||
use super::effectchain;
|
||||
effectchain::chain(g, block_id)
|
||||
.into_iter()
|
||||
.filter(|&fx| effectchain::effect_input_of(g, fx).is_some())
|
||||
.filter_map(|fx| {
|
||||
let entry = g.get(fx)?;
|
||||
Some(oak_render::ticket::MontageEffect {
|
||||
type_id: entry.behavior.type_id().to_string(),
|
||||
enabled: effectchain::is_enabled(g, fx),
|
||||
effect_input_id: effectchain::effect_input_of(g, fx),
|
||||
params: effectchain::effect_params(g, fx)
|
||||
.unwrap_or_default()
|
||||
.into_iter()
|
||||
.map(|p| (p.input_id, p.value))
|
||||
.collect(),
|
||||
})
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// The video montage at sequence time `time`: every clip covering `time`
|
||||
/// on video tracks, ordered bottom-to-top (track index 0 is topmost, so
|
||||
/// it is composited last). Hidden tracks (the muted flag doubles as the
|
||||
@@ -187,6 +219,7 @@ pub fn video_montage(p: &ProjectRef, seq: NodeId, time: Rational) -> Vec<Montage
|
||||
out_time: out,
|
||||
media_in: clip.core.media_in,
|
||||
gain: 1.0,
|
||||
effects: clip_effects(&g.graph, block_id),
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -244,6 +277,7 @@ pub fn single_track_video_montage(
|
||||
out_time: out,
|
||||
media_in: clip.core.media_in,
|
||||
gain: 1.0,
|
||||
effects: clip_effects(&g.graph, block_id),
|
||||
});
|
||||
}
|
||||
clips
|
||||
@@ -335,6 +369,9 @@ pub fn audio_montage(p: &ProjectRef, seq: NodeId, range: TimeRange) -> Vec<Monta
|
||||
out_time: out,
|
||||
media_in: clip.core.media_in,
|
||||
gain: 1.0,
|
||||
// Audio clips: the effect stack is a video-side
|
||||
// concept; the mixer never consults it.
|
||||
effects: Vec::new(),
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -907,6 +944,126 @@ mod tests {
|
||||
let _ = std::fs::remove_file(&media);
|
||||
}
|
||||
|
||||
/// The acceptance gate for montage effect rendering: a 50% Opacity on
|
||||
/// a real clip (generated media, real graph, real decode, real
|
||||
/// compositing — nothing mocked) must change the rendered pixels, and
|
||||
/// disabling the effect must restore the plain render byte-for-byte.
|
||||
#[test]
|
||||
fn montage_effect_stack_reaches_the_rendered_pixels() {
|
||||
let _media = media_lock();
|
||||
oak_undo::global::clear().unwrap();
|
||||
let media =
|
||||
std::env::temp_dir().join(format!("oakapp_montage_fx_{}.mp4", std::process::id()));
|
||||
oak_codec::testmedia::write_test_clip(&media, 64, 64, 10, 10).expect("generate test media");
|
||||
|
||||
let project = graphops::create_project();
|
||||
let seq = graphops::create_sequence(&project, "Effect Montage");
|
||||
let footage = graphops::import_footage(&project, &media).expect("import the generated media");
|
||||
graphops::add_track(&project, seq, TrackType::Video).expect("add a video track");
|
||||
let block = graphops::place_footage_clip(&project, seq, footage, TrackType::Video, 0, 0, 10, 0)
|
||||
.expect("place the clip");
|
||||
|
||||
// The effect-library double-click flow: append a 50% Opacity to the
|
||||
// footage-backed clip (the insert rewires footage -> effect -> clip).
|
||||
let fx = crate::oakui::effectchain::insert(
|
||||
&project,
|
||||
block,
|
||||
usize::MAX,
|
||||
"org.olivevideoeditor.Olive.opacity",
|
||||
)
|
||||
.expect("append the opacity effect");
|
||||
crate::oakui::effectchain::set_input_value(
|
||||
&project,
|
||||
fx,
|
||||
"opacity_in",
|
||||
oak_node::value::NodeValue::Float(0.5),
|
||||
)
|
||||
.expect("set the opacity value");
|
||||
|
||||
let tb = graphops::sequence_time_base(&lock(&project).graph, seq).unwrap();
|
||||
let time = graphops::ts_to_rational(0, tb);
|
||||
|
||||
// Render the montage through the same entry point the render worker
|
||||
// uses (`render_montage_frame_into`, F32 RGBA rows).
|
||||
let render = |montage: Vec<MontageClip>| {
|
||||
let params = VideoTicketParams {
|
||||
viewer: 0,
|
||||
time,
|
||||
force_size: Some((64, 64)),
|
||||
force_format: Some(oak_core::PixelFormat::F32),
|
||||
cache: None,
|
||||
cache_dir: None,
|
||||
cache_id: None,
|
||||
cache_timebase: None,
|
||||
footage: None,
|
||||
montage,
|
||||
};
|
||||
let mut dst = vec![0u8; 64 * 64 * 16];
|
||||
oak_render::eval::render_montage_frame_into(time, ¶ms, (64, 64), &mut dst, 64 * 16)
|
||||
.expect("montage render");
|
||||
dst
|
||||
};
|
||||
// A left-half pixel of the test pattern (known content, r ~= 0.9).
|
||||
let pixel = |frame: &[u8]| {
|
||||
let off = (8 * 64 + 8) * 16;
|
||||
[0, 1, 2, 3].map(|i| f32::from_le_bytes(frame[off + i * 4..off + i * 4 + 4].try_into().unwrap()))
|
||||
};
|
||||
|
||||
let with_fx = video_montage(&project, seq, time);
|
||||
assert_eq!(with_fx.len(), 1, "one clip covers frame 0");
|
||||
assert_eq!(
|
||||
with_fx[0].effects.len(),
|
||||
1,
|
||||
"the montage carries the clip's effect stack (not the footage source node)"
|
||||
);
|
||||
assert_eq!(with_fx[0].effects[0].type_id, "org.olivevideoeditor.Olive.opacity");
|
||||
assert!(with_fx[0].effects[0].enabled);
|
||||
|
||||
let plain = render(
|
||||
with_fx
|
||||
.iter()
|
||||
.cloned()
|
||||
.map(|mut c| {
|
||||
c.effects.clear();
|
||||
c
|
||||
})
|
||||
.collect(),
|
||||
);
|
||||
let effected = render(with_fx);
|
||||
let (p, e) = (pixel(&plain), pixel(&effected));
|
||||
assert!(p[0] > 0.6, "the plain render shows the test pattern (r={})", p[0]);
|
||||
// 50% opacity: the shader halves every channel, then the composite
|
||||
// over transparent black halves it again via the halved alpha.
|
||||
assert!(
|
||||
(e[0] - p[0] * 0.25).abs() < 0.1,
|
||||
"50% opacity quarters the output ({} vs {})",
|
||||
e[0],
|
||||
p[0] * 0.25
|
||||
);
|
||||
assert!(
|
||||
(e[3] - 0.5).abs() < 0.1,
|
||||
"the composited alpha halves (a={})",
|
||||
e[3]
|
||||
);
|
||||
|
||||
// Disabled effect = bypass: pixels match the plain render.
|
||||
crate::oakui::effectchain::set_enabled(&project, fx, false).expect("disable the effect");
|
||||
let disabled_montage = video_montage(&project, seq, time);
|
||||
assert_eq!(disabled_montage[0].effects.len(), 1);
|
||||
assert!(!disabled_montage[0].effects[0].enabled);
|
||||
let disabled = render(disabled_montage);
|
||||
let d = pixel(&disabled);
|
||||
assert!(
|
||||
(d[0] - p[0]).abs() < 0.05,
|
||||
"a disabled effect leaves the pixels alone ({} vs {})",
|
||||
d[0],
|
||||
p[0]
|
||||
);
|
||||
|
||||
oak_undo::global::clear().unwrap();
|
||||
let _ = std::fs::remove_file(&media);
|
||||
}
|
||||
|
||||
/// The original media decodes from the footage's actual first stream of
|
||||
/// the kind (not the hardcoded 0/1 of a typical layout): a file whose
|
||||
/// video stream is not stream 0 must still decode its own video when
|
||||
|
||||
@@ -535,6 +535,9 @@ pub fn video_montage(p: &ProjectRef, seq_id: NodeId, time: Rational) -> Vec<Mont
|
||||
out_time: out,
|
||||
media_in: clip.core.media_in,
|
||||
gain: 1.0,
|
||||
// The CLI's simplified montage builder does not resolve
|
||||
// effect chains (unchanged behavior).
|
||||
effects: Vec::new(),
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -585,6 +588,7 @@ pub fn audio_montage(p: &ProjectRef, seq_id: NodeId, range: TimeRange) -> Vec<Mo
|
||||
out_time: out,
|
||||
media_in: clip.core.media_in,
|
||||
gain: 1.0,
|
||||
effects: Vec::new(),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -927,10 +927,47 @@ static EXECUTOR_INSTALLED: OnceLock<()> = OnceLock::new();
|
||||
pub fn install_render_executor() {
|
||||
EXECUTOR_INSTALLED.get_or_init(|| {
|
||||
oak_render::eval::set_plugin_executor(Some(Arc::new(execute_plugin_job)));
|
||||
oak_render::eval::set_plugin_instance_factory(Some(Arc::new(shared_plugin_instance)));
|
||||
oak_node::nodes::plugin::set_plugin_duplicator(Some(Arc::new(duplicate_instance)));
|
||||
});
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// montage 渲染路径的共享实例(渲染进程按插件标识惰性创建 + 进程级缓存)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// montage 效果栈携带插件标识而非实例 id(montage 在主进程从时间线
|
||||
/// 解析,实例活在渲染进程)。渲染进程经此工厂按需创建实例并缓存:
|
||||
/// 参数每次渲染前由 montage 注入([`execute_plugin_job`] 的 values
|
||||
/// 覆盖),worker 单线程顺序渲染,共享实例无跨帧状态冲突。
|
||||
fn shared_plugin_instance(identifier: &str) -> Option<u64> {
|
||||
static SHARED: OnceLock<Mutex<HashMap<String, u64>>> = OnceLock::new();
|
||||
let cache = SHARED.get_or_init(|| Mutex::new(HashMap::new()));
|
||||
if let Some(&id) = cache
|
||||
.lock()
|
||||
.unwrap_or_else(|e| e.into_inner())
|
||||
.get(identifier)
|
||||
{
|
||||
return Some(id);
|
||||
}
|
||||
let host = Host::global();
|
||||
let plugin = host.cache.find(identifier)?;
|
||||
// 上下文选择与 [`register_plugin_nodes`] 一致(filter 优先,否则
|
||||
// 首个支持上下文;factory.cpp:171-177)。
|
||||
let context = if plugin.contexts.iter().any(|c| c == "OfxImageEffectContextFilter") {
|
||||
"OfxImageEffectContextFilter".to_string()
|
||||
} else {
|
||||
plugin.contexts.first()?.clone()
|
||||
};
|
||||
let inst = host.create_instance(identifier, Some(&context)).ok()?;
|
||||
let id = register_instance(inst);
|
||||
cache
|
||||
.lock()
|
||||
.unwrap_or_else(|e| e.into_inner())
|
||||
.insert(identifier.to_string(), id);
|
||||
Some(id)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 测试
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -1085,6 +1122,13 @@ mod tests {
|
||||
unregister_instance(u64::MAX);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn shared_instance_unknown_identifier_yields_none() {
|
||||
// 缓存里查无此标识 → None(montage 路径据此按"无求值器"
|
||||
// 告警并直通)。
|
||||
assert!(shared_plugin_instance("com.example.definitely-missing").is_none());
|
||||
}
|
||||
|
||||
/// 构造一个只含 push-button 参数的最小实例(直接登记进注册表)。
|
||||
fn instance_with_push_button() -> u64 {
|
||||
use std::ffi::{c_char, c_void};
|
||||
|
||||
@@ -149,6 +149,36 @@ pub fn plugin_executor() -> Option<Arc<PluginExecutor>> {
|
||||
.clone()
|
||||
}
|
||||
|
||||
/// Plugin instance factory: resolves an OFX plugin identifier to a live
|
||||
/// instance-registry id, creating (and caching) the instance lazily.
|
||||
/// Implemented by the oakplugin crate — the montage effect path carries
|
||||
/// plugin identifiers, not instance ids (the montage is resolved from
|
||||
/// the timeline in the main process; the instance lives in whichever
|
||||
/// process renders the frame).
|
||||
pub type PluginInstanceFactory = dyn Fn(&str) -> Option<u64> + Send + Sync;
|
||||
|
||||
static PLUGIN_INSTANCE_FACTORY: std::sync::OnceLock<
|
||||
std::sync::Mutex<Option<Arc<PluginInstanceFactory>>>,
|
||||
> = std::sync::OnceLock::new();
|
||||
|
||||
fn instance_factory_slot() -> &'static std::sync::Mutex<Option<Arc<PluginInstanceFactory>>> {
|
||||
PLUGIN_INSTANCE_FACTORY.get_or_init(|| std::sync::Mutex::new(None))
|
||||
}
|
||||
|
||||
/// Install the plugin instance factory (oakplugin registration point;
|
||||
/// `None` clears it).
|
||||
pub fn set_plugin_instance_factory(factory: Option<Arc<PluginInstanceFactory>>) {
|
||||
*instance_factory_slot().lock().unwrap_or_else(|e| e.into_inner()) = factory;
|
||||
}
|
||||
|
||||
/// The installed plugin instance factory, if any.
|
||||
pub fn plugin_instance_factory() -> Option<Arc<PluginInstanceFactory>> {
|
||||
instance_factory_slot()
|
||||
.lock()
|
||||
.unwrap_or_else(|e| e.into_inner())
|
||||
.clone()
|
||||
}
|
||||
|
||||
/// The failure marker frame: solid magenta (1, 0, 1, 1) F32 RGBA —
|
||||
/// the C++ plugin renderer paints failed plugin output purple so a
|
||||
/// broken plugin is visible instead of silently black.
|
||||
@@ -775,7 +805,11 @@ pub fn render_montage_frame_into(
|
||||
(w, h),
|
||||
PixelFormat::F32,
|
||||
)?;
|
||||
let (src_data, src_stride) = match &decoded {
|
||||
// The clip's effect stack runs between decode and compositing
|
||||
// (C++ semantics: the clip texture passes through the chain
|
||||
// bottom-up, the chain top feeds the track composite).
|
||||
let effected = apply_clip_effects(decoded, clip, time);
|
||||
let (src_data, src_stride) = match &effected {
|
||||
Texture::Cpu(src) => (&src.data, src.linesize_bytes() as i32),
|
||||
_ => continue,
|
||||
};
|
||||
@@ -825,6 +859,139 @@ pub fn composite_over(
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Montage clip effect stacks
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// The built-in Opacity effect's type id (oaknode `OpacityEffect`) — the
|
||||
/// one built-in video effect with a CPU evaluator on the montage path.
|
||||
const OPACITY_EFFECT_TYPE_ID: &str = "org.olivevideoeditor.Olive.opacity";
|
||||
|
||||
/// The Opacity effect's value input id (oaknode `opacity_in`).
|
||||
const OPACITY_VALUE_INPUT: &str = "opacity_in";
|
||||
|
||||
/// The effect type ids the montage path already warned about (one log
|
||||
/// line per type per process instead of one per frame).
|
||||
fn unsupported_warned() -> std::sync::MutexGuard<'static, std::collections::HashSet<String>> {
|
||||
static WARNED: std::sync::OnceLock<std::sync::Mutex<std::collections::HashSet<String>>> =
|
||||
std::sync::OnceLock::new();
|
||||
WARNED
|
||||
.get_or_init(|| std::sync::Mutex::new(std::collections::HashSet::new()))
|
||||
.lock()
|
||||
.unwrap_or_else(|e| e.into_inner())
|
||||
}
|
||||
|
||||
/// Log an unsupported-effect passthrough once per type id.
|
||||
fn warn_unsupported_once(type_id: &str, reason: &str) {
|
||||
if unsupported_warned().insert(type_id.to_string()) {
|
||||
eprintln!("montage effect \"{type_id}\" passes through unchanged: {reason}");
|
||||
}
|
||||
}
|
||||
|
||||
/// Run a clip's effect stack over its decoded frame (source-first order;
|
||||
/// disabled effects are bypassed — the C++ traverser's bypass pushes the
|
||||
/// effect input through unchanged). Effects the montage path cannot
|
||||
/// evaluate log a warning once and pass the frame through.
|
||||
fn apply_clip_effects(
|
||||
src: Texture,
|
||||
clip: &crate::ticket::MontageClip,
|
||||
time: Rational,
|
||||
) -> Texture {
|
||||
let mut tex = src;
|
||||
for effect in &clip.effects {
|
||||
if !effect.enabled {
|
||||
continue;
|
||||
}
|
||||
tex = apply_montage_effect(tex, effect, time);
|
||||
}
|
||||
tex
|
||||
}
|
||||
|
||||
/// Apply one effect to `src` (an F32 RGBA CPU frame of the montage
|
||||
/// pipeline). `time` is the sequence time the frame is rendered at (the
|
||||
/// C++ node evaluation time).
|
||||
fn apply_montage_effect(
|
||||
src: Texture,
|
||||
effect: &crate::ticket::MontageEffect,
|
||||
time: Rational,
|
||||
) -> Texture {
|
||||
// Built-in Opacity: multiply every channel by the opacity factor
|
||||
// (C++ `:/shaders/opacity.frag`: `frag_color = texture(tex_in, …) *
|
||||
// opacity_in` — the shader scales the whole vec4, alpha included).
|
||||
if effect.type_id == OPACITY_EFFECT_TYPE_ID {
|
||||
let factor = effect
|
||||
.params
|
||||
.iter()
|
||||
.find(|(id, _)| id == OPACITY_VALUE_INPUT)
|
||||
.map(|(_, v)| v.to_double())
|
||||
.unwrap_or(1.0);
|
||||
// Unity is a pass-through (C++ `qFuzzyCompare(opacity, 1.0)`).
|
||||
if (factor - 1.0).abs() * 1e12 <= factor.abs().min(1.0) {
|
||||
return src;
|
||||
}
|
||||
// Texture implements Drop, so scale in place through a mutable
|
||||
// borrow instead of moving the frame out.
|
||||
let mut tex = src;
|
||||
match &mut tex {
|
||||
Texture::Cpu(frame) => {
|
||||
let stride = frame.linesize_bytes();
|
||||
for row in frame.data.chunks_exact_mut(stride).take(frame.height.max(0) as usize) {
|
||||
for px in row[..(frame.width.max(0) as usize) * 16].chunks_exact_mut(16) {
|
||||
for c in px.chunks_exact_mut(4) {
|
||||
let v = f32::from_le_bytes(c.try_into().unwrap());
|
||||
c.copy_from_slice(&(v * factor as f32).to_le_bytes());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
_ => {
|
||||
warn_unsupported_once(
|
||||
&effect.type_id,
|
||||
"opacity on a non-CPU texture is not supported by the montage path",
|
||||
);
|
||||
}
|
||||
}
|
||||
return tex;
|
||||
}
|
||||
|
||||
// Everything else: an OFX plugin effect. The montage carries the
|
||||
// plugin identifier; the rendering process resolves it to a live
|
||||
// instance through the oakplugin-installed factory (lazily created
|
||||
// and cached per identifier), then dispatches through the plugin
|
||||
// executor exactly like the graph path's plugin jobs.
|
||||
let Some(factory) = plugin_instance_factory() else {
|
||||
warn_unsupported_once(
|
||||
&effect.type_id,
|
||||
"no plugin instance factory installed (oakplugin init missing in this process)",
|
||||
);
|
||||
return src;
|
||||
};
|
||||
let Some(instance) = factory(&effect.type_id) else {
|
||||
warn_unsupported_once(
|
||||
&effect.type_id,
|
||||
"no evaluator: unknown built-in effect or OFX plugin unavailable in this process",
|
||||
);
|
||||
return src;
|
||||
};
|
||||
let spec = JobSpec::Plugin {
|
||||
instance,
|
||||
time: time.to_f64(),
|
||||
effect_input_id: effect.effect_input_id.clone(),
|
||||
inputs: Vec::new(),
|
||||
values: effect.params.clone(),
|
||||
};
|
||||
let size = src.size();
|
||||
match RenderEvalHooks::new().process_plugin_job(src, &spec) {
|
||||
Ok(texture) => texture,
|
||||
Err(err) => {
|
||||
// Unreachable for a Plugin spec (the executor failure path
|
||||
// yields a purple frame); stay loud rather than silent.
|
||||
eprintln!("montage plugin job failed to dispatch: {err:#}");
|
||||
purple_frame(time, size)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
@@ -1165,4 +1332,118 @@ mod tests {
|
||||
let mut dst = [0u8; 8]; // far too small for 2000x2 f32 samples
|
||||
assert!(render_audio_samples_into(¶ms, &mut dst).is_err());
|
||||
}
|
||||
|
||||
// ---- Montage clip effect stacks -------------------------------------
|
||||
|
||||
/// A 2x1 F32 texture filled with a known color.
|
||||
fn solid_texture(r: f32, g: f32, b: f32, a: f32) -> Texture {
|
||||
let mut frame = generate_frame(Rational::new(0, 1), (2, 1), PixelFormat::F32).unwrap();
|
||||
for px in frame.data.chunks_exact_mut(16) {
|
||||
for (c, v) in px.chunks_exact_mut(4).zip([r, g, b, a]) {
|
||||
c.copy_from_slice(&v.to_le_bytes());
|
||||
}
|
||||
}
|
||||
Texture::Cpu(frame)
|
||||
}
|
||||
|
||||
fn opacity_effect(enabled: bool, value: f64) -> crate::ticket::MontageEffect {
|
||||
crate::ticket::MontageEffect {
|
||||
type_id: OPACITY_EFFECT_TYPE_ID.to_string(),
|
||||
enabled,
|
||||
effect_input_id: Some("tex_in".to_string()),
|
||||
params: vec![(OPACITY_VALUE_INPUT.to_string(), NodeValue::Float(value))],
|
||||
}
|
||||
}
|
||||
|
||||
fn clip_with_effects(effects: Vec<crate::ticket::MontageEffect>) -> crate::ticket::MontageClip {
|
||||
crate::ticket::MontageClip {
|
||||
filename: String::new(),
|
||||
stream_index: 0,
|
||||
in_time: Rational::new(0, 1),
|
||||
out_time: Rational::new(1, 1),
|
||||
media_in: Rational::new(0, 1),
|
||||
gain: 1.0,
|
||||
effects,
|
||||
}
|
||||
}
|
||||
|
||||
/// The built-in Opacity effect really scales the decoded pixels
|
||||
/// (every channel, C++ `opacity.frag` parity).
|
||||
#[test]
|
||||
fn montage_opacity_effect_scales_pixels() {
|
||||
let clip = clip_with_effects(vec![opacity_effect(true, 0.5)]);
|
||||
let out = apply_clip_effects(solid_texture(0.8, 0.4, 0.2, 1.0), &clip, Rational::new(0, 1));
|
||||
assert_eq!(first_pixel(&out), [0.4, 0.2, 0.1, 0.5]);
|
||||
}
|
||||
|
||||
/// Disabled effects are bypassed (C++ traverser parity), and unity
|
||||
/// opacity is a pass-through.
|
||||
#[test]
|
||||
fn montage_disabled_or_unity_effects_pass_through() {
|
||||
let disabled = clip_with_effects(vec![opacity_effect(false, 0.5)]);
|
||||
let out = apply_clip_effects(solid_texture(0.8, 0.4, 0.2, 1.0), &disabled, Rational::new(0, 1));
|
||||
assert_eq!(first_pixel(&out), [0.8, 0.4, 0.2, 1.0]);
|
||||
|
||||
let unity = clip_with_effects(vec![opacity_effect(true, 1.0)]);
|
||||
let out = apply_clip_effects(solid_texture(0.8, 0.4, 0.2, 1.0), &unity, Rational::new(0, 1));
|
||||
assert_eq!(first_pixel(&out), [0.8, 0.4, 0.2, 1.0]);
|
||||
}
|
||||
|
||||
/// An OFX effect (any non-built-in type id) resolves its instance
|
||||
/// through the installed factory and dispatches through the executor,
|
||||
/// carrying the montage's parameter values.
|
||||
#[test]
|
||||
fn montage_plugin_effect_dispatches_with_params() {
|
||||
let _guard = PLUGIN_TEST_LOCK.lock().unwrap();
|
||||
set_plugin_instance_factory(Some(Arc::new(|identifier: &str| {
|
||||
(identifier == "com.example.darken").then_some(7)
|
||||
})));
|
||||
set_plugin_executor(Some(Arc::new(|req: &PluginJobRequest<'_>| {
|
||||
let JobSpec::Plugin { instance, values, .. } = req.spec else {
|
||||
return Err(Error::Invalid);
|
||||
};
|
||||
assert_eq!(*instance, 7);
|
||||
// The injected parameter drives the output: paint the frame
|
||||
// with the "gain" value so the test observes the param path.
|
||||
let gain = values
|
||||
.iter()
|
||||
.find(|(k, _)| k == "gain")
|
||||
.map(|(_, v)| v.to_double() as f32)
|
||||
.unwrap_or(1.0);
|
||||
let mut frame = generate_frame(Rational::new(0, 1), req.src.size(), PixelFormat::F32)?;
|
||||
for px in frame.data.chunks_exact_mut(16) {
|
||||
for (c, v) in px.chunks_exact_mut(4).zip([gain, gain, gain, 1.0]) {
|
||||
c.copy_from_slice(&v.to_le_bytes());
|
||||
}
|
||||
}
|
||||
Ok(Texture::Cpu(frame))
|
||||
})));
|
||||
let clip = clip_with_effects(vec![crate::ticket::MontageEffect {
|
||||
type_id: "com.example.darken".to_string(),
|
||||
enabled: true,
|
||||
effect_input_id: Some("Source".to_string()),
|
||||
params: vec![("gain".to_string(), NodeValue::Float(0.25))],
|
||||
}]);
|
||||
let out = apply_clip_effects(solid_texture(0.8, 0.4, 0.2, 1.0), &clip, Rational::new(0, 1));
|
||||
assert_eq!(first_pixel(&out), [0.25, 0.25, 0.25, 1.0]);
|
||||
set_plugin_executor(None);
|
||||
set_plugin_instance_factory(None);
|
||||
}
|
||||
|
||||
/// An effect nobody can evaluate (no factory / unknown type) passes
|
||||
/// the frame through unchanged — loudly (the warn-once log), never a
|
||||
/// silent no-op.
|
||||
#[test]
|
||||
fn montage_unknown_effect_passes_through() {
|
||||
let _guard = PLUGIN_TEST_LOCK.lock().unwrap();
|
||||
set_plugin_instance_factory(None);
|
||||
let clip = clip_with_effects(vec![crate::ticket::MontageEffect {
|
||||
type_id: "com.example.missing".to_string(),
|
||||
enabled: true,
|
||||
effect_input_id: Some("Source".to_string()),
|
||||
params: Vec::new(),
|
||||
}]);
|
||||
let out = apply_clip_effects(solid_texture(0.8, 0.4, 0.2, 1.0), &clip, Rational::new(0, 1));
|
||||
assert_eq!(first_pixel(&out), [0.8, 0.4, 0.2, 1.0]);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -272,6 +272,122 @@ pub struct HelloCapsMsg {
|
||||
pub max_slot_bytes: i64,
|
||||
}
|
||||
|
||||
/// A node value on the wire (protocol v2, montage effect parameters).
|
||||
/// `oak_node::value::NodeValue` itself is not serde-able (texture/sample
|
||||
/// payloads, handles); this enum covers the plain-data variants an effect
|
||||
/// parameter can carry. Connection/handle variants (texture, samples,
|
||||
/// node refs, video/audio params, push buttons) have no wire
|
||||
/// representation and are dropped by [`WireNodeValue::from_node_value`].
|
||||
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq)]
|
||||
#[serde(tag = "t", content = "v", rename_all = "snake_case")]
|
||||
pub enum WireNodeValue {
|
||||
/// No value.
|
||||
None,
|
||||
/// Integer.
|
||||
Int(i64),
|
||||
/// Float.
|
||||
Float(f64),
|
||||
/// RGBA color.
|
||||
Color([f64; 4]),
|
||||
/// Text.
|
||||
Text(String),
|
||||
/// Boolean.
|
||||
Boolean(bool),
|
||||
/// Rational (num, den).
|
||||
Rational(i64, i64),
|
||||
/// Vec2.
|
||||
Vec2([f64; 2]),
|
||||
/// Vec3.
|
||||
Vec3([f64; 3]),
|
||||
/// Vec4.
|
||||
Vec4([f64; 4]),
|
||||
/// 4x4 matrix, row-major.
|
||||
Matrix([f64; 16]),
|
||||
/// Combo index.
|
||||
Combo(i64),
|
||||
/// String combo.
|
||||
StrCombo(String),
|
||||
/// Opaque bytes.
|
||||
Binary(Vec<u8>),
|
||||
}
|
||||
|
||||
impl Default for WireNodeValue {
|
||||
fn default() -> Self {
|
||||
WireNodeValue::None
|
||||
}
|
||||
}
|
||||
|
||||
impl WireNodeValue {
|
||||
/// The wire form of a node value, or `None` when the variant has no
|
||||
/// wire representation (the parameter is then not carried).
|
||||
pub fn from_node_value(v: &oak_node::value::NodeValue) -> Option<WireNodeValue> {
|
||||
use oak_node::value::NodeValue as NV;
|
||||
Some(match v {
|
||||
NV::None => WireNodeValue::None,
|
||||
NV::Int(i) => WireNodeValue::Int(*i),
|
||||
NV::Float(f) => WireNodeValue::Float(*f),
|
||||
NV::Color(c) => WireNodeValue::Color(*c),
|
||||
NV::Text(s) => WireNodeValue::Text(s.clone()),
|
||||
NV::Boolean(b) => WireNodeValue::Boolean(*b),
|
||||
NV::Rational(r) => WireNodeValue::Rational(r.numerator(), r.denominator()),
|
||||
NV::Vec2(v2) => WireNodeValue::Vec2(*v2),
|
||||
NV::Vec3(v3) => WireNodeValue::Vec3(*v3),
|
||||
NV::Vec4(v4) => WireNodeValue::Vec4(*v4),
|
||||
NV::Matrix(m) => WireNodeValue::Matrix(*m),
|
||||
NV::Combo(i) => WireNodeValue::Combo(*i),
|
||||
NV::StrCombo(s) => WireNodeValue::StrCombo(s.clone()),
|
||||
NV::Binary(b) => WireNodeValue::Binary(b.clone()),
|
||||
_ => return None,
|
||||
})
|
||||
}
|
||||
|
||||
/// Back to a node value (worker side).
|
||||
pub fn to_node_value(&self) -> oak_node::value::NodeValue {
|
||||
use oak_node::value::NodeValue as NV;
|
||||
match self {
|
||||
WireNodeValue::None => NV::None,
|
||||
WireNodeValue::Int(i) => NV::Int(*i),
|
||||
WireNodeValue::Float(f) => NV::Float(*f),
|
||||
WireNodeValue::Color(c) => NV::Color(*c),
|
||||
WireNodeValue::Text(s) => NV::Text(s.clone()),
|
||||
WireNodeValue::Boolean(b) => NV::Boolean(*b),
|
||||
WireNodeValue::Rational(n, d) => NV::Rational(oak_core::Rational::new(*n, *d)),
|
||||
WireNodeValue::Vec2(v2) => NV::Vec2(*v2),
|
||||
WireNodeValue::Vec3(v3) => NV::Vec3(*v3),
|
||||
WireNodeValue::Vec4(v4) => NV::Vec4(*v4),
|
||||
WireNodeValue::Matrix(m) => NV::Matrix(*m),
|
||||
WireNodeValue::Combo(i) => NV::Combo(*i),
|
||||
WireNodeValue::StrCombo(s) => NV::StrCombo(s.clone()),
|
||||
WireNodeValue::Binary(b) => NV::Binary(b.clone()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One effect parameter on the wire (input id + value).
|
||||
#[derive(Serialize, Deserialize, Default, Debug, Clone, PartialEq)]
|
||||
#[serde(default)]
|
||||
pub struct WireEffectParam {
|
||||
/// Node input id.
|
||||
pub input: String,
|
||||
/// Parameter value.
|
||||
pub value: WireNodeValue,
|
||||
}
|
||||
|
||||
/// One effect of a montage clip's effect stack on the wire (protocol v2
|
||||
/// additive field of [`WireMontageClip`]; source-first order).
|
||||
#[derive(Serialize, Deserialize, Default, Debug, Clone, PartialEq)]
|
||||
#[serde(default)]
|
||||
pub struct WireMontageEffect {
|
||||
/// Built-in node type id or OFX plugin identifier.
|
||||
pub type_id: String,
|
||||
/// Enabled flag (disabled effects are bypassed).
|
||||
pub enabled: bool,
|
||||
/// Effect input (clip) name; "" = none.
|
||||
pub effect_input_id: String,
|
||||
/// Parameter values.
|
||||
pub params: Vec<WireEffectParam>,
|
||||
}
|
||||
|
||||
/// One montage clip on the wire (rationals flattened to num/den pairs).
|
||||
#[derive(Serialize, Deserialize, Default, Debug, Clone, PartialEq)]
|
||||
#[serde(default)]
|
||||
@@ -294,6 +410,47 @@ pub struct WireMontageClip {
|
||||
pub media_in_den: i64,
|
||||
/// Playback gain (1.0 = unity).
|
||||
pub gain: f32,
|
||||
/// The clip's effect stack (protocol v2 additive: older peers omit the
|
||||
/// field and it defaults to an empty stack).
|
||||
pub effects: Vec<WireMontageEffect>,
|
||||
}
|
||||
|
||||
/// Map a ticket-side montage effect to its wire form (main process;
|
||||
/// parameters without a wire representation are dropped).
|
||||
pub fn wire_effect_from(effect: &crate::ticket::MontageEffect) -> WireMontageEffect {
|
||||
WireMontageEffect {
|
||||
type_id: effect.type_id.clone(),
|
||||
enabled: effect.enabled,
|
||||
effect_input_id: effect.effect_input_id.clone().unwrap_or_default(),
|
||||
params: effect
|
||||
.params
|
||||
.iter()
|
||||
.filter_map(|(input, value)| {
|
||||
WireNodeValue::from_node_value(value).map(|value| WireEffectParam {
|
||||
input: input.clone(),
|
||||
value,
|
||||
})
|
||||
})
|
||||
.collect(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Map a wire effect back to the ticket-side form (worker).
|
||||
pub fn montage_effect_from(wire: &WireMontageEffect) -> crate::ticket::MontageEffect {
|
||||
crate::ticket::MontageEffect {
|
||||
type_id: wire.type_id.clone(),
|
||||
enabled: wire.enabled,
|
||||
effect_input_id: if wire.effect_input_id.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(wire.effect_input_id.clone())
|
||||
},
|
||||
params: wire
|
||||
.params
|
||||
.iter()
|
||||
.map(|p| (p.input.clone(), p.value.to_node_value()))
|
||||
.collect(),
|
||||
}
|
||||
}
|
||||
|
||||
/// One frame ticket inside a [`RenderBatchMsg`] — the main process
|
||||
@@ -1710,6 +1867,15 @@ mod tests {
|
||||
media_in_num: 10,
|
||||
media_in_den: 24,
|
||||
gain: 0.5,
|
||||
effects: vec![WireMontageEffect {
|
||||
type_id: "org.olivevideoeditor.Olive.opacity".into(),
|
||||
enabled: true,
|
||||
effect_input_id: "tex_in".into(),
|
||||
params: vec![WireEffectParam {
|
||||
input: "opacity_in".into(),
|
||||
value: WireNodeValue::Float(0.5),
|
||||
}],
|
||||
}],
|
||||
}],
|
||||
},
|
||||
],
|
||||
@@ -1721,6 +1887,15 @@ mod tests {
|
||||
assert_eq!(value["tickets"][0]["format"], SLOT_FORMAT_BGRA8);
|
||||
assert_eq!(value["tickets"][1]["montage"][0]["filename"], "b.mp4");
|
||||
assert_eq!(value["tickets"][1]["montage"][0]["media_in_num"], 10);
|
||||
// The effect stack rides the clip as an additive v2 field.
|
||||
assert_eq!(
|
||||
value["tickets"][1]["montage"][0]["effects"][0]["type_id"],
|
||||
"org.olivevideoeditor.Olive.opacity"
|
||||
);
|
||||
assert_eq!(
|
||||
value["tickets"][1]["montage"][0]["effects"][0]["params"][0]["value"],
|
||||
json!({ "t": "float", "v": 0.5 })
|
||||
);
|
||||
// Round-trip back to the struct.
|
||||
let round: RenderBatchMsg = serde_json::from_value(value).unwrap();
|
||||
assert_eq!(round, batch);
|
||||
@@ -1733,6 +1908,49 @@ mod tests {
|
||||
assert!(bare.montage.is_empty());
|
||||
}
|
||||
|
||||
/// Protocol v2 compatibility: a v1-shaped montage clip (no `effects`
|
||||
/// field) parses with an empty effect stack, and the node-value wire
|
||||
/// enum round-trips every plain-data variant.
|
||||
#[test]
|
||||
fn montage_clip_effects_are_additive_and_values_roundtrip() {
|
||||
let legacy = json!({
|
||||
"filename": "a.mp4",
|
||||
"stream_index": 0,
|
||||
"in_num": 0, "in_den": 24,
|
||||
"out_num": 48, "out_den": 24,
|
||||
"media_in_num": 0, "media_in_den": 24,
|
||||
"gain": 1.0,
|
||||
});
|
||||
let parsed: WireMontageClip = serde_json::from_value(legacy).unwrap();
|
||||
assert!(parsed.effects.is_empty(), "v1 clips carry no effects");
|
||||
|
||||
use oak_node::value::NodeValue as NV;
|
||||
let cases = [
|
||||
NV::None,
|
||||
NV::Int(-3),
|
||||
NV::Float(0.25),
|
||||
NV::Color([0.1, 0.2, 0.3, 1.0]),
|
||||
NV::Text("hello".into()),
|
||||
NV::Boolean(true),
|
||||
NV::Rational(oak_core::Rational::new(1, 24)),
|
||||
NV::Vec2([1.0, 2.0]),
|
||||
NV::Vec3([1.0, 2.0, 3.0]),
|
||||
NV::Vec4([1.0, 2.0, 3.0, 4.0]),
|
||||
NV::Matrix([0.0; 16]),
|
||||
NV::Combo(2),
|
||||
NV::StrCombo("choice".into()),
|
||||
NV::Binary(vec![1, 2, 3]),
|
||||
];
|
||||
for value in cases {
|
||||
let wire = WireNodeValue::from_node_value(&value).expect("plain data has a wire form");
|
||||
let json = serde_json::to_string(&wire).unwrap();
|
||||
let back: WireNodeValue = serde_json::from_str(&json).unwrap();
|
||||
assert_eq!(back.to_node_value(), value, "wire round-trip of {value:?}");
|
||||
}
|
||||
// Connection/handle variants have no wire form.
|
||||
assert!(WireNodeValue::from_node_value(&NV::PushButton).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn batch_accepted_and_frame_failed_wire_roundtrip() {
|
||||
let accepted = BatchAcceptedMsg {
|
||||
@@ -1790,6 +2008,7 @@ mod tests {
|
||||
media_in_num: 10,
|
||||
media_in_den: 24,
|
||||
gain: 0.5,
|
||||
effects: vec![],
|
||||
}],
|
||||
}],
|
||||
};
|
||||
|
||||
@@ -1936,6 +1936,7 @@ fn build_ticket_spec(
|
||||
media_in_num: c.media_in.numerator(),
|
||||
media_in_den: c.media_in.denominator(),
|
||||
gain: c.gain,
|
||||
effects: c.effects.iter().map(crate::ipc::wire_effect_from).collect(),
|
||||
})
|
||||
.collect();
|
||||
BatchTicketSpec {
|
||||
@@ -1970,6 +1971,7 @@ fn build_audio_ticket_spec(ticket: i64, slot: u32, params: &AudioTicketParams) -
|
||||
media_in_num: c.media_in.numerator(),
|
||||
media_in_den: c.media_in.denominator(),
|
||||
gain: c.gain,
|
||||
effects: c.effects.iter().map(crate::ipc::wire_effect_from).collect(),
|
||||
})
|
||||
.collect();
|
||||
AudioTicketSpec {
|
||||
|
||||
@@ -35,6 +35,31 @@ use crate::eval;
|
||||
use crate::texture::Texture;
|
||||
use crate::worker::{JobDispatch, JobSchedule};
|
||||
|
||||
/// One effect of a montage clip's effect stack (M14 R3 effect-chain
|
||||
/// wiring into the montage render path). The stack is ordered
|
||||
/// source-first (the chain walk's signal order: the first element feeds
|
||||
/// the media side, the last feeds the clip's effect input); the renderer
|
||||
/// applies them in sequence after decode, before compositing.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct MontageEffect {
|
||||
/// Node type id: the built-in factory type id (e.g.
|
||||
/// `org.olivevideoeditor.Olive.opacity`) or the OFX plugin identifier.
|
||||
pub type_id: String,
|
||||
/// The effect's enabled flag. Disabled effects are bypassed (the C++
|
||||
/// traverser's bypass pushes the effect input through unchanged), so
|
||||
/// the renderer skips them; they are carried anyway so the render side
|
||||
/// can log what it skipped.
|
||||
pub enabled: bool,
|
||||
/// The clip/input name the source texture arrives on (the node's
|
||||
/// effect input id — "tex_in" for built-ins, "Source" for typical OFX
|
||||
/// filters; `None` when the node has no effect input).
|
||||
pub effect_input_id: Option<String>,
|
||||
/// Parameter values (input id -> value): the node's non-hidden,
|
||||
/// non-connection data inputs at their standard (non-keyframed)
|
||||
/// values — the same parameter set the inspector exposes.
|
||||
pub params: Vec<(String, oak_node::value::NodeValue)>,
|
||||
}
|
||||
|
||||
/// One clip of a sequence montage (M12 P0): the facade resolves the
|
||||
/// timeline into an ordered list of clips; the producer decodes each and
|
||||
/// composites them topmost-last.
|
||||
@@ -52,6 +77,9 @@ pub struct MontageClip {
|
||||
pub media_in: Rational,
|
||||
/// Playback gain (1.0 = unity).
|
||||
pub gain: f32,
|
||||
/// The clip's effect stack (source-first; empty for audio clips and
|
||||
/// for montage builders that do not resolve effect chains).
|
||||
pub effects: Vec<MontageEffect>,
|
||||
}
|
||||
|
||||
/// Audio ticket parameters (M12 P1): the output format plus the audio
|
||||
|
||||
@@ -61,7 +61,7 @@ use oak_render::procpool::{
|
||||
bgra8_to_f32_rgba, DispatcherConfig, ProcessDispatcher, ShmAudioRef, ShmFrameRef,
|
||||
};
|
||||
use oak_render::ticket::{
|
||||
ticket_kind, AudioTicketParams, MontageClip, TicketArena, TicketId, TicketPayload,
|
||||
ticket_kind, AudioTicketParams, MontageClip, MontageEffect, TicketArena, TicketId, TicketPayload,
|
||||
TicketResult, VideoTicketParams,
|
||||
};
|
||||
use oak_render::worker::JobDispatch;
|
||||
@@ -245,6 +245,90 @@ impl RenderTask {
|
||||
}
|
||||
}
|
||||
|
||||
/// The effect stack of a clip block as montage effect descriptors
|
||||
/// (source-first order). Mirrors the app's `effectchain` walk —
|
||||
/// oak-task cannot link oak-app, so the export path keeps its own
|
||||
/// copy over the public oaknode graph API. Disabled effects ride
|
||||
/// along with `enabled = false`; the worker bypasses them (the C++
|
||||
/// traverser's bypass pushes the effect input through unchanged).
|
||||
/// Parameters are the non-hidden, non-connection data inputs at
|
||||
/// their standard (non-keyframed) values.
|
||||
fn clip_effects(graph: &oak_node::graph::Graph, host: oak_node::id::NodeId) -> Vec<MontageEffect> {
|
||||
// Walk the chain: from the host's effect input upstream until an
|
||||
// unconnected input or a node without an effect input; a `seen`
|
||||
// guard protects against malformed cycles.
|
||||
let mut chain = Vec::new();
|
||||
let mut cur = host;
|
||||
let mut seen: Vec<oak_node::id::NodeId> = Vec::new();
|
||||
loop {
|
||||
if seen.contains(&cur) {
|
||||
break;
|
||||
}
|
||||
seen.push(cur);
|
||||
let Some(entry) = graph.get(cur) else {
|
||||
break;
|
||||
};
|
||||
let input = &entry.core.effect_input;
|
||||
if input.is_empty() {
|
||||
break;
|
||||
}
|
||||
let Some(up) = graph.connected_output(cur, input, -1) else {
|
||||
break;
|
||||
};
|
||||
chain.push(up);
|
||||
cur = up;
|
||||
}
|
||||
chain.reverse();
|
||||
chain
|
||||
.into_iter()
|
||||
// The walk runs all the way to the media source; drop the
|
||||
// source node (the one WITHOUT an effect input) — the montage
|
||||
// decodes the footage itself.
|
||||
.filter(|&fx| {
|
||||
graph
|
||||
.get(fx)
|
||||
.map(|e| !e.core.effect_input.is_empty())
|
||||
.unwrap_or(false)
|
||||
})
|
||||
.filter_map(|fx| {
|
||||
let entry = graph.get(fx)?;
|
||||
let enabled = matches!(
|
||||
entry.core.standard_value(oak_node::node::ENABLED_INPUT, -1),
|
||||
oak_node::value::NodeValue::Boolean(true)
|
||||
);
|
||||
let effect_input_id = if entry.core.effect_input.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(entry.core.effect_input.clone())
|
||||
};
|
||||
let mut params = Vec::new();
|
||||
for input in &entry.core.inputs {
|
||||
if matches!(
|
||||
input.value_type,
|
||||
oak_node::value::ValueType::Texture
|
||||
| oak_node::value::ValueType::Samples
|
||||
| oak_node::value::ValueType::Matrix
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
if input.flags & oak_node::input::flags::HIDDEN != 0 {
|
||||
continue;
|
||||
}
|
||||
if input.id == oak_node::node::ENABLED_INPUT {
|
||||
continue;
|
||||
}
|
||||
params.push((input.id.clone(), entry.core.standard_value(&input.id, -1)));
|
||||
}
|
||||
Some(MontageEffect {
|
||||
type_id: entry.behavior.type_id().to_string(),
|
||||
enabled,
|
||||
effect_input_id,
|
||||
params,
|
||||
})
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Flatten the video tracks of `sequence` into an ordered montage
|
||||
/// (bottom-most track first so the topmost track composites last;
|
||||
/// `// CPP-PARITY: M12 P0 montage contract`).
|
||||
@@ -316,6 +400,7 @@ impl RenderTask {
|
||||
out_time: core.out(),
|
||||
media_in: core.media_in,
|
||||
gain: 1.0,
|
||||
effects: Self::clip_effects(&guard.graph, *block_id),
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -393,6 +478,9 @@ impl RenderTask {
|
||||
out_time: core.out(),
|
||||
media_in: core.media_in,
|
||||
gain: 1.0,
|
||||
// Audio clips carry no effect stack (video-side
|
||||
// concept; the mixer never consults it).
|
||||
effects: Vec::new(),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -975,6 +975,7 @@ impl WorkerSession {
|
||||
out_time: Rational::new(c.out_num, c.out_den),
|
||||
media_in: Rational::new(c.media_in_num, c.media_in_den),
|
||||
gain: c.gain,
|
||||
effects: c.effects.iter().map(crate::ipc::montage_effect_from).collect(),
|
||||
})
|
||||
.collect();
|
||||
Ok(AudioTicketParams {
|
||||
@@ -1069,6 +1070,7 @@ impl WorkerSession {
|
||||
out_time: Rational::new(c.out_num, c.out_den),
|
||||
media_in: Rational::new(c.media_in_num, c.media_in_den),
|
||||
gain: c.gain,
|
||||
effects: c.effects.iter().map(crate::ipc::montage_effect_from).collect(),
|
||||
})
|
||||
.collect();
|
||||
VideoTicketParams {
|
||||
|
||||
@@ -321,6 +321,105 @@ fn worker_decodes_real_footage_into_slot() {
|
||||
dispatcher.shutdown();
|
||||
}
|
||||
|
||||
/// M14 R3 end-to-end: a montage clip carrying an effect stack renders
|
||||
/// through a REAL worker process — the effects ride the wire (protocol
|
||||
/// v2 additive `effects` field on the montage clip), the worker applies
|
||||
/// the stack between decode and compositing, and the main process reads
|
||||
/// the changed pixels back from the shm slot. 50% Opacity quarters the
|
||||
/// output (the shader halves every channel; the composite over
|
||||
/// transparent black halves again via the halved alpha).
|
||||
#[test]
|
||||
fn montage_effects_render_through_the_worker() {
|
||||
let _guard = lock_test();
|
||||
let demo = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../oak-app/tests/demo.mp4");
|
||||
assert!(demo.exists(), "repo fixture tests/demo.mp4 missing");
|
||||
|
||||
let dispatcher = ProcessDispatcher::new(config(1, 4)).expect("dispatcher config");
|
||||
dispatcher.start().expect("worker starts");
|
||||
|
||||
let clip = |effects| oak_render::ticket::MontageClip {
|
||||
filename: demo.display().to_string(),
|
||||
stream_index: 0,
|
||||
in_time: Rational::new(0, 1),
|
||||
out_time: Rational::new(10, 1),
|
||||
media_in: Rational::new(0, 1),
|
||||
gain: 1.0,
|
||||
effects,
|
||||
};
|
||||
let montages = [
|
||||
vec![clip(Vec::new())],
|
||||
vec![clip(vec![oak_render::ticket::MontageEffect {
|
||||
type_id: "org.olivevideoeditor.Olive.opacity".into(),
|
||||
enabled: true,
|
||||
effect_input_id: Some("tex_in".into()),
|
||||
params: vec![(
|
||||
"opacity_in".into(),
|
||||
oak_node::value::NodeValue::Float(0.5),
|
||||
)],
|
||||
}])],
|
||||
];
|
||||
|
||||
let results = Arc::new(Mutex::new(Vec::new()));
|
||||
for (i, montage) in montages.into_iter().enumerate() {
|
||||
let results = results.clone();
|
||||
let time = Rational::new(i as i64, 25);
|
||||
let mut p = params(time, None).as_ref().clone();
|
||||
p.montage = montage;
|
||||
let job = Job {
|
||||
node_identity: 1,
|
||||
time,
|
||||
params: Arc::new(p),
|
||||
audio: None,
|
||||
produce: Arc::new(|_, _| {
|
||||
Err(oak_render::error::Error::Failed(
|
||||
"process backend does not use the in-process producer".into(),
|
||||
))
|
||||
}),
|
||||
done: Box::new(move |result| {
|
||||
results.lock().unwrap_or_else(|e| e.into_inner()).push(result);
|
||||
}),
|
||||
schedule: JobSchedule::seek(),
|
||||
};
|
||||
assert!(dispatcher.post(job), "post accepted while alive");
|
||||
}
|
||||
pump_until(&dispatcher, &results, 2);
|
||||
|
||||
// Results arrive in ticket order (one batch, in-order rendering).
|
||||
let mut frames: Vec<Vec<u8>> = Vec::new();
|
||||
for result in results.lock().unwrap().drain(..) {
|
||||
let payload = result.expect("montage frame rendered");
|
||||
let TicketPayload::ShmFrame(frame) = payload else {
|
||||
panic!("ShmFrame payload");
|
||||
};
|
||||
frames.push(frame.shm.slot_bytes(frame.slot)[..frame.meta.data_size as usize].to_vec());
|
||||
dispatcher.release_frame(&frame);
|
||||
}
|
||||
dispatcher.shutdown();
|
||||
|
||||
let [plain, dimmed] = &frames[..] else {
|
||||
panic!("two frames rendered");
|
||||
};
|
||||
// Pick an opaque, non-black pixel in the plain render as the probe.
|
||||
let probe = plain
|
||||
.chunks_exact(4)
|
||||
.position(|px| px[3] == 255 && px[0] > 40)
|
||||
.expect("the fixture frame has an opaque non-black pixel");
|
||||
let p = &plain[probe * 4..probe * 4 + 4];
|
||||
let d = &dimmed[probe * 4..probe * 4 + 4];
|
||||
assert_eq!(p[3], 255, "the plain montage render is opaque");
|
||||
assert!(
|
||||
(d[3] as i32 - 128).abs() <= 3,
|
||||
"50% opacity halves the alpha ({} vs 128)",
|
||||
d[3]
|
||||
);
|
||||
assert!(
|
||||
(d[0] as i32 - p[0] as i32 / 4).abs() <= 6,
|
||||
"50% opacity quarters the color ({} vs {}/4)",
|
||||
d[0],
|
||||
p[0]
|
||||
);
|
||||
}
|
||||
|
||||
/// Submit an empty-montage audio range pull through the dispatcher
|
||||
/// (M15 S3): the worker mixes silence into a shm slot and the main
|
||||
/// process reads it back as [`TicketPayload::ShmAudio`].
|
||||
|
||||
Reference in New Issue
Block a user