render: apply clip effect stacks in the montage path
Adding an effect to a clip did nothing: the sequence render is flattened into a montage (decode + composite), and MontageClip carried no effect data at all. - MontageClip gains an ordered effect stack (type id / enabled / effect input / parameter values); protocol v2 carries it as an additive wire field (older peers default to an empty stack). - renderops::video_montage fills the stack from the effect chain (the footage source node — the chain end without an effect input — is dropped; the montage decodes the footage itself). Export (oak-task) and the multicam single-track montage fill it too. - The worker applies the stack between decode and composite: built-in Opacity gets a CPU evaluator (C++ opacity.frag parity — whole vec4, alpha included, unity pass-through); everything else dispatches as an OFX plugin job through a new instance-factory slot (oak-plugin lazily creates + caches one instance per identifier per render process) with the montage's parameters injected. Disabled effects bypass (the C++ traverser pushes the effect input through). Unknown types warn once per type id and pass through — no silent no-ops. Not covered (explicitly): Transform/Crop and the other ~30 built-in effects have no CPU evaluator in oak-render (they pass through with a warning), keyframed parameter animation, audio effect chains, and the CLI's simplified montage. Acceptance: a real 50% Opacity on real media quarters the rendered pixels both in-process (renderops test) and through a real worker process over IPC + shared memory (procpool_integration test); disabling restores the plain render byte-for-byte.
This commit is contained in:
@@ -149,6 +149,36 @@ pub fn plugin_executor() -> Option<Arc<PluginExecutor>> {
|
||||
.clone()
|
||||
}
|
||||
|
||||
/// Plugin instance factory: resolves an OFX plugin identifier to a live
|
||||
/// instance-registry id, creating (and caching) the instance lazily.
|
||||
/// Implemented by the oakplugin crate — the montage effect path carries
|
||||
/// plugin identifiers, not instance ids (the montage is resolved from
|
||||
/// the timeline in the main process; the instance lives in whichever
|
||||
/// process renders the frame).
|
||||
pub type PluginInstanceFactory = dyn Fn(&str) -> Option<u64> + Send + Sync;
|
||||
|
||||
static PLUGIN_INSTANCE_FACTORY: std::sync::OnceLock<
|
||||
std::sync::Mutex<Option<Arc<PluginInstanceFactory>>>,
|
||||
> = std::sync::OnceLock::new();
|
||||
|
||||
fn instance_factory_slot() -> &'static std::sync::Mutex<Option<Arc<PluginInstanceFactory>>> {
|
||||
PLUGIN_INSTANCE_FACTORY.get_or_init(|| std::sync::Mutex::new(None))
|
||||
}
|
||||
|
||||
/// Install the plugin instance factory (oakplugin registration point;
|
||||
/// `None` clears it).
|
||||
pub fn set_plugin_instance_factory(factory: Option<Arc<PluginInstanceFactory>>) {
|
||||
*instance_factory_slot().lock().unwrap_or_else(|e| e.into_inner()) = factory;
|
||||
}
|
||||
|
||||
/// The installed plugin instance factory, if any.
|
||||
pub fn plugin_instance_factory() -> Option<Arc<PluginInstanceFactory>> {
|
||||
instance_factory_slot()
|
||||
.lock()
|
||||
.unwrap_or_else(|e| e.into_inner())
|
||||
.clone()
|
||||
}
|
||||
|
||||
/// The failure marker frame: solid magenta (1, 0, 1, 1) F32 RGBA —
|
||||
/// the C++ plugin renderer paints failed plugin output purple so a
|
||||
/// broken plugin is visible instead of silently black.
|
||||
@@ -775,7 +805,11 @@ pub fn render_montage_frame_into(
|
||||
(w, h),
|
||||
PixelFormat::F32,
|
||||
)?;
|
||||
let (src_data, src_stride) = match &decoded {
|
||||
// The clip's effect stack runs between decode and compositing
|
||||
// (C++ semantics: the clip texture passes through the chain
|
||||
// bottom-up, the chain top feeds the track composite).
|
||||
let effected = apply_clip_effects(decoded, clip, time);
|
||||
let (src_data, src_stride) = match &effected {
|
||||
Texture::Cpu(src) => (&src.data, src.linesize_bytes() as i32),
|
||||
_ => continue,
|
||||
};
|
||||
@@ -825,6 +859,139 @@ pub fn composite_over(
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Montage clip effect stacks
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// The built-in Opacity effect's type id (oaknode `OpacityEffect`) — the
|
||||
/// one built-in video effect with a CPU evaluator on the montage path.
|
||||
const OPACITY_EFFECT_TYPE_ID: &str = "org.olivevideoeditor.Olive.opacity";
|
||||
|
||||
/// The Opacity effect's value input id (oaknode `opacity_in`).
|
||||
const OPACITY_VALUE_INPUT: &str = "opacity_in";
|
||||
|
||||
/// The effect type ids the montage path already warned about (one log
|
||||
/// line per type per process instead of one per frame).
|
||||
fn unsupported_warned() -> std::sync::MutexGuard<'static, std::collections::HashSet<String>> {
|
||||
static WARNED: std::sync::OnceLock<std::sync::Mutex<std::collections::HashSet<String>>> =
|
||||
std::sync::OnceLock::new();
|
||||
WARNED
|
||||
.get_or_init(|| std::sync::Mutex::new(std::collections::HashSet::new()))
|
||||
.lock()
|
||||
.unwrap_or_else(|e| e.into_inner())
|
||||
}
|
||||
|
||||
/// Log an unsupported-effect passthrough once per type id.
|
||||
fn warn_unsupported_once(type_id: &str, reason: &str) {
|
||||
if unsupported_warned().insert(type_id.to_string()) {
|
||||
eprintln!("montage effect \"{type_id}\" passes through unchanged: {reason}");
|
||||
}
|
||||
}
|
||||
|
||||
/// Run a clip's effect stack over its decoded frame (source-first order;
|
||||
/// disabled effects are bypassed — the C++ traverser's bypass pushes the
|
||||
/// effect input through unchanged). Effects the montage path cannot
|
||||
/// evaluate log a warning once and pass the frame through.
|
||||
fn apply_clip_effects(
|
||||
src: Texture,
|
||||
clip: &crate::ticket::MontageClip,
|
||||
time: Rational,
|
||||
) -> Texture {
|
||||
let mut tex = src;
|
||||
for effect in &clip.effects {
|
||||
if !effect.enabled {
|
||||
continue;
|
||||
}
|
||||
tex = apply_montage_effect(tex, effect, time);
|
||||
}
|
||||
tex
|
||||
}
|
||||
|
||||
/// Apply one effect to `src` (an F32 RGBA CPU frame of the montage
|
||||
/// pipeline). `time` is the sequence time the frame is rendered at (the
|
||||
/// C++ node evaluation time).
|
||||
fn apply_montage_effect(
|
||||
src: Texture,
|
||||
effect: &crate::ticket::MontageEffect,
|
||||
time: Rational,
|
||||
) -> Texture {
|
||||
// Built-in Opacity: multiply every channel by the opacity factor
|
||||
// (C++ `:/shaders/opacity.frag`: `frag_color = texture(tex_in, …) *
|
||||
// opacity_in` — the shader scales the whole vec4, alpha included).
|
||||
if effect.type_id == OPACITY_EFFECT_TYPE_ID {
|
||||
let factor = effect
|
||||
.params
|
||||
.iter()
|
||||
.find(|(id, _)| id == OPACITY_VALUE_INPUT)
|
||||
.map(|(_, v)| v.to_double())
|
||||
.unwrap_or(1.0);
|
||||
// Unity is a pass-through (C++ `qFuzzyCompare(opacity, 1.0)`).
|
||||
if (factor - 1.0).abs() * 1e12 <= factor.abs().min(1.0) {
|
||||
return src;
|
||||
}
|
||||
// Texture implements Drop, so scale in place through a mutable
|
||||
// borrow instead of moving the frame out.
|
||||
let mut tex = src;
|
||||
match &mut tex {
|
||||
Texture::Cpu(frame) => {
|
||||
let stride = frame.linesize_bytes();
|
||||
for row in frame.data.chunks_exact_mut(stride).take(frame.height.max(0) as usize) {
|
||||
for px in row[..(frame.width.max(0) as usize) * 16].chunks_exact_mut(16) {
|
||||
for c in px.chunks_exact_mut(4) {
|
||||
let v = f32::from_le_bytes(c.try_into().unwrap());
|
||||
c.copy_from_slice(&(v * factor as f32).to_le_bytes());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
_ => {
|
||||
warn_unsupported_once(
|
||||
&effect.type_id,
|
||||
"opacity on a non-CPU texture is not supported by the montage path",
|
||||
);
|
||||
}
|
||||
}
|
||||
return tex;
|
||||
}
|
||||
|
||||
// Everything else: an OFX plugin effect. The montage carries the
|
||||
// plugin identifier; the rendering process resolves it to a live
|
||||
// instance through the oakplugin-installed factory (lazily created
|
||||
// and cached per identifier), then dispatches through the plugin
|
||||
// executor exactly like the graph path's plugin jobs.
|
||||
let Some(factory) = plugin_instance_factory() else {
|
||||
warn_unsupported_once(
|
||||
&effect.type_id,
|
||||
"no plugin instance factory installed (oakplugin init missing in this process)",
|
||||
);
|
||||
return src;
|
||||
};
|
||||
let Some(instance) = factory(&effect.type_id) else {
|
||||
warn_unsupported_once(
|
||||
&effect.type_id,
|
||||
"no evaluator: unknown built-in effect or OFX plugin unavailable in this process",
|
||||
);
|
||||
return src;
|
||||
};
|
||||
let spec = JobSpec::Plugin {
|
||||
instance,
|
||||
time: time.to_f64(),
|
||||
effect_input_id: effect.effect_input_id.clone(),
|
||||
inputs: Vec::new(),
|
||||
values: effect.params.clone(),
|
||||
};
|
||||
let size = src.size();
|
||||
match RenderEvalHooks::new().process_plugin_job(src, &spec) {
|
||||
Ok(texture) => texture,
|
||||
Err(err) => {
|
||||
// Unreachable for a Plugin spec (the executor failure path
|
||||
// yields a purple frame); stay loud rather than silent.
|
||||
eprintln!("montage plugin job failed to dispatch: {err:#}");
|
||||
purple_frame(time, size)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
@@ -1165,4 +1332,118 @@ mod tests {
|
||||
let mut dst = [0u8; 8]; // far too small for 2000x2 f32 samples
|
||||
assert!(render_audio_samples_into(¶ms, &mut dst).is_err());
|
||||
}
|
||||
|
||||
// ---- Montage clip effect stacks -------------------------------------
|
||||
|
||||
/// A 2x1 F32 texture filled with a known color.
|
||||
fn solid_texture(r: f32, g: f32, b: f32, a: f32) -> Texture {
|
||||
let mut frame = generate_frame(Rational::new(0, 1), (2, 1), PixelFormat::F32).unwrap();
|
||||
for px in frame.data.chunks_exact_mut(16) {
|
||||
for (c, v) in px.chunks_exact_mut(4).zip([r, g, b, a]) {
|
||||
c.copy_from_slice(&v.to_le_bytes());
|
||||
}
|
||||
}
|
||||
Texture::Cpu(frame)
|
||||
}
|
||||
|
||||
fn opacity_effect(enabled: bool, value: f64) -> crate::ticket::MontageEffect {
|
||||
crate::ticket::MontageEffect {
|
||||
type_id: OPACITY_EFFECT_TYPE_ID.to_string(),
|
||||
enabled,
|
||||
effect_input_id: Some("tex_in".to_string()),
|
||||
params: vec![(OPACITY_VALUE_INPUT.to_string(), NodeValue::Float(value))],
|
||||
}
|
||||
}
|
||||
|
||||
fn clip_with_effects(effects: Vec<crate::ticket::MontageEffect>) -> crate::ticket::MontageClip {
|
||||
crate::ticket::MontageClip {
|
||||
filename: String::new(),
|
||||
stream_index: 0,
|
||||
in_time: Rational::new(0, 1),
|
||||
out_time: Rational::new(1, 1),
|
||||
media_in: Rational::new(0, 1),
|
||||
gain: 1.0,
|
||||
effects,
|
||||
}
|
||||
}
|
||||
|
||||
/// The built-in Opacity effect really scales the decoded pixels
|
||||
/// (every channel, C++ `opacity.frag` parity).
|
||||
#[test]
|
||||
fn montage_opacity_effect_scales_pixels() {
|
||||
let clip = clip_with_effects(vec![opacity_effect(true, 0.5)]);
|
||||
let out = apply_clip_effects(solid_texture(0.8, 0.4, 0.2, 1.0), &clip, Rational::new(0, 1));
|
||||
assert_eq!(first_pixel(&out), [0.4, 0.2, 0.1, 0.5]);
|
||||
}
|
||||
|
||||
/// Disabled effects are bypassed (C++ traverser parity), and unity
|
||||
/// opacity is a pass-through.
|
||||
#[test]
|
||||
fn montage_disabled_or_unity_effects_pass_through() {
|
||||
let disabled = clip_with_effects(vec![opacity_effect(false, 0.5)]);
|
||||
let out = apply_clip_effects(solid_texture(0.8, 0.4, 0.2, 1.0), &disabled, Rational::new(0, 1));
|
||||
assert_eq!(first_pixel(&out), [0.8, 0.4, 0.2, 1.0]);
|
||||
|
||||
let unity = clip_with_effects(vec![opacity_effect(true, 1.0)]);
|
||||
let out = apply_clip_effects(solid_texture(0.8, 0.4, 0.2, 1.0), &unity, Rational::new(0, 1));
|
||||
assert_eq!(first_pixel(&out), [0.8, 0.4, 0.2, 1.0]);
|
||||
}
|
||||
|
||||
/// An OFX effect (any non-built-in type id) resolves its instance
|
||||
/// through the installed factory and dispatches through the executor,
|
||||
/// carrying the montage's parameter values.
|
||||
#[test]
|
||||
fn montage_plugin_effect_dispatches_with_params() {
|
||||
let _guard = PLUGIN_TEST_LOCK.lock().unwrap();
|
||||
set_plugin_instance_factory(Some(Arc::new(|identifier: &str| {
|
||||
(identifier == "com.example.darken").then_some(7)
|
||||
})));
|
||||
set_plugin_executor(Some(Arc::new(|req: &PluginJobRequest<'_>| {
|
||||
let JobSpec::Plugin { instance, values, .. } = req.spec else {
|
||||
return Err(Error::Invalid);
|
||||
};
|
||||
assert_eq!(*instance, 7);
|
||||
// The injected parameter drives the output: paint the frame
|
||||
// with the "gain" value so the test observes the param path.
|
||||
let gain = values
|
||||
.iter()
|
||||
.find(|(k, _)| k == "gain")
|
||||
.map(|(_, v)| v.to_double() as f32)
|
||||
.unwrap_or(1.0);
|
||||
let mut frame = generate_frame(Rational::new(0, 1), req.src.size(), PixelFormat::F32)?;
|
||||
for px in frame.data.chunks_exact_mut(16) {
|
||||
for (c, v) in px.chunks_exact_mut(4).zip([gain, gain, gain, 1.0]) {
|
||||
c.copy_from_slice(&v.to_le_bytes());
|
||||
}
|
||||
}
|
||||
Ok(Texture::Cpu(frame))
|
||||
})));
|
||||
let clip = clip_with_effects(vec![crate::ticket::MontageEffect {
|
||||
type_id: "com.example.darken".to_string(),
|
||||
enabled: true,
|
||||
effect_input_id: Some("Source".to_string()),
|
||||
params: vec![("gain".to_string(), NodeValue::Float(0.25))],
|
||||
}]);
|
||||
let out = apply_clip_effects(solid_texture(0.8, 0.4, 0.2, 1.0), &clip, Rational::new(0, 1));
|
||||
assert_eq!(first_pixel(&out), [0.25, 0.25, 0.25, 1.0]);
|
||||
set_plugin_executor(None);
|
||||
set_plugin_instance_factory(None);
|
||||
}
|
||||
|
||||
/// An effect nobody can evaluate (no factory / unknown type) passes
|
||||
/// the frame through unchanged — loudly (the warn-once log), never a
|
||||
/// silent no-op.
|
||||
#[test]
|
||||
fn montage_unknown_effect_passes_through() {
|
||||
let _guard = PLUGIN_TEST_LOCK.lock().unwrap();
|
||||
set_plugin_instance_factory(None);
|
||||
let clip = clip_with_effects(vec![crate::ticket::MontageEffect {
|
||||
type_id: "com.example.missing".to_string(),
|
||||
enabled: true,
|
||||
effect_input_id: Some("Source".to_string()),
|
||||
params: Vec::new(),
|
||||
}]);
|
||||
let out = apply_clip_effects(solid_texture(0.8, 0.4, 0.2, 1.0), &clip, Rational::new(0, 1));
|
||||
assert_eq!(first_pixel(&out), [0.8, 0.4, 0.2, 1.0]);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -272,6 +272,122 @@ pub struct HelloCapsMsg {
|
||||
pub max_slot_bytes: i64,
|
||||
}
|
||||
|
||||
/// A node value on the wire (protocol v2, montage effect parameters).
|
||||
/// `oak_node::value::NodeValue` itself is not serde-able (texture/sample
|
||||
/// payloads, handles); this enum covers the plain-data variants an effect
|
||||
/// parameter can carry. Connection/handle variants (texture, samples,
|
||||
/// node refs, video/audio params, push buttons) have no wire
|
||||
/// representation and are dropped by [`WireNodeValue::from_node_value`].
|
||||
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq)]
|
||||
#[serde(tag = "t", content = "v", rename_all = "snake_case")]
|
||||
pub enum WireNodeValue {
|
||||
/// No value.
|
||||
None,
|
||||
/// Integer.
|
||||
Int(i64),
|
||||
/// Float.
|
||||
Float(f64),
|
||||
/// RGBA color.
|
||||
Color([f64; 4]),
|
||||
/// Text.
|
||||
Text(String),
|
||||
/// Boolean.
|
||||
Boolean(bool),
|
||||
/// Rational (num, den).
|
||||
Rational(i64, i64),
|
||||
/// Vec2.
|
||||
Vec2([f64; 2]),
|
||||
/// Vec3.
|
||||
Vec3([f64; 3]),
|
||||
/// Vec4.
|
||||
Vec4([f64; 4]),
|
||||
/// 4x4 matrix, row-major.
|
||||
Matrix([f64; 16]),
|
||||
/// Combo index.
|
||||
Combo(i64),
|
||||
/// String combo.
|
||||
StrCombo(String),
|
||||
/// Opaque bytes.
|
||||
Binary(Vec<u8>),
|
||||
}
|
||||
|
||||
impl Default for WireNodeValue {
|
||||
fn default() -> Self {
|
||||
WireNodeValue::None
|
||||
}
|
||||
}
|
||||
|
||||
impl WireNodeValue {
|
||||
/// The wire form of a node value, or `None` when the variant has no
|
||||
/// wire representation (the parameter is then not carried).
|
||||
pub fn from_node_value(v: &oak_node::value::NodeValue) -> Option<WireNodeValue> {
|
||||
use oak_node::value::NodeValue as NV;
|
||||
Some(match v {
|
||||
NV::None => WireNodeValue::None,
|
||||
NV::Int(i) => WireNodeValue::Int(*i),
|
||||
NV::Float(f) => WireNodeValue::Float(*f),
|
||||
NV::Color(c) => WireNodeValue::Color(*c),
|
||||
NV::Text(s) => WireNodeValue::Text(s.clone()),
|
||||
NV::Boolean(b) => WireNodeValue::Boolean(*b),
|
||||
NV::Rational(r) => WireNodeValue::Rational(r.numerator(), r.denominator()),
|
||||
NV::Vec2(v2) => WireNodeValue::Vec2(*v2),
|
||||
NV::Vec3(v3) => WireNodeValue::Vec3(*v3),
|
||||
NV::Vec4(v4) => WireNodeValue::Vec4(*v4),
|
||||
NV::Matrix(m) => WireNodeValue::Matrix(*m),
|
||||
NV::Combo(i) => WireNodeValue::Combo(*i),
|
||||
NV::StrCombo(s) => WireNodeValue::StrCombo(s.clone()),
|
||||
NV::Binary(b) => WireNodeValue::Binary(b.clone()),
|
||||
_ => return None,
|
||||
})
|
||||
}
|
||||
|
||||
/// Back to a node value (worker side).
|
||||
pub fn to_node_value(&self) -> oak_node::value::NodeValue {
|
||||
use oak_node::value::NodeValue as NV;
|
||||
match self {
|
||||
WireNodeValue::None => NV::None,
|
||||
WireNodeValue::Int(i) => NV::Int(*i),
|
||||
WireNodeValue::Float(f) => NV::Float(*f),
|
||||
WireNodeValue::Color(c) => NV::Color(*c),
|
||||
WireNodeValue::Text(s) => NV::Text(s.clone()),
|
||||
WireNodeValue::Boolean(b) => NV::Boolean(*b),
|
||||
WireNodeValue::Rational(n, d) => NV::Rational(oak_core::Rational::new(*n, *d)),
|
||||
WireNodeValue::Vec2(v2) => NV::Vec2(*v2),
|
||||
WireNodeValue::Vec3(v3) => NV::Vec3(*v3),
|
||||
WireNodeValue::Vec4(v4) => NV::Vec4(*v4),
|
||||
WireNodeValue::Matrix(m) => NV::Matrix(*m),
|
||||
WireNodeValue::Combo(i) => NV::Combo(*i),
|
||||
WireNodeValue::StrCombo(s) => NV::StrCombo(s.clone()),
|
||||
WireNodeValue::Binary(b) => NV::Binary(b.clone()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One effect parameter on the wire (input id + value).
|
||||
#[derive(Serialize, Deserialize, Default, Debug, Clone, PartialEq)]
|
||||
#[serde(default)]
|
||||
pub struct WireEffectParam {
|
||||
/// Node input id.
|
||||
pub input: String,
|
||||
/// Parameter value.
|
||||
pub value: WireNodeValue,
|
||||
}
|
||||
|
||||
/// One effect of a montage clip's effect stack on the wire (protocol v2
|
||||
/// additive field of [`WireMontageClip`]; source-first order).
|
||||
#[derive(Serialize, Deserialize, Default, Debug, Clone, PartialEq)]
|
||||
#[serde(default)]
|
||||
pub struct WireMontageEffect {
|
||||
/// Built-in node type id or OFX plugin identifier.
|
||||
pub type_id: String,
|
||||
/// Enabled flag (disabled effects are bypassed).
|
||||
pub enabled: bool,
|
||||
/// Effect input (clip) name; "" = none.
|
||||
pub effect_input_id: String,
|
||||
/// Parameter values.
|
||||
pub params: Vec<WireEffectParam>,
|
||||
}
|
||||
|
||||
/// One montage clip on the wire (rationals flattened to num/den pairs).
|
||||
#[derive(Serialize, Deserialize, Default, Debug, Clone, PartialEq)]
|
||||
#[serde(default)]
|
||||
@@ -294,6 +410,47 @@ pub struct WireMontageClip {
|
||||
pub media_in_den: i64,
|
||||
/// Playback gain (1.0 = unity).
|
||||
pub gain: f32,
|
||||
/// The clip's effect stack (protocol v2 additive: older peers omit the
|
||||
/// field and it defaults to an empty stack).
|
||||
pub effects: Vec<WireMontageEffect>,
|
||||
}
|
||||
|
||||
/// Map a ticket-side montage effect to its wire form (main process;
|
||||
/// parameters without a wire representation are dropped).
|
||||
pub fn wire_effect_from(effect: &crate::ticket::MontageEffect) -> WireMontageEffect {
|
||||
WireMontageEffect {
|
||||
type_id: effect.type_id.clone(),
|
||||
enabled: effect.enabled,
|
||||
effect_input_id: effect.effect_input_id.clone().unwrap_or_default(),
|
||||
params: effect
|
||||
.params
|
||||
.iter()
|
||||
.filter_map(|(input, value)| {
|
||||
WireNodeValue::from_node_value(value).map(|value| WireEffectParam {
|
||||
input: input.clone(),
|
||||
value,
|
||||
})
|
||||
})
|
||||
.collect(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Map a wire effect back to the ticket-side form (worker).
|
||||
pub fn montage_effect_from(wire: &WireMontageEffect) -> crate::ticket::MontageEffect {
|
||||
crate::ticket::MontageEffect {
|
||||
type_id: wire.type_id.clone(),
|
||||
enabled: wire.enabled,
|
||||
effect_input_id: if wire.effect_input_id.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(wire.effect_input_id.clone())
|
||||
},
|
||||
params: wire
|
||||
.params
|
||||
.iter()
|
||||
.map(|p| (p.input.clone(), p.value.to_node_value()))
|
||||
.collect(),
|
||||
}
|
||||
}
|
||||
|
||||
/// One frame ticket inside a [`RenderBatchMsg`] — the main process
|
||||
@@ -1710,6 +1867,15 @@ mod tests {
|
||||
media_in_num: 10,
|
||||
media_in_den: 24,
|
||||
gain: 0.5,
|
||||
effects: vec![WireMontageEffect {
|
||||
type_id: "org.olivevideoeditor.Olive.opacity".into(),
|
||||
enabled: true,
|
||||
effect_input_id: "tex_in".into(),
|
||||
params: vec![WireEffectParam {
|
||||
input: "opacity_in".into(),
|
||||
value: WireNodeValue::Float(0.5),
|
||||
}],
|
||||
}],
|
||||
}],
|
||||
},
|
||||
],
|
||||
@@ -1721,6 +1887,15 @@ mod tests {
|
||||
assert_eq!(value["tickets"][0]["format"], SLOT_FORMAT_BGRA8);
|
||||
assert_eq!(value["tickets"][1]["montage"][0]["filename"], "b.mp4");
|
||||
assert_eq!(value["tickets"][1]["montage"][0]["media_in_num"], 10);
|
||||
// The effect stack rides the clip as an additive v2 field.
|
||||
assert_eq!(
|
||||
value["tickets"][1]["montage"][0]["effects"][0]["type_id"],
|
||||
"org.olivevideoeditor.Olive.opacity"
|
||||
);
|
||||
assert_eq!(
|
||||
value["tickets"][1]["montage"][0]["effects"][0]["params"][0]["value"],
|
||||
json!({ "t": "float", "v": 0.5 })
|
||||
);
|
||||
// Round-trip back to the struct.
|
||||
let round: RenderBatchMsg = serde_json::from_value(value).unwrap();
|
||||
assert_eq!(round, batch);
|
||||
@@ -1733,6 +1908,49 @@ mod tests {
|
||||
assert!(bare.montage.is_empty());
|
||||
}
|
||||
|
||||
/// Protocol v2 compatibility: a v1-shaped montage clip (no `effects`
|
||||
/// field) parses with an empty effect stack, and the node-value wire
|
||||
/// enum round-trips every plain-data variant.
|
||||
#[test]
|
||||
fn montage_clip_effects_are_additive_and_values_roundtrip() {
|
||||
let legacy = json!({
|
||||
"filename": "a.mp4",
|
||||
"stream_index": 0,
|
||||
"in_num": 0, "in_den": 24,
|
||||
"out_num": 48, "out_den": 24,
|
||||
"media_in_num": 0, "media_in_den": 24,
|
||||
"gain": 1.0,
|
||||
});
|
||||
let parsed: WireMontageClip = serde_json::from_value(legacy).unwrap();
|
||||
assert!(parsed.effects.is_empty(), "v1 clips carry no effects");
|
||||
|
||||
use oak_node::value::NodeValue as NV;
|
||||
let cases = [
|
||||
NV::None,
|
||||
NV::Int(-3),
|
||||
NV::Float(0.25),
|
||||
NV::Color([0.1, 0.2, 0.3, 1.0]),
|
||||
NV::Text("hello".into()),
|
||||
NV::Boolean(true),
|
||||
NV::Rational(oak_core::Rational::new(1, 24)),
|
||||
NV::Vec2([1.0, 2.0]),
|
||||
NV::Vec3([1.0, 2.0, 3.0]),
|
||||
NV::Vec4([1.0, 2.0, 3.0, 4.0]),
|
||||
NV::Matrix([0.0; 16]),
|
||||
NV::Combo(2),
|
||||
NV::StrCombo("choice".into()),
|
||||
NV::Binary(vec![1, 2, 3]),
|
||||
];
|
||||
for value in cases {
|
||||
let wire = WireNodeValue::from_node_value(&value).expect("plain data has a wire form");
|
||||
let json = serde_json::to_string(&wire).unwrap();
|
||||
let back: WireNodeValue = serde_json::from_str(&json).unwrap();
|
||||
assert_eq!(back.to_node_value(), value, "wire round-trip of {value:?}");
|
||||
}
|
||||
// Connection/handle variants have no wire form.
|
||||
assert!(WireNodeValue::from_node_value(&NV::PushButton).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn batch_accepted_and_frame_failed_wire_roundtrip() {
|
||||
let accepted = BatchAcceptedMsg {
|
||||
@@ -1790,6 +2008,7 @@ mod tests {
|
||||
media_in_num: 10,
|
||||
media_in_den: 24,
|
||||
gain: 0.5,
|
||||
effects: vec![],
|
||||
}],
|
||||
}],
|
||||
};
|
||||
|
||||
@@ -1936,6 +1936,7 @@ fn build_ticket_spec(
|
||||
media_in_num: c.media_in.numerator(),
|
||||
media_in_den: c.media_in.denominator(),
|
||||
gain: c.gain,
|
||||
effects: c.effects.iter().map(crate::ipc::wire_effect_from).collect(),
|
||||
})
|
||||
.collect();
|
||||
BatchTicketSpec {
|
||||
@@ -1970,6 +1971,7 @@ fn build_audio_ticket_spec(ticket: i64, slot: u32, params: &AudioTicketParams) -
|
||||
media_in_num: c.media_in.numerator(),
|
||||
media_in_den: c.media_in.denominator(),
|
||||
gain: c.gain,
|
||||
effects: c.effects.iter().map(crate::ipc::wire_effect_from).collect(),
|
||||
})
|
||||
.collect();
|
||||
AudioTicketSpec {
|
||||
|
||||
@@ -35,6 +35,31 @@ use crate::eval;
|
||||
use crate::texture::Texture;
|
||||
use crate::worker::{JobDispatch, JobSchedule};
|
||||
|
||||
/// One effect of a montage clip's effect stack (M14 R3 effect-chain
|
||||
/// wiring into the montage render path). The stack is ordered
|
||||
/// source-first (the chain walk's signal order: the first element feeds
|
||||
/// the media side, the last feeds the clip's effect input); the renderer
|
||||
/// applies them in sequence after decode, before compositing.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct MontageEffect {
|
||||
/// Node type id: the built-in factory type id (e.g.
|
||||
/// `org.olivevideoeditor.Olive.opacity`) or the OFX plugin identifier.
|
||||
pub type_id: String,
|
||||
/// The effect's enabled flag. Disabled effects are bypassed (the C++
|
||||
/// traverser's bypass pushes the effect input through unchanged), so
|
||||
/// the renderer skips them; they are carried anyway so the render side
|
||||
/// can log what it skipped.
|
||||
pub enabled: bool,
|
||||
/// The clip/input name the source texture arrives on (the node's
|
||||
/// effect input id — "tex_in" for built-ins, "Source" for typical OFX
|
||||
/// filters; `None` when the node has no effect input).
|
||||
pub effect_input_id: Option<String>,
|
||||
/// Parameter values (input id -> value): the node's non-hidden,
|
||||
/// non-connection data inputs at their standard (non-keyframed)
|
||||
/// values — the same parameter set the inspector exposes.
|
||||
pub params: Vec<(String, oak_node::value::NodeValue)>,
|
||||
}
|
||||
|
||||
/// One clip of a sequence montage (M12 P0): the facade resolves the
|
||||
/// timeline into an ordered list of clips; the producer decodes each and
|
||||
/// composites them topmost-last.
|
||||
@@ -52,6 +77,9 @@ pub struct MontageClip {
|
||||
pub media_in: Rational,
|
||||
/// Playback gain (1.0 = unity).
|
||||
pub gain: f32,
|
||||
/// The clip's effect stack (source-first; empty for audio clips and
|
||||
/// for montage builders that do not resolve effect chains).
|
||||
pub effects: Vec<MontageEffect>,
|
||||
}
|
||||
|
||||
/// Audio ticket parameters (M12 P1): the output format plus the audio
|
||||
|
||||
Reference in New Issue
Block a user