From e37a87be6728199c767edc6fd5a525325c9f29db Mon Sep 17 00:00:00 2001 From: Mike Solar Date: Mon, 13 Jul 2026 22:01:13 +0800 Subject: [PATCH] docs: Update test plan. --- ...color-audio-performance-community-tasks.md | 334 ++++++++++++++++++ 1 file changed, 334 insertions(+) create mode 100644 docs/v04-color-audio-performance-community-tasks.md diff --git a/docs/v04-color-audio-performance-community-tasks.md b/docs/v04-color-audio-performance-community-tasks.md new file mode 100644 index 000000000..d7b1d87e6 --- /dev/null +++ b/docs/v04-color-audio-performance-community-tasks.md @@ -0,0 +1,334 @@ +# v0.4 Color, Audio & Performance — Community Testing Tasks + +This document breaks the v0.4 manual test plan into small, volunteer-friendly tasks. Pick one, run it, and check the box when it passes. If it fails, file an issue using the bug template at the bottom. + +## How to contribute + +1. Pick an unchecked task below. +2. Build or download the latest v0.4.0-alpha build for your platform. +3. Follow the steps and compare against the pass criteria. +4. Check the box in your local copy and report the result in a comment/issue. + +--- + +## 0. Pre-flight checks + +- [ ] **0.1 Import sample media** + - Create a new project. + - Import a 4K H.264/HEVC clip with audio, plus a WAV file. + - Save the project, close Oak, reopen it. + - **Pass:** all clips are still online and no crash on reopen. + +- [ ] **0.2 Smoke test on clean build** + - Launch Oak from a fresh build or CI package. + - Open the About dialog and confirm the version reads `v0.4.0-alpha`. + - **Pass:** app launches without crash and version is correct. + +--- + +## 1. LUT & Color + +- [ ] **1.1 Apply a `.cube` LUT** + - Add a clip to the timeline. + - Add an OCIO LUT effect and load a `.cube` LUT. + - Play 5 seconds and scrub the timeline. + - **Pass:** colors change visibly; no hang; scrubbing updates the viewer. + +- [ ] **1.2 Apply a `.3dl` LUT** + - Repeat 1.1 with a `.3dl` LUT. + - Toggle the effect on/off. + - **Pass:** `.3dl` works; toggle is instant; no crash. + +- [ ] **1.3 Corrupt LUT handling** + - Try to load a malformed LUT file. + - Observe the error UI, then continue playing the original clip. + - Save, close, and reopen the project. + - **Pass:** no crash; original clip still plays; project reopens cleanly. + +- [ ] **1.4 Three-way color wheels** + - Open the three-way color wheels panel. + - Adjust Shadows, Midtones, and Highlights on a color-chart clip. + - Play, pause, and step frames. + - **Pass:** each region changes the image; undo/redo works; values persist after reopen. + +- [ ] **1.5 LUT + proxy combination** + - Generate a proxy for a heavy clip and enable `Use Proxy`. + - Apply a LUT and three-way adjustments. + - Disable proxy and compare the same frame. + - **Pass:** color looks consistent; export uses full-resolution source, not proxy. + +--- + +## 2. Scopes + +- [ ] **2.1 Waveform scope** + - Open the Scope panel and select Waveform. + - Play `color_chart.mov` and adjust brightness/exposure. + - **Pass:** waveform updates with current frame and grade; no lag buildup. + +- [ ] **2.2 Vectorscope** + - Switch the Scope panel to Vectorscope. + - Use a frame with skin tones and saturated colors. + - Adjust saturation or color wheels. + - **Pass:** plot moves with saturation/hue changes; switching panels does not crash. + +- [ ] **2.3 Histogram** + - Switch the Scope panel to Histogram. + - Adjust brightness/contrast on a high-contrast or grayscale frame. + - Play and pause on different frames. + - **Pass:** histogram updates correctly; no ghosting from previous frames. + +- [ ] **2.4 Scope switching stress** + - Switch between Waveform, Vectorscope, and Histogram 20 times. + - While switching, scrub the timeline and adjust a grade. + - **Pass:** no OpenGL/shader crash; UI stays responsive. + +--- + +## 3. Waveform Auto-Sync + +- [ ] **3.1 Dual-system audio sync** + - Add a video clip with in-camera audio and a separate external WAV. + - Offset the external audio by 1–3 seconds. + - Select both clips and run `Sync by Waveform`. + - Play the slate/lip-sync section. + - **Pass:** slate peaks align; lip-sync error is ≤ 1 frame; no clips deleted. + +- [ ] **3.2 Multi-clip sync** + - Add 3 video clips and 3 external audio tracks with different offsets. + - Select all and run waveform sync. + - **Pass:** each pair aligns; unmatched clips stay in place or report failure. + +- [ ] **3.3 Low-quality reference audio** + - Use a noisy or quiet reference track for waveform sync. + - **Pass:** if sync fails, it does not create a false alignment; user gets a warning or the clip stays put. + +- [ ] **3.4 Sync persistence** + - After a successful sync, save the project, close Oak, and reopen. + - Play the sync point. + - **Pass:** clip positions are preserved; sync does not drift on reopen. + +--- + +## 4. BWF Timecode Sync + +- [ ] **4.1 Read BWF timecode** + - Import a WAV file with BWF timecode metadata. + - Check the clip properties/timecode field. + - Place it on the timeline and run `Sync by Timecode`. + - **Pass:** start timecode is recognized; sync does not jump to an extreme timestamp. + +- [ ] **4.2 BWF + video timecode alignment** + - Import a video clip and a BWF audio file with matching timecode. + - Run `Sync by Source Timecode`. + - **Pass:** audio and video align; timeline position matches the timecode difference. + +- [ ] **4.3 Missing timecode fallback** + - Select a plain WAV without BWF timecode and run `Sync by Timecode`. + - **Pass:** a clear error or skip; no extreme offset applied. + +--- + +## 5. Audio Meters + +- [ ] **5.1 VU meter response** + - Add `noisy_dialogue.wav` to the timeline. + - Open the audio meter panel and play silence, dialogue, and music sections. + - **Pass:** needle moves with volume; drops on silence; settles after stop. + +- [ ] **5.2 LUFS reading** + - Play 30 seconds of dialogue or music. + - Watch the short-term/integrated LUFS readout. + - Change clip gain and replay. + - **Pass:** LUFS follows gain changes; no NaN, inf, or wild jumps. + +- [ ] **5.3 Multi-channel audio** + - Import stereo and multi-channel clips. + - Play and mute/lower one clip. + - **Pass:** channels display correctly; mute/gain changes are instant. + +--- + +## 6. Proxy Media + +- [ ] **6.1 Generate proxy** + - Add an 8K or heavy 4K clip to the timeline. + - Right-click → `Proxy > Generate Proxy`. + - Watch the task list and proxy cache folder. + - **Pass:** `.working` file becomes `.mp4`; `.working` is cleaned; UI does not block. + +- [ ] **6.2 Enable/disable proxy** + - After generation, check `Proxy > Use Proxy` and play. + - Uncheck `Use Proxy` and play again. + - **Pass:** proxy playback works; disabling returns to original; missing proxy falls back safely. + +- [ ] **6.3 Reveal and delete proxy** + - Run `Proxy > Reveal Proxy` and confirm the folder opens. + - Run `Proxy > Delete Proxy` and play again. + - **Pass:** reveal points to cache/proxy; delete removes proxy and working files; playback falls back. + +- [ ] **6.4 Proxy state persistence** + - Generate, enable proxy, and save. + - Close and reopen the project. + - **Pass:** `Use Proxy` state is restored; proxy file is used if it exists. + +- [ ] **6.5 Proxy failure path** + - Temporarily break `ffmpeg` or use an unreadable source. + - Try to generate a proxy. + - **Pass:** clear failure message; `.working` cleaned; original clip still plays. + +- [ ] **6.6 Export ignores proxy** + - Generate a 720p proxy for a 4K clip and enable it. + - Export a 4K segment. + - **Pass:** exported file is 4K and uses full-resolution source. + +--- + +## 7. Hardware-Accelerated Export + +- [ ] **7.1 NVENC export (NVIDIA only)** + - On an NVIDIA machine, choose H.264/H.265 NVENC in export settings. + - Export a 30-second 4K segment. + - Check codec with `ffprobe`. + - **Pass:** export succeeds; codec is correct; unavailable hardware reports a clear error. + +- [ ] **7.2 VideoToolbox export (macOS only)** + - On macOS, choose VideoToolbox H.264/H.265. + - Export a 30-second 4K segment. + - **Pass:** export succeeds; output plays; system load looks like hardware encoding. + +- [ ] **7.3 Hardware export fallback** + - Select a hardware encoder your machine does not support. + - Attempt export. + - **Pass:** understandable failure; option to switch to software encoding. + +--- + +## 8. Batch Render Queue + +- [ ] **8.1 Multi-job queue** + - Create 3 sequences: a short clip, one with a LUT, one with proxy. + - Add all 3 to the batch render queue and start. + - **Pass:** jobs run in order; each output file is created; one failure does not crash the app. + +- [ ] **8.2 Queue cancel** + - Add a long export job, start it, then cancel immediately. + - Add a short job and run it. + - **Pass:** cancel leaves no dead state; next job runs; partial file status is clear. + +- [ ] **8.3 Queue project save** + - Configure several queue jobs and save the project. + - Close and reopen Oak. + - **Pass:** behavior is consistent: queue is either restored cleanly or empty, never half-broken. + +--- + +## 9. Regression & Path Tests + +- [ ] **9.1 Full editing chain** + - Make a 60-second sequence mixing 4K/8K, external audio, LUT, three-way grade, and proxy. + - Waveform-sync some clips. + - Open Scopes and audio meters and play the whole sequence. + - Export a software-encoded version. + - **Pass:** no crash; exported file is in sync; color and audio match preview. + +- [ ] **9.2 Chinese and space characters in paths** + - Put project, media, cache, and export target in a path like `Oak v04 测试/素材 A`. + - Repeat proxy generation, LUT load, and export. + - **Pass:** paths work; `Reveal Proxy` and exported file open correctly. + +- [ ] **9.3 Long playback stability** + - Open a 4K/8K project and loop playback for 20 minutes. + - Toggle proxy, scopes, and audio meters during playback. + - **Pass:** memory does not grow uncontrollably; app remains responsive; save works after stop. + +--- + +## 10. Graphics Backend (OpenGL/Vulkan) + +> Only test Vulkan if `vulkaninfo --summary` reports a working instance and device. Otherwise, mark Vulkan tasks as skipped and test OpenGL fallback only. + +- [ ] **10.1 Default OpenGL backend** + - Delete/backup user config and launch Oak. + - Open Preferences > Behavior > Rendering. + - **Pass:** default backend is OpenGL; viewer, scopes, grade, and playback work. + +- [ ] **10.2 Vulkan runtime preflight** + - Run `vulkaninfo --summary` and note GPU, driver, and API version. + - Confirm `liboakvulkan.so/dylib/dll` exists next to the Oak binary. + - Run `olive-gtest --gtest_filter='DynamicRenderBackend.*'`. + - **Pass:** on a valid Vulkan setup, backend load/upload/download/Blit tests run; on invalid setups, tests SKIP and OpenGL fallback passes. + +- [ ] **10.3 Switch to Vulkan and restart** + - Select `Vulkan (experimental)` in Preferences, save, and restart Oak. + - Reopen Preferences and confirm Vulkan is still selected. + - Import and play a 4K clip. + - **Pass:** no crash; logs show actual backend matches selection; fallback to OpenGL is explicit if Vulkan fails. + +- [ ] **10.4 Vulkan viewer playback** + - On a working Vulkan setup, import two clips and play 10 seconds. + - Pause, step frames, scrub, and resize the viewer. + - **Pass:** viewer displays correctly; no black screen, flicker, or hang. + +- [ ] **10.5 Vulkan LUT/grade consistency** + - In Vulkan mode, load a LUT and make a strong three-way adjustment. + - Note the viewer appearance. + - Switch to OpenGL, reopen the same project at the same frame. + - **Pass:** colors are directionally consistent; no channel swap or gamma reversal. + +- [ ] **10.6 Vulkan proxy export** + - In Vulkan mode, generate and enable a proxy for a heavy clip. + - Export a software-encoded segment. + - **Pass:** export uses full-resolution source; output is playable and in sync. + +- [ ] **10.7 Vulkan scope behavior** + - In Vulkan mode, open Waveform, Vectorscope, and Histogram while grading. + - **Pass:** no crash; if scopes are backend-neutral and skip, that is documented; OpenGL scopes still work. + +- [ ] **10.8 Missing Vulkan runtime fallback** + - On a machine without Vulkan, select Vulkan and restart. + - Open a project and play. + - **Pass:** app starts; logs say Vulkan unavailable and OpenGL fallback active; Preferences can switch back to OpenGL. + +- [ ] **10.9 Switch back from Vulkan to OpenGL** + - Change Preferences from Vulkan to OpenGL, save, and restart. + - Play and export the same project. + - **Pass:** backend shows OpenGL; playback and export work; no stale Vulkan state. + +- [ ] **10.10 Dynamic OpenGL backend load** + - On an experimental dynamic-backend build, confirm `liboakgl` exists. + - Select OpenGL and restart. + - Play, scrub, open scopes, and exit. + - **Pass:** logs show dynamic OpenGL loaded; behavior matches default OpenGL; clean exit. + +- [ ] **10.11 Missing dynamic backend handling** + - Temporarily rename `liboakgl` / `oakgl.dll`. + - Launch Oak and open a project. + - **Pass:** no silent crash; logs report backend load failure; user can restore the library or use a non-dynamic build. + +--- + +## Bug report template + +When a task fails, file an issue with: + +- Platform, GPU, driver, FFmpeg version. +- Oak commit hash. +- Task number (e.g., **1.1**). +- Project file path and media types used. +- Exact reproduction steps. +- Expected vs. actual result. +- Is it reproducible every time? +- For export failures: attach `ffprobe` output and export settings screenshot. +- For Vulkan failures: attach `vulkaninfo --summary`, backend log, and whether OpenGL fallback occurred. + +## Release blockers + +Before calling v0.4.0-alpha ready, the following must pass: + +- Pre-flight, LUT, three-way wheels, all three scopes, waveform sync, BWF sync, audio meters, proxy generate/enable/delete, and software export. +- At least one hardware encoder environment (NVENC or VideoToolbox). +- OpenGL/Vulkan backend selection and persistence; Vulkan real rendering on at least one valid setup; clean OpenGL fallback when Vulkan is unavailable. +- Batch queue multi-job and cancel tests. +- Full editing chain regression test. +- No silent crashes remain; every failure is filed as an issue or documented as a known limitation.