docs: Update test plan.

This commit is contained in:
2026-07-13 22:01:13 +08:00
parent 93d92c93db
commit e37a87be67
@@ -0,0 +1,334 @@
# v0.4 Color, Audio & Performance — Community Testing Tasks
This document breaks the v0.4 manual test plan into small, volunteer-friendly tasks. Pick one, run it, and check the box when it passes. If it fails, file an issue using the bug template at the bottom.
## How to contribute
1. Pick an unchecked task below.
2. Build or download the latest v0.4.0-alpha build for your platform.
3. Follow the steps and compare against the pass criteria.
4. Check the box in your local copy and report the result in a comment/issue.
---
## 0. Pre-flight checks
- [ ] **0.1 Import sample media**
- Create a new project.
- Import a 4K H.264/HEVC clip with audio, plus a WAV file.
- Save the project, close Oak, reopen it.
- **Pass:** all clips are still online and no crash on reopen.
- [ ] **0.2 Smoke test on clean build**
- Launch Oak from a fresh build or CI package.
- Open the About dialog and confirm the version reads `v0.4.0-alpha`.
- **Pass:** app launches without crash and version is correct.
---
## 1. LUT & Color
- [ ] **1.1 Apply a `.cube` LUT**
- Add a clip to the timeline.
- Add an OCIO LUT effect and load a `.cube` LUT.
- Play 5 seconds and scrub the timeline.
- **Pass:** colors change visibly; no hang; scrubbing updates the viewer.
- [ ] **1.2 Apply a `.3dl` LUT**
- Repeat 1.1 with a `.3dl` LUT.
- Toggle the effect on/off.
- **Pass:** `.3dl` works; toggle is instant; no crash.
- [ ] **1.3 Corrupt LUT handling**
- Try to load a malformed LUT file.
- Observe the error UI, then continue playing the original clip.
- Save, close, and reopen the project.
- **Pass:** no crash; original clip still plays; project reopens cleanly.
- [ ] **1.4 Three-way color wheels**
- Open the three-way color wheels panel.
- Adjust Shadows, Midtones, and Highlights on a color-chart clip.
- Play, pause, and step frames.
- **Pass:** each region changes the image; undo/redo works; values persist after reopen.
- [ ] **1.5 LUT + proxy combination**
- Generate a proxy for a heavy clip and enable `Use Proxy`.
- Apply a LUT and three-way adjustments.
- Disable proxy and compare the same frame.
- **Pass:** color looks consistent; export uses full-resolution source, not proxy.
---
## 2. Scopes
- [ ] **2.1 Waveform scope**
- Open the Scope panel and select Waveform.
- Play `color_chart.mov` and adjust brightness/exposure.
- **Pass:** waveform updates with current frame and grade; no lag buildup.
- [ ] **2.2 Vectorscope**
- Switch the Scope panel to Vectorscope.
- Use a frame with skin tones and saturated colors.
- Adjust saturation or color wheels.
- **Pass:** plot moves with saturation/hue changes; switching panels does not crash.
- [ ] **2.3 Histogram**
- Switch the Scope panel to Histogram.
- Adjust brightness/contrast on a high-contrast or grayscale frame.
- Play and pause on different frames.
- **Pass:** histogram updates correctly; no ghosting from previous frames.
- [ ] **2.4 Scope switching stress**
- Switch between Waveform, Vectorscope, and Histogram 20 times.
- While switching, scrub the timeline and adjust a grade.
- **Pass:** no OpenGL/shader crash; UI stays responsive.
---
## 3. Waveform Auto-Sync
- [ ] **3.1 Dual-system audio sync**
- Add a video clip with in-camera audio and a separate external WAV.
- Offset the external audio by 13 seconds.
- Select both clips and run `Sync by Waveform`.
- Play the slate/lip-sync section.
- **Pass:** slate peaks align; lip-sync error is ≤ 1 frame; no clips deleted.
- [ ] **3.2 Multi-clip sync**
- Add 3 video clips and 3 external audio tracks with different offsets.
- Select all and run waveform sync.
- **Pass:** each pair aligns; unmatched clips stay in place or report failure.
- [ ] **3.3 Low-quality reference audio**
- Use a noisy or quiet reference track for waveform sync.
- **Pass:** if sync fails, it does not create a false alignment; user gets a warning or the clip stays put.
- [ ] **3.4 Sync persistence**
- After a successful sync, save the project, close Oak, and reopen.
- Play the sync point.
- **Pass:** clip positions are preserved; sync does not drift on reopen.
---
## 4. BWF Timecode Sync
- [ ] **4.1 Read BWF timecode**
- Import a WAV file with BWF timecode metadata.
- Check the clip properties/timecode field.
- Place it on the timeline and run `Sync by Timecode`.
- **Pass:** start timecode is recognized; sync does not jump to an extreme timestamp.
- [ ] **4.2 BWF + video timecode alignment**
- Import a video clip and a BWF audio file with matching timecode.
- Run `Sync by Source Timecode`.
- **Pass:** audio and video align; timeline position matches the timecode difference.
- [ ] **4.3 Missing timecode fallback**
- Select a plain WAV without BWF timecode and run `Sync by Timecode`.
- **Pass:** a clear error or skip; no extreme offset applied.
---
## 5. Audio Meters
- [ ] **5.1 VU meter response**
- Add `noisy_dialogue.wav` to the timeline.
- Open the audio meter panel and play silence, dialogue, and music sections.
- **Pass:** needle moves with volume; drops on silence; settles after stop.
- [ ] **5.2 LUFS reading**
- Play 30 seconds of dialogue or music.
- Watch the short-term/integrated LUFS readout.
- Change clip gain and replay.
- **Pass:** LUFS follows gain changes; no NaN, inf, or wild jumps.
- [ ] **5.3 Multi-channel audio**
- Import stereo and multi-channel clips.
- Play and mute/lower one clip.
- **Pass:** channels display correctly; mute/gain changes are instant.
---
## 6. Proxy Media
- [ ] **6.1 Generate proxy**
- Add an 8K or heavy 4K clip to the timeline.
- Right-click → `Proxy > Generate Proxy`.
- Watch the task list and proxy cache folder.
- **Pass:** `.working` file becomes `.mp4`; `.working` is cleaned; UI does not block.
- [ ] **6.2 Enable/disable proxy**
- After generation, check `Proxy > Use Proxy` and play.
- Uncheck `Use Proxy` and play again.
- **Pass:** proxy playback works; disabling returns to original; missing proxy falls back safely.
- [ ] **6.3 Reveal and delete proxy**
- Run `Proxy > Reveal Proxy` and confirm the folder opens.
- Run `Proxy > Delete Proxy` and play again.
- **Pass:** reveal points to cache/proxy; delete removes proxy and working files; playback falls back.
- [ ] **6.4 Proxy state persistence**
- Generate, enable proxy, and save.
- Close and reopen the project.
- **Pass:** `Use Proxy` state is restored; proxy file is used if it exists.
- [ ] **6.5 Proxy failure path**
- Temporarily break `ffmpeg` or use an unreadable source.
- Try to generate a proxy.
- **Pass:** clear failure message; `.working` cleaned; original clip still plays.
- [ ] **6.6 Export ignores proxy**
- Generate a 720p proxy for a 4K clip and enable it.
- Export a 4K segment.
- **Pass:** exported file is 4K and uses full-resolution source.
---
## 7. Hardware-Accelerated Export
- [ ] **7.1 NVENC export (NVIDIA only)**
- On an NVIDIA machine, choose H.264/H.265 NVENC in export settings.
- Export a 30-second 4K segment.
- Check codec with `ffprobe`.
- **Pass:** export succeeds; codec is correct; unavailable hardware reports a clear error.
- [ ] **7.2 VideoToolbox export (macOS only)**
- On macOS, choose VideoToolbox H.264/H.265.
- Export a 30-second 4K segment.
- **Pass:** export succeeds; output plays; system load looks like hardware encoding.
- [ ] **7.3 Hardware export fallback**
- Select a hardware encoder your machine does not support.
- Attempt export.
- **Pass:** understandable failure; option to switch to software encoding.
---
## 8. Batch Render Queue
- [ ] **8.1 Multi-job queue**
- Create 3 sequences: a short clip, one with a LUT, one with proxy.
- Add all 3 to the batch render queue and start.
- **Pass:** jobs run in order; each output file is created; one failure does not crash the app.
- [ ] **8.2 Queue cancel**
- Add a long export job, start it, then cancel immediately.
- Add a short job and run it.
- **Pass:** cancel leaves no dead state; next job runs; partial file status is clear.
- [ ] **8.3 Queue project save**
- Configure several queue jobs and save the project.
- Close and reopen Oak.
- **Pass:** behavior is consistent: queue is either restored cleanly or empty, never half-broken.
---
## 9. Regression & Path Tests
- [ ] **9.1 Full editing chain**
- Make a 60-second sequence mixing 4K/8K, external audio, LUT, three-way grade, and proxy.
- Waveform-sync some clips.
- Open Scopes and audio meters and play the whole sequence.
- Export a software-encoded version.
- **Pass:** no crash; exported file is in sync; color and audio match preview.
- [ ] **9.2 Chinese and space characters in paths**
- Put project, media, cache, and export target in a path like `Oak v04 测试/素材 A`.
- Repeat proxy generation, LUT load, and export.
- **Pass:** paths work; `Reveal Proxy` and exported file open correctly.
- [ ] **9.3 Long playback stability**
- Open a 4K/8K project and loop playback for 20 minutes.
- Toggle proxy, scopes, and audio meters during playback.
- **Pass:** memory does not grow uncontrollably; app remains responsive; save works after stop.
---
## 10. Graphics Backend (OpenGL/Vulkan)
> Only test Vulkan if `vulkaninfo --summary` reports a working instance and device. Otherwise, mark Vulkan tasks as skipped and test OpenGL fallback only.
- [ ] **10.1 Default OpenGL backend**
- Delete/backup user config and launch Oak.
- Open Preferences > Behavior > Rendering.
- **Pass:** default backend is OpenGL; viewer, scopes, grade, and playback work.
- [ ] **10.2 Vulkan runtime preflight**
- Run `vulkaninfo --summary` and note GPU, driver, and API version.
- Confirm `liboakvulkan.so/dylib/dll` exists next to the Oak binary.
- Run `olive-gtest --gtest_filter='DynamicRenderBackend.*'`.
- **Pass:** on a valid Vulkan setup, backend load/upload/download/Blit tests run; on invalid setups, tests SKIP and OpenGL fallback passes.
- [ ] **10.3 Switch to Vulkan and restart**
- Select `Vulkan (experimental)` in Preferences, save, and restart Oak.
- Reopen Preferences and confirm Vulkan is still selected.
- Import and play a 4K clip.
- **Pass:** no crash; logs show actual backend matches selection; fallback to OpenGL is explicit if Vulkan fails.
- [ ] **10.4 Vulkan viewer playback**
- On a working Vulkan setup, import two clips and play 10 seconds.
- Pause, step frames, scrub, and resize the viewer.
- **Pass:** viewer displays correctly; no black screen, flicker, or hang.
- [ ] **10.5 Vulkan LUT/grade consistency**
- In Vulkan mode, load a LUT and make a strong three-way adjustment.
- Note the viewer appearance.
- Switch to OpenGL, reopen the same project at the same frame.
- **Pass:** colors are directionally consistent; no channel swap or gamma reversal.
- [ ] **10.6 Vulkan proxy export**
- In Vulkan mode, generate and enable a proxy for a heavy clip.
- Export a software-encoded segment.
- **Pass:** export uses full-resolution source; output is playable and in sync.
- [ ] **10.7 Vulkan scope behavior**
- In Vulkan mode, open Waveform, Vectorscope, and Histogram while grading.
- **Pass:** no crash; if scopes are backend-neutral and skip, that is documented; OpenGL scopes still work.
- [ ] **10.8 Missing Vulkan runtime fallback**
- On a machine without Vulkan, select Vulkan and restart.
- Open a project and play.
- **Pass:** app starts; logs say Vulkan unavailable and OpenGL fallback active; Preferences can switch back to OpenGL.
- [ ] **10.9 Switch back from Vulkan to OpenGL**
- Change Preferences from Vulkan to OpenGL, save, and restart.
- Play and export the same project.
- **Pass:** backend shows OpenGL; playback and export work; no stale Vulkan state.
- [ ] **10.10 Dynamic OpenGL backend load**
- On an experimental dynamic-backend build, confirm `liboakgl` exists.
- Select OpenGL and restart.
- Play, scrub, open scopes, and exit.
- **Pass:** logs show dynamic OpenGL loaded; behavior matches default OpenGL; clean exit.
- [ ] **10.11 Missing dynamic backend handling**
- Temporarily rename `liboakgl` / `oakgl.dll`.
- Launch Oak and open a project.
- **Pass:** no silent crash; logs report backend load failure; user can restore the library or use a non-dynamic build.
---
## Bug report template
When a task fails, file an issue with:
- Platform, GPU, driver, FFmpeg version.
- Oak commit hash.
- Task number (e.g., **1.1**).
- Project file path and media types used.
- Exact reproduction steps.
- Expected vs. actual result.
- Is it reproducible every time?
- For export failures: attach `ffprobe` output and export settings screenshot.
- For Vulkan failures: attach `vulkaninfo --summary`, backend log, and whether OpenGL fallback occurred.
## Release blockers
Before calling v0.4.0-alpha ready, the following must pass:
- Pre-flight, LUT, three-way wheels, all three scopes, waveform sync, BWF sync, audio meters, proxy generate/enable/delete, and software export.
- At least one hardware encoder environment (NVENC or VideoToolbox).
- OpenGL/Vulkan backend selection and persistence; Vulkan real rendering on at least one valid setup; clean OpenGL fallback when Vulkan is unavailable.
- Batch queue multi-job and cancel tests.
- Full editing chain regression test.
- No silent crashes remain; every failure is filed as an issue or documented as a known limitation.