Fixes a number of playback stuttering and general UI lag issues by setting all
background tasks to IdlePriority rather than LowPriority. While it was assumed
LowPriority tasks would always get scheduled below NormalPriority (e.g. main
thread) tasks, it turns out this is not always the case. If the background tasks
start consuming a lot of CPU cycles, the scheduler may use "dynamic scheduling"
to schedule them above the main thread regardless leading to UI lag. This is
apparently the case for all thread priorities apart from IdlePriority, which
is allegedly a special case where threads are *only* scheduled when other
threads aren't busy ensuring the main thread stays responsive.
context
More intuitive code flow and allows the user to undock the viewer (which
forcibly destroys and recreates the context) and the viewer will handle
creation of the new texture in said new context.
The workers run in separate threads meaning if any significant change is made
(e.g. parameters changing, or even closing the program), these workers may still
be mid-render. This is particularly problematic when closing since the nodes a
worker is rendering may be deleted mid-render. Render backends now have a
function that pauses the main thread (but starts a second event loop so the UI
isn't frozen) until the worker threads are all finished. This way, massive
changes can be made safely without race conditions.
We try to handle aspect ratios in metadata (e.g. stretching 1440x1080 videos
out to 1920x1080 when requested), and FFmpeg will usually return a 1/1
aspect ratio even if it can't determine an aspect ratio. However, as mentioned
in the documentation, sometimes it returns a 0/0 aspect ratio when it can't
determine an aspect ratio, which we didn't handle and would lead to the code
allocating a buffer with a 0px height. This commit handles both 1/1 and 0/0
aspect ratios in accordance with the FFmpeg documentation.
Reference: https://www.ffmpeg.org/doxygen/4.0/structAVFrame.html#a62f9c20541a83d37db7072126ff0060d
Implementation isn't perfect yet, viewer/renderer doesn't update yet when
the preference is changed so a sequence needs to be re-opened for the change to
take effect.
By using one thread per logical CPU thread, we seemed to completely saturate
the CPU which would kill the performance of the main/GUI thread (despite the
other threads being low priority). We now use half of the logical threads, which
still sees good CPU usage and minimal performance impact while allowing the
main thread to respond to user actions.
This mostly builds on the keyframing we already set up for video, but the
audio renderer will now appropriately updated keyframe inputs per sample in
accordance with keyframe values.
Since EmitCachedFrameReady() makes a copy of the texture, we can economize a lot
by re-using the same copied texture for all the times rather than making a
separate copy for each time.
The encoder was moved to its own thread and will transcode the PCM from the
audio renderer into the chosen codec while the video frames are still
received. The implementation isn't perfect and could use some cleaning up, but
it is functional at the moment.
Sending a texture directly from the texture cache is dangerous since once the
reference is relinquished, it could be picked up and used by another thread.
Copying the texture to a separate one takes a little extra time but lets the
viewer remain in control of that texture.
Various backend improvements are included in this commit, mostly for the
benefit of exporting. These include:
- Moving more non-GL code from OpenGL derivatives into base classes
- An "export mode" that changes the cache behavior of video backends
- Using the Viewer's UUID introduced a few commits ago
- No longer hardcoding the pixel format/render mode in the backend (since
they'll inevitably differ when exporting vs previewing)
- Improved signalling for frames that are completed
OCIO config was set on a per-project basis, but we were using a singleton for
the ColorManager that would break if more than one project was ever open.
Now the ColorManager belongs to the Project and is always accessed through the
Project.
This commit adds the background functionality of the render cache invalidating
whenever a footage's color space is changed. This includes when the project's
configuration is changed as well.
Uniform values cannot be set if the shader is not bound, and calling Blit would
release the shader making subsequent iteration count updates not work. This
commit makes sure the shader is bound before trying to do so.
Ultimately it would be nice if nodes could configure this themselves, but
CLAMP_TO_BORDER behavior can be written manually into the shader if that
behavior is desired, whereas CLAMP_TO_EDGE cannot.
Creating and destroying textures is a slow process, particularly when we can
re-use them throughout most of the render chain. We now keep them stored so
they can be re-used which improves performance substantially.
itsmattkc