A huge optimization that ensures only the parts of a node graph that have
changed get pushed to the renderer. For thread-safety, the node graph is
copied elsewhere so that users can make changes asynchronously and the graph
can update when its threads are ready. Up until now, if an input value changed,
every node's values would be re-copied, or worse, if a connection was changed,
the entire graph would be recopied. This has been negligible in testing since
we've been largely testing with small graphs, but for massive projects, it's
important that this be as optimized as possible.
Fixes a number of playback stuttering and general UI lag issues by setting all
background tasks to IdlePriority rather than LowPriority. While it was assumed
LowPriority tasks would always get scheduled below NormalPriority (e.g. main
thread) tasks, it turns out this is not always the case. If the background tasks
start consuming a lot of CPU cycles, the scheduler may use "dynamic scheduling"
to schedule them above the main thread regardless leading to UI lag. This is
apparently the case for all thread priorities apart from IdlePriority, which
is allegedly a special case where threads are *only* scheduled when other
threads aren't busy ensuring the main thread stays responsive.
context
More intuitive code flow and allows the user to undock the viewer (which
forcibly destroys and recreates the context) and the viewer will handle
creation of the new texture in said new context.
The workers run in separate threads meaning if any significant change is made
(e.g. parameters changing, or even closing the program), these workers may still
be mid-render. This is particularly problematic when closing since the nodes a
worker is rendering may be deleted mid-render. Render backends now have a
function that pauses the main thread (but starts a second event loop so the UI
isn't frozen) until the worker threads are all finished. This way, massive
changes can be made safely without race conditions.
By using one thread per logical CPU thread, we seemed to completely saturate
the CPU which would kill the performance of the main/GUI thread (despite the
other threads being low priority). We now use half of the logical threads, which
still sees good CPU usage and minimal performance impact while allowing the
main thread to respond to user actions.
Since EmitCachedFrameReady() makes a copy of the texture, we can economize a lot
by re-using the same copied texture for all the times rather than making a
separate copy for each time.
Sending a texture directly from the texture cache is dangerous since once the
reference is relinquished, it could be picked up and used by another thread.
Copying the texture to a separate one takes a little extra time but lets the
viewer remain in control of that texture.
Various backend improvements are included in this commit, mostly for the
benefit of exporting. These include:
- Moving more non-GL code from OpenGL derivatives into base classes
- An "export mode" that changes the cache behavior of video backends
- Using the Viewer's UUID introduced a few commits ago
- No longer hardcoding the pixel format/render mode in the backend (since
they'll inevitably differ when exporting vs previewing)
- Improved signalling for frames that are completed
Creating and destroying textures is a slow process, particularly when we can
re-use them throughout most of the render chain. We now keep them stored so
they can be re-used which improves performance substantially.
This was many changes that were largely fundamentally related. They included:
- More const modifiers to enforce read only node graphs
- Support for fragment and vertex shaders from the nodes
- Support for node code loaded externally (embedded into the binary)
- Fixed issue preventing two textures from being used in a shader
- Removed several unused functions and cleaned up code
- Fixed video media node misreading its matrix input
Workers run in different threads and the backends can poll whether the worker
is currently busy or not. However the previous iteration has the worker (and an
atomic int) provide the busy state which could easily desync with the main
thread (since all workers run in different threads). By holding the busy states
in the main thread, the main thread will always be able to poll the busy state
accurately.
If the nodes are now stateless, there's nothing stopping the renderer from
rendering multiple frames at once. Earlier since the nodes held some of their
input/output data (and that data could change per frame), it was not possible
to render multiple frames at once without conflicts. Now that the node state is
held in render threads, they can do whatever they want at any time.
If Nodes only have the one output, we don't need to do so much differentiation
between them. Previous iteration used outputs as like a distinct function
within a Node (e.g. length output would return one result, buffer output would
produce a different result - each run different code to produce their results).
Now in this iteration, it's more accurate to say a Node is just one function
(which seems more appropriate for a node system anyway).
Old code assumes that a NodeParam's parent will always be a Node. The function
has been separated off and tweaked in the event that this is not the case.
Previous iterations would use mutexes to prevent changing of the graph
mid-render, however several user actions would need to capture these mutexes
causing the main thread to hang until the current render job (frame/range of
samples) was complete. We now copy the nodes necessary as part of the "compile"
process so that the main thread shouldn't need nearly as much blocking while
caching occurs.
Updating values rapidly would cause strange jitters as a
byproduct of the viewer trying to update from the renderer while
it was still working. Rather than the viewer trying to access the
the renderer, we now send textures in the initial update signal
to keep everything synchronized.
Major refactoring work to try sharing as much code as possible between the
video renderers and audio renderers, as well as make them as
platform-independent as possible.
commit
For testing the new iteration, the texture cache disk download was written
into the main thread instead of into the separate threads. Now they're back
in separate threads again.
Also I think some of these files probably should have been in the previous
commit.
Once again, conceptually this system should work, however it does not seem to
be the most efficient and it wouldn't surprise me if the multithreading was
eventually upgraded to an even more coherent system one day. However for
"core principles" this should be fairly decent.
The previous iteration was fairly OpenGL-heavy. It's now been separated into
a base class that is OpenGL independent and a derived class that is
OpenGL-based. Over time this should allow for portability away from OpenGL
if necessary.
itsmattkc