Turned the two-step PCM transcode into one step and simplified/removed much of
the unnecessary infrastructure that supported it. This makes the code cleaner
and generally improves the code paths.
A huge optimization that ensures only the parts of a node graph that have
changed get pushed to the renderer. For thread-safety, the node graph is
copied elsewhere so that users can make changes asynchronously and the graph
can update when its threads are ready. Up until now, if an input value changed,
every node's values would be re-copied, or worse, if a connection was changed,
the entire graph would be recopied. This has been negligible in testing since
we've been largely testing with small graphs, but for massive projects, it's
important that this be as optimized as possible.
The audio rendering system uses an event-based mechanism where if audio needs
to be converted (i.e. to a different sample rate), it will create a conform
task and re-render that section again when the audio is ready. Unfortunately,
the export code didn't respect this and would start encoding audio once the
initial queue was done. This usually resulted in silent audio, but could also
result in "uninitialized" audio that would crash any float-based encoders
(e.g. AAC). Now we use a different signal that only emits once the queue is done
AND all conforms are done.
Implemented the ability to copy/paste blocks/clips in the timeline. This did
require some large scale changes and reworking of the copy/paste system
introduced a few commits ago, but should be largely functional now.
By using one thread per logical CPU thread, we seemed to completely saturate
the CPU which would kill the performance of the main/GUI thread (despite the
other threads being low priority). We now use half of the logical threads, which
still sees good CPU usage and minimal performance impact while allowing the
main thread to respond to user actions.
frames
Previously, when the video renderer received a dirty cache signal, it would
proceed to extract all frames from the range and queue them. However, this could
be extremely slow for long ranges since it had to iterate through the entire
range and calculate the individual frames it contained. Now, we use the same
range combining system as audio and automatically calculate the next frame
within the range only when necessary. Essentially the same work, but split up
over time and done only when needed leading to no discernible UI pause when
invalidating cache.
Various backend improvements are included in this commit, mostly for the
benefit of exporting. These include:
- Moving more non-GL code from OpenGL derivatives into base classes
- An "export mode" that changes the cache behavior of video backends
- Using the Viewer's UUID introduced a few commits ago
- No longer hardcoding the pixel format/render mode in the backend (since
they'll inevitably differ when exporting vs previewing)
- Improved signalling for frames that are completed
This was many changes that were largely fundamentally related. They included:
- More const modifiers to enforce read only node graphs
- Support for fragment and vertex shaders from the nodes
- Support for node code loaded externally (embedded into the binary)
- Fixed issue preventing two textures from being used in a shader
- Removed several unused functions and cleaned up code
- Fixed video media node misreading its matrix input
If the nodes are now stateless, there's nothing stopping the renderer from
rendering multiple frames at once. Earlier since the nodes held some of their
input/output data (and that data could change per frame), it was not possible
to render multiple frames at once without conflicts. Now that the node state is
held in render threads, they can do whatever they want at any time.
Previous iteration had params attached to the backend and the params couldn't
change without being destroyed and re-instantiated. This is not necessary in
this iteration so doing so only wastes resources.
Since we're now working with a separate proxy copy of the original node graph,
if the user changes a parameter in one of those nodes (triggering an
InvalidateCache signal), the values in our copied graph need to be updated
with these new values too.
Previous iterations would use mutexes to prevent changing of the graph
mid-render, however several user actions would need to capture these mutexes
causing the main thread to hang until the current render job (frame/range of
samples) was complete. We now copy the nodes necessary as part of the "compile"
process so that the main thread shouldn't need nearly as much blocking while
caching occurs.
Both audio and video renderers were working off the same invalidation signal
(i.e. changing audio would also trigger a re-render of the video). This is
obviously suboptimal and now they are separate.
Major refactoring work to try sharing as much code as possible between the
video renderers and audio renderers, as well as make them as
platform-independent as possible.
itsmattkc