Turned the two-step PCM transcode into one step and simplified/removed much of
the unnecessary infrastructure that supported it. This makes the code cleaner
and generally improves the code paths.
A huge optimization that ensures only the parts of a node graph that have
changed get pushed to the renderer. For thread-safety, the node graph is
copied elsewhere so that users can make changes asynchronously and the graph
can update when its threads are ready. Up until now, if an input value changed,
every node's values would be re-copied, or worse, if a connection was changed,
the entire graph would be recopied. This has been negligible in testing since
we've been largely testing with small graphs, but for massive projects, it's
important that this be as optimized as possible.
functions
Indexing is a lengthy process and had a high chance of getting RenderWorkers
stuck doing it rather than being responsive to cache requests. This commit
introduces a system where workers never index media, but instead signal that
media is not ready to their RenderBackends which ensure that the media gets
indexed and re-queues the affected frames when those indexes are ready.
context
More intuitive code flow and allows the user to undock the viewer (which
forcibly destroys and recreates the context) and the viewer will handle
creation of the new texture in said new context.
The workers run in separate threads meaning if any significant change is made
(e.g. parameters changing, or even closing the program), these workers may still
be mid-render. This is particularly problematic when closing since the nodes a
worker is rendering may be deleted mid-render. Render backends now have a
function that pauses the main thread (but starts a second event loop so the UI
isn't frozen) until the worker threads are all finished. This way, massive
changes can be made safely without race conditions.
frames
Previously, when the video renderer received a dirty cache signal, it would
proceed to extract all frames from the range and queue them. However, this could
be extremely slow for long ranges since it had to iterate through the entire
range and calculate the individual frames it contained. Now, we use the same
range combining system as audio and automatically calculate the next frame
within the range only when necessary. Essentially the same work, but split up
over time and done only when needed leading to no discernible UI pause when
invalidating cache.
The renderer backend can now distinguish between jobs. Previously if two jobs
of the same frame were started (which is legal if the user made a change while
frames were still being rendered), an earlier job in some situations could
finish AFTER a later job, and the backend would have no way of distinguishing
between them. This meant a frame could be erroneously set to an old value
rather than the newest. This commit introduces job identification so that old
jobs are automatically discarded.
The encoder was moved to its own thread and will transcode the PCM from the
audio renderer into the chosen codec while the video frames are still
received. The implementation isn't perfect and could use some cleaning up, but
it is functional at the moment.
Various backend improvements are included in this commit, mostly for the
benefit of exporting. These include:
- Moving more non-GL code from OpenGL derivatives into base classes
- An "export mode" that changes the cache behavior of video backends
- Using the Viewer's UUID introduced a few commits ago
- No longer hardcoding the pixel format/render mode in the backend (since
they'll inevitably differ when exporting vs previewing)
- Improved signalling for frames that are completed
This was many changes that were largely fundamentally related. They included:
- More const modifiers to enforce read only node graphs
- Support for fragment and vertex shaders from the nodes
- Support for node code loaded externally (embedded into the binary)
- Fixed issue preventing two textures from being used in a shader
- Removed several unused functions and cleaned up code
- Fixed video media node misreading its matrix input
Workers run in different threads and the backends can poll whether the worker
is currently busy or not. However the previous iteration has the worker (and an
atomic int) provide the busy state which could easily desync with the main
thread (since all workers run in different threads). By holding the busy states
in the main thread, the main thread will always be able to poll the busy state
accurately.
If the nodes are now stateless, there's nothing stopping the renderer from
rendering multiple frames at once. Earlier since the nodes held some of their
input/output data (and that data could change per frame), it was not possible
to render multiple frames at once without conflicts. Now that the node state is
held in render threads, they can do whatever they want at any time.
I was under the impression Q_DISABLE_COPY_MOVE was a much older function than
it is. Since we historically target 5.6 and these functions are small
convenience functions, I just implemented them in the actual codebase.
Since we're now working with a separate proxy copy of the original node graph,
if the user changes a parameter in one of those nodes (triggering an
InvalidateCache signal), the values in our copied graph need to be updated
with these new values too.
Previous iterations would use mutexes to prevent changing of the graph
mid-render, however several user actions would need to capture these mutexes
causing the main thread to hang until the current render job (frame/range of
samples) was complete. We now copy the nodes necessary as part of the "compile"
process so that the main thread shouldn't need nearly as much blocking while
caching occurs.
Both audio and video renderers were working off the same invalidation signal
(i.e. changing audio would also trigger a re-render of the video). This is
obviously suboptimal and now they are separate.
Major refactoring work to try sharing as much code as possible between the
video renderers and audio renderers, as well as make them as
platform-independent as possible.
The previous iteration was fairly OpenGL-heavy. It's now been separated into
a base class that is OpenGL independent and a derived class that is
OpenGL-based. Over time this should allow for portability away from OpenGL
if necessary.
itsmattkc