* [x] Re-send operations that weren't sent while disconnected
* [x] Apply other clients' operations that were missed while
disconnected
* [x] Update collaborators that joined / left while disconnected
* [x] Inform current collaborators that your peer id has changed
* [x] Refresh channel buffer collaborators on server restart
* [x] randomized test
This PR ships a series of optimizations for the semantic search engine.
Mostly focused on removing invalid states, optimizing requests to
OpenAI, and reducing token usage.
Release Notes (Preview-Only):
- Added eager incremental indexing in the background on a debounce.
- Added a local embeddings cache for reducing redundant calls to OpenAI.
- Moved to an Embeddings Queue model which ensures optimal batch sizes
at the token level, and atomic file & document writes.
- Adjusted OpenAI Embedding API requests to use provided backoff delays
during Rate Limiting.
- Removed flush races between parsing files step and embedding queue
steps.
- Moved truncation to parsing step reducing the probability that OpenAI
encounters bad data.
Fresh off the press, memchr 2.6.0 adds vector search routines for
aarch64. That directly improves our search performance for both text and
regex searches. Per BurntSushi's claims, the simple string searches in
ripgrep got ~2 times faster (more details available in
https://github.com/BurntSushi/memchr/pull/129).
Release Notes:
- N/A
Optimization to the Semantic Indexing Engine.
We've transitioned from a framework in which the entire project tree is
walked at each index command, to an eager queuing method, in which an
initial queue of outstanding indexing work is initialized upon workspace
creation, and then subscriptions are leveraged for file change events to
continually keep an updated view on outstanding work.
This optimization contributes towards quicker user feedback, when
initializing or using Semantic Search functionality. It also opens the
doors towards better transparency across the system on outstanding
indexing work.
Release Notes:
- Refactored index operation queue to an eager queuing framework.
- Moved semantic search initialization to workspace creation.
- Adjusted rate limiting strategy on api delays to reduce time spent
waiting for rate limits.
Nathan Sobo