This introduces semantic indexing in Zed based on chunking text from
files in the developer's workspace and creating vector embeddings using
an embedding model. As part of this, we've created an embeddings
provider trait that allows us to work with OpenAI, a local Ollama model,
or a Zed hosted embedding.
The semantic index is built by breaking down text for known
(programming) languages into manageable chunks that are smaller than the
max token size. Each chunk is then fed to a language model to create a
high dimensional vector which is then normalized to a unit vector to
allow fast comparison with other vectors with a simple dot product.
Alongside the vector, we store the path of the file and the range within
the document where the vector was sourced from.
Zed will soon grok contextual similarity across different text snippets,
allowing for natural language search beyond keyword matching. This is
being put together both for human-based search as well as providing
results to Large Language Models to allow them to refine how they help
developers.
Remaining todo:
* [x] Change `provider` to `model` within the zed hosted embeddings
database (as its currently a combo of the provider and the model in one
name)
Release Notes:
- N/A
---------
Co-authored-by: Nathan Sobo <nathan@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Conrad Irwin <conrad@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Antonio <antonio@zed.dev>
@ConradIrwin This is a followup for #9035 as agreed.
Release Notes:
- Fixed mention notifications are updated when channel message is
updated. And mention notifications are removed when message is removed.
---------
Co-authored-by: Bennet Bo Fenner <53836821+bennetbo@users.noreply.github.com>
This PR modifies the extension installation and update process to
respect version constraints (schema version and Wasm API version) to
ensure only compatible versions of extensions are able to be installed.
To achieve this there is a new `GET /extensions/updates` endpoint that
will return extension versions based on the provided constraints.
Release Notes:
- N/A
---------
Co-authored-by: Max <max@zed.dev>
This PR extends the extension API with some additional features to
support auto-updating extensions:
- The `GET /extensions` endpoint now accepts an optional `ids` parameter
that can be used to filter the results down to just the extensions with
the specified IDs.
- This should be a comma-delimited list of extension IDs (e.g.,
`wgsl,gleam,tokyo-night`).
- A new `GET /extensions/:extension_id` endpoint that returns all of the
extension versions for a particular extension.
Extracted from #9890, as these changes can be landed and deployed
independently.
Release Notes:
- N/A
Co-authored-by: Max <max@zed.dev>
* Store extensions versions' wasm API version in the database
* Share a common struct for extension API responses between collab and
client
* Add wasm API version and schema version to extension API responses
Release Notes:
- N/A
Co-authored-by: Marshall <marshall@zed.dev>
This PR provides some of the plumbing needed for a "remote" zed
instance.
The way this will work is:
* From zed on your laptop you'll be able to manage a set of dev servers,
each of which is identified by a token.
* You'll run `zed --dev-server-token XXXX` to boot a remotable dev
server.
* From the zed on your laptop you'll be able to open directories and
work on the projects on the remote server (exactly like collaboration
works today).
For now all this PR does is provide the ability for a zed instance to
sign in
using a "dev server token". The next steps will be:
* Adding support to the collaboration protocol to instruct a dev server
to "open" a directory and share it into a channel.
* Adding UI to manage these servers and tokens (manually for now)
Related #5347
Release Notes:
- N/A
---------
Co-authored-by: Nathan <nathan@zed.dev>
Co-authored-by: Antonio <antonio@zed.dev>
Resurrected this from some assistant work I did in Spring of 2023.
- [x] Resurrect streaming responses
- [x] Use streaming responses to enable AI via Zed's servers by default
(but preserve API key option for now)
- [x] Simplify protobuf
- [x] Proxy to OpenAI on zed.dev
- [x] Proxy to Gemini on zed.dev
- [x] Improve UX for switching between openAI and google models
- We current disallow cycling when setting a custom model, but we need a
better solution to keep OpenAI models available while testing the google
ones
- [x] Show remaining tokens correctly for Google models
- [x] Remove semantic index
- [x] Delete `ai` crate
- [x] Cloud front so we can ban abuse
- [x] Rate-limiting
- [x] Fix panic when using inline assistant
- [x] Double check the upgraded `AssistantSettings` are
backwards-compatible
- [x] Add hosted LLM interaction behind a `language-models` feature
flag.
Release Notes:
- We are temporarily removing the semantic index in order to redesign it
from scratch.
---------
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Thorsten <thorsten@zed.dev>
Co-authored-by: Max <max@zed.dev>
This should significantly reduce database load on redeploy.
Co-Authored-By: Max <max@zed.dev>
Co-Authored-By: Nathan <nathan@zed.dev>
Release Notes:
- Reduced likelihood of being disconnected during deploys
Co-authored-by: Max <max@zed.dev>
Co-authored-by: Nathan <nathan@zed.dev>
This reduces the server time to compute the hash from 40ms to 5µs,
which should remove this as a noticable chunk of CPU time in production.
(An attacker who has access to our database will now need only 10^54
years of CPU time instead of 10^58 to brute force a token).
Release Notes:
- Improved sign in latency by 40ms.
Add plumbing for hosted projects. This will currently show them if they
exist
but provides no UX to create/rename/delete them.
Also changed the `ChannelId` type to not auto-cast to u64; this avoids
type
confusion if you have multiple id types.
Release Notes:
- N/A
This PR makes it so extensions are secondarily sorted by their name
(instead of by ID) after we sort them by their download count.
Release Notes:
- N/A
This PR adds a REST API to the collab server for searching and
downloading extensions. Previously, we had implemented this API in
zed.dev directly, but this implementation is better, because we use the
collab database to store the download counts for extensions.
Release Notes:
- N/A
---------
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Conrad <conrad@zed.dev>
1. The client-side comparison was wrong
2. The server never told the client about the version it remembered
3. The server generated broken timestamps in some cases
Release Notes:
- Fixed the notes/chat appearing as unread too often
**or**
- N/A
- Open channel notes and chat on channel click
- WIP
- Fix compile error
- Don't join live kit until requested
- Track in_call state separately from in_room
Release Notes:
- Improved channels: you can now be in a channel without joining the
audio call automatically
**or**
- N/A
---------
Co-authored-by: Nathan Sobo <nathan@zed.dev>
We noticed the following message in my logs when trying to debug some
lag when collaborating:
```
2024-02-06T09:42:09-08:00 [ERROR] error handling message. client_id:3, sender_id:Some(PeerId { owner_id: 327, id: 1123430 }), type:GetCompletions, error:no such connection: 0/0
```
That `0/0` looks like a bogus connection id, constructed via a derived
`Default`. We didn't ever find a code path that would *use* a default
`ConnectionId` and lead to this error, but it did seem like an
improvement to not have a `Default` for that type.
Release Notes:
- N/A
Co-authored-by: Marshall <marshall@zed.dev>
Feature
- [x] Allow to click on reply to go to the real message
- [x] In chat
- [x] Show only a part of the message that you reply to
- [x] In chat
- [x] In reply preview
TODO’s
- [x] Fix migration
- [x] timestamp(in filename)
- [x] remove the reference to the reply_message_id
- [x] Fix markdown cache for reply message
- [x] Fix spacing when first message is a reply to you and you want to
reply to that message.
- [x] Fetch message that you replied to
- [x] allow fetching messages that are not inside the current view
- [x] When message is deleted, we should show a text like `message
deleted` or something
- [x] Show correct GitHub username + icon after `Replied to: `
- [x] Show correct message(now it's hard-coded)
- [x] Add icon to reply + add the onClick logic
- [x] Show message that you want to reply to
- [x] Allow to click away the message that you want to reply to
- [x] Fix hard-coded GitHub user + icon after `Reply tp:`
- [x] Add tests
<img width="242" alt="Screenshot 2024-02-06 at 20 51 40"
src="https://github.com/zed-industries/zed/assets/62463826/a7a5f3e0-dee3-4d38-95db-258b169e4498">
<img width="240" alt="Screenshot 2024-02-06 at 20 52 02"
src="https://github.com/zed-industries/zed/assets/62463826/3e136de3-4135-4c07-bd43-30089b677c0a">
Release Notes:
- Added the ability to reply to a message.
- Added highlight message when you click on mention notifications or a
reply message.
---------
Co-authored-by: Bennet Bo Fenner <53836821+bennetbo@users.noreply.github.com>
Co-authored-by: Conrad Irwin <conrad.irwin@gmail.com>
Remco SmitsandBennet Bo Fenner
Max Brunsfeld
Hans
Piotr Osiewicz