We also eliminate the `completion` crate and moved its logic into
`LanguageModelRegistry`.
Release Notes:
- N/A
---------
Co-authored-by: Nathan <nathan@zed.dev>
<img width="624" alt="image"
src="https://github.com/user-attachments/assets/f492b0bd-14c3-49e2-b2ff-dc78e52b0815">
- [x] Correctly set custom model token count
- [x] How to count tokens for Gemini models?
- [x] Feature flag zed.dev provider
- [x] Figure out how to configure custom models
- [ ] Update docs
Release Notes:
- Added support for quickly switching between multiple language model
providers in the assistant panel
---------
Co-authored-by: Antonio <antonio@zed.dev>
We will soon need `semantic_index` to be able to use
`CompletionProvider`. This is currently impossible due to a cyclic crate
dependency, because `CompletionProvider` lives in the `assistant` crate,
which depends on `semantic_index`.
This PR breaks the dependency cycle by extracting two crates out of
`assistant`: `language_model` and `completion`.
Only one piece of logic changed: [this
code](https://github.com/zed-industries/zed/commit/922fcaf5a6076e56890373035b1065b13512546d#diff-3857b3707687a4d585f1200eec4c34a7a079eae8d303b4ce5b4fce46234ace9fR61-R69).
* As of https://github.com/zed-industries/zed/pull/13276, whenever we
ask a given completion provider for its available models, OpenAI
providers would go and ask the global assistant settings whether the
user had configured an `available_models` setting, and if so, return
that.
* This PR changes it so that instead of eagerly asking the assistant
settings for this info (the new crate must not depend on `assistant`, or
else the dependency cycle would be back), OpenAI completion providers
now store the user-configured settings as part of their struct, and
whenever the settings change, we update the provider.
In theory, this change should not change user-visible behavior...but
since it's the only change in this large PR that's more than just moving
code around, I'm mentioning it here in case there's an unexpected
regression in practice! (cc @amtoaer in case you'd like to try out this
branch and verify that the feature is still working the way you expect.)
Release Notes:
- N/A
---------
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Release Notes:
- Fixed a bug that caused focus to be lost when renames and inline
assists were scrolled offscreen.
---------
Co-authored-by: Nathan <nathan@zed.dev>
This PR adds a streamlined way to consistently construct tab content for
items that only have textual content in the tabs.
The `Item` trait now has a new `tab_content_text` method that can be
used to return the textual content for the tab.
The `tab_content` method now has a default implementation that—unless
overridden—will construct a `Label` out of the text. This default
implementation also takes care of setting the label color based on the
active state of the tab, something that previously had to be repeated in
each `tab_content` implementation.
The majority of our tabs are now using `tab_content_text`.
Release Notes:
- N/A
This PR is an internal refactor in preparation for remote editing. It
restructures the public interface of `Worktree`, reducing the number of
call sites that assume that a worktree is local or remote.
* The Project no longer calls `worktree.as_local_mut().unwrap()` in code
paths related to basic file operations
* Fewer code paths in the app rely on the worktree's `LocalSnapshot`
* Worktree-related RPC message handling is more fully encapsulated by
the `Worktree` type.
to do:
* [x] file manipulation operations
* [x] sending worktree updates when sharing
for later
* opening buffers
* updating open buffers upon worktree changes
Release Notes:
- N/A
When running the tests for linux, I found a lot of benign errors getting
logged. This PR cuts down some of the noise from unnecessary workspace
serialization and SVG renders
Release Notes:
- N/A
This PR restructures the way that tools and attachments add information
about the current project to a conversation with the assistant. Rather
than each tool call or attachment generating a new tool or system
message containing information about the project, they can all
collectively mutate a new type called a `ProjectContext`, which stores
all of the project data that should be sent to the assistant. That data
is then formatted in a single place, and passed to the assistant in one
system message.
This prevents multiple tools/attachments from including redundant
context.
Release Notes:
- N/A
---------
Co-authored-by: Kyle <kylek@zed.dev>
This fixes a tricky intermittent issue I was seeing, where failed to
chunk certain files correctly because of the way we reuse Tree-sitter
`Parser` instances across parses.
I've also accounted for leading comments in chunk boundaries, so that
items are grouped with their leading comments whenever possible when
chunking.
Finally, we've changed the `debug project index` action so that it opens
a simple debug view in a pane, instead of printing paths to the console.
This lets you click into a path and see how it was chunked.
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
This chunking strategy uses the existing `outline` query to chunk files.
We try to find chunk boundaries that are:
* at starts or ends of lines
* nested within as few outline items as possible
Release Notes:
- N/A
Previously, a failure to embed the search query (due to a rate limit
error) would appear the same as if there were no results.
* Avoid repeatedly embedding the search query for each worktree
* Unify tasks for searching all worktree
Release Notes:
- N/A
* [x] Fixed an issue where embeddings would be assigned incorrectly to
files if a subset of embedding batches failed
* [x] Added a command to debug which paths are present in the semantic
index
* [x] Determine why so many paths are often missing from the semantic
index
* we erroring out if an embedding batch contained multiple texts that
were the same, which can happen if a worktree contains multiple copies
of the same text (e.g. a license).
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Nathan <nathan@zed.dev>
Co-authored-by: Kyle <kylek@zed.dev>
Co-authored-by: Kyle Kelley <rgbkrk@gmail.com>
fixes https://github.com/zed-industries/zed/issues/9132
By setting the app id, window managers like `sway` can apply custom
configuration like `for_window [app_id="zed"] floating enable`.
Tested using `wlprop`/`hyprctl activewindow` for wayland, `xprop` for
x11.
Release Notes:
- Zed now sets the window app id / class, which can be used e.g. in
window managers like `sway`/`i3` to define custom rules
Release Notes:
- Allow reusing the window when opening a remote project from the recent
projects picker
- Fixed an issue, which would not let you rejoin a remote project after
disconnecting from it for the first time
---------
Co-authored-by: Conrad <conrad@zed.dev>
Co-authored-by: Remco <djsmits12@gmail.com>
This is a crate only addition of a new version of the AssistantPanel.
We'll be putting this behind a feature flag while we iron out the new
experience.
Release Notes:
- N/A
---------
Co-authored-by: Nathan Sobo <nathan@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Conrad Irwin <conrad@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Antonio Scandurra <antonio@zed.dev>
Co-authored-by: Nate Butler <nate@zed.dev>
Co-authored-by: Nate Butler <iamnbutler@gmail.com>
Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com>
Co-authored-by: Max <max@zed.dev>
This introduces semantic indexing in Zed based on chunking text from
files in the developer's workspace and creating vector embeddings using
an embedding model. As part of this, we've created an embeddings
provider trait that allows us to work with OpenAI, a local Ollama model,
or a Zed hosted embedding.
The semantic index is built by breaking down text for known
(programming) languages into manageable chunks that are smaller than the
max token size. Each chunk is then fed to a language model to create a
high dimensional vector which is then normalized to a unit vector to
allow fast comparison with other vectors with a simple dot product.
Alongside the vector, we store the path of the file and the range within
the document where the vector was sourced from.
Zed will soon grok contextual similarity across different text snippets,
allowing for natural language search beyond keyword matching. This is
being put together both for human-based search as well as providing
results to Large Language Models to allow them to refine how they help
developers.
Remaining todo:
* [x] Change `provider` to `model` within the zed hosted embeddings
database (as its currently a combo of the provider and the model in one
name)
Release Notes:
- N/A
---------
Co-authored-by: Nathan Sobo <nathan@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Conrad Irwin <conrad@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Antonio <antonio@zed.dev>
Resurrected this from some assistant work I did in Spring of 2023.
- [x] Resurrect streaming responses
- [x] Use streaming responses to enable AI via Zed's servers by default
(but preserve API key option for now)
- [x] Simplify protobuf
- [x] Proxy to OpenAI on zed.dev
- [x] Proxy to Gemini on zed.dev
- [x] Improve UX for switching between openAI and google models
- We current disallow cycling when setting a custom model, but we need a
better solution to keep OpenAI models available while testing the google
ones
- [x] Show remaining tokens correctly for Google models
- [x] Remove semantic index
- [x] Delete `ai` crate
- [x] Cloud front so we can ban abuse
- [x] Rate-limiting
- [x] Fix panic when using inline assistant
- [x] Double check the upgraded `AssistantSettings` are
backwards-compatible
- [x] Add hosted LLM interaction behind a `language-models` feature
flag.
Release Notes:
- We are temporarily removing the semantic index in order to redesign it
from scratch.
---------
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Thorsten <thorsten@zed.dev>
Co-authored-by: Max <max@zed.dev>
Closes#5178
Release Notes:
- Added a `file_types` setting that can be used to associate languages
with file names and file extensions. For example, to interpret all `.c`
files as C++, and files called `MyLockFile` as TOML, add the following
to `settings.json`:
```json
{
"file_types": {
"C++": ["c"],
"TOML": ["MyLockFile"]
}
}
```
As with most zed settings, this can be configured on a per-directory
basis by including a local `.zed/settings.json` file in that directory.
---------
Co-authored-by: Marshall <marshall@zed.dev>
This is just a refactor. I noticed that we now have a `project_core`
crate, which mainly contains the `Worktree` type and its private
helpers, plus the project's settings.
In this PR, I've renamed that crate to `worktree` and did some minor
simplification to its module structure. I also extracted a new
`WorktreeSettings` settings type from the `ProjectSettings`, so that the
worktree settings could live in the worktree crate. This way, the crate
is now exclusively about worktree logic.
Release Notes:
- N/A
Partially fixes#4321, since Azure OpenAI API can be converted to OpenAI
API.
Release Notes:
- Added `assistant.openai_api_url` setting to allow OpenAI API URL to be
configured.
---------
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
This PR adds the initial support for loading extensions in Zed.
### Extensions Directory
Extensions are loaded from the extensions directory.
The extensions directory has the following structure:
```
extensions/
installed/
extension-a/
grammars/
languages/
extension-b/
themes/
manifest.json
```
The `manifest.json` file is used internally by Zed to keep track of
which extensions are installed. This file should be maintained
automatically, and shouldn't require any direct interaction with it.
Extensions can provide Tree-sitter grammars, languages, and themes.
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
Reverts zed-industries/zed#7481
This would regress performance because we'd be using the standard
library's hash maps everywhere, so reverting for now.
This should prevent a class of bugs where one queries the wrong type of
global, which results in oddities at runtime.
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
This PR updates the casing of "OpenAI" when used in Rust identifiers to
match the [Rust naming
guidelines](https://rust-lang.github.io/api-guidelines/naming.html):
> In `UpperCamelCase`, acronyms and contractions of compound words count
as one word: use `Uuid` rather than `UUID`, `Usize` rather than `USize`
or `Stdin` rather than `StdIn`.
Release Notes:
- N/A
