This PR removes the unused `ignore_checksum_mismatch` parameter to
`run_database_migrations`.
We were always passing `false`, which meant the behavior didn't need to
be parameterized.
Release Notes:
- N/A
This PR puts the initial infrastructure for the LLM service's database
in place.
The LLM service will be using a separate Postgres database, with its own
set of migrations.
Currently we only connect to the database in development, as we don't
yet have the database setup for the staging/production environments.
Release Notes:
- N/A
This PR updates the rate limits to adapt based on the user's current
plan.
For the free plan rate limits I just took one-tenth of the existing rate
limits (which are now the Pro limits). We can adjust, as needed.
Release Notes:
- N/A
---------
Co-authored-by: Max <max@zed.dev>
This PR adds a new `GET /billing/subscriptions` endpoint to collab for
retrieving the subscriptions to display on the account settings page.
Release Notes:
- N/A
This PR reworks how we process Stripe events for reconciliation
purposes.
The previous approach in #15480 turns out to not be workable, on account
of the Stripe event IDs not being strictly in order. This meant that we
couldn't reliably compare two arbitrary event IDs and determine which
one was more recent.
This new approach leans on the guidance that Stripe provides for
webhooks events:
> Webhook endpoints might occasionally receive the same event more than
once. You can guard against duplicated event receipts by logging the
[event IDs](https://docs.stripe.com/api/events/object#event_object-id)
you’ve processed, and then not processing already-logged events.
>
> https://docs.stripe.com/webhooks#handle-duplicate-events
We now record processed Stripe events in the `processed_stripe_events`
table and use this to filter out events that have already been
processed, so we do not process them again.
When retrieving events from the Stripe events API we now buffer the
unprocessed events so that we can sort them by their `created` timestamp
and process them in (roughly) the order they occurred.
Release Notes:
- N/A
This PR improves our Stripe event handling by keeping track of the last
event we've seen for each record.
The `billing_customers` and `billing_subscriptions` tables both have a
new `last_stripe_event_id` column. When we apply an event to one of
these records, we store the event ID that was applied.
Then, when we are going through events we can ignore any event that has
an ID that came before the `last_stripe_event_id` (based on the
lexicographical ordering of the IDs).
Release Notes:
- N/A
This PR lays the initial groundwork for using the Stripe events API to
reconcile the data in our system with what's in Stripe.
We're using the events API over webhooks so that we don't need to stand
up the associated infrastructure needed to handle webhooks effectively
(namely an asynchronous job queue).
Since we haven't configured the Stripe API keys yet, we won't actually
spawn the reconciliation background task yet, so this is currently a
no-op.
Release Notes:
- N/A
This PR adds a new `billing_customers` table to hold the billing
customers.
Previously we were storing both the `stripe_customer_id` and
`stripe_subscription_id` in the `billable_subscriptions` table. However,
this creates problems when we need to correlate subscription events back
to the subscription record, as we don't know the user that the Stripe
event corresponds to.
By moving the `stripe_customer_id` to a separate table we can create the
Stripe customer earlier in the flow—before we create the Stripe Checkout
session—and associate that customer with a user. This way when we
receive events down the line we can use the Stripe customer ID to
correlate it back to the user.
We're doing some destructive actions to the `billing_subscriptions`
table, but this is fine, as we haven't started using them yet.
Release Notes:
- N/A
This PR adds a new `POST /billing/subscriptions/manage` endpoint that
can be used to manage a billing subscription.
The endpoint accepts a `github_user_id` to identify the user, as well as
an optional `subscription_id` for managing a specific subscription. If
`subscription_id` is not provided, it try and use the active
subscription, if there is only one.
Right now the endpoint only supports cancelling an active subscription.
This is done by passing `"intent": "cancel"` in the request body.
The endpoint will return the URL to a Stripe customer portal session,
which the caller can redirect the user to.
Here's an example of how to call it:
```sh
curl -X POST "http://localhost:8080/billing/subscriptions/manage" \
-H "Authorization: <ADMIN_TOKEN>" \
-H "Content-Type: application/json" \
-d '{"github_user_id": 12345, "intent": "cancel"}'
```
Release Notes:
- N/A
This PR adds a new `POST /billing/subscriptions` endpoint that can be
used to initiate a billing subscription.
The endpoint will use the provided `github_user_id` to look up a user,
generate a Stripe Checkout session, and then return the URL.
The caller would then redirect the user to the URL to initiate the
checkout flow.
Here's an example of how to call it:
```sh
curl -X POST "http://localhost:8080/billing/subscriptions" \
-H "Authorization: <ADMIN_TOKEN>" \
-H "Content-Type: application/json" \
-d '{"github_user_id": 12345}'
```
Release Notes:
- N/A
This PR adds a new `billing_subscriptions` table to the database, as
well as some accompanying models/queries.
In this table we store a minimal amount of data from Stripe:
- The Stripe customer ID
- The Stripe subscription ID
- The status of the Stripe subscription
This should be enough for interactions with the Stripe API (e.g., to
[create a customer portal
session](https://docs.stripe.com/api/customer_portal/sessions/create)),
as well as determine whether a subscription is active (based on the
`status`).
Release Notes:
- N/A
The biggest hurdle turned out to be use of `Arc<Language>` in maps, as
`clippy::mutable_key_type` started triggering on it (due to - I suppose
- internal mutability on `HighlightMap`?). I switched over to using
`LanguageId` as the key type in some of the callsites, as that's what
`Language` uses anyways for it's hash/eq, though I've still had to
suppress the lint outside of language crate.
/cc @maxdeviant , le clippy guru.
Release Notes:
- N/A
Run any Jupyter kernel in Zed on any buffer (editor):
<img width="1074" alt="image"
src="https://github.com/zed-industries/zed/assets/836375/eac8ed69-d02b-4d46-b379-6186d8f59470">
## TODO
### Lifecycle
* [x] Launch kernels on demand
* [x] Wait for kernel to be started
* [x] Request Kernel info on start
* [x] Show in progress indicator
* [ ] Allow picking kernel (it defaults to first matching language name)
* [ ] Menu for interrupting and shutting down the kernel
* [ ] Drop running kernels once editor is dropped
### Media Outputs
* [x] Render text and tracebacks with ANSI color handling
* [x] Render markdown as text
* [x] Render PNG and JPEG images using an explicit height based on
line-height
* ~~Render SVG~~ -- not happening for this PR due to lack of text in SVG
support
* [ ] Process `update_display_data` message and related `display_id`
* [x] Process `page` data from payloads as outputs
* [ ] Render markdown as, well, rendered markdown -- Note: unsure if we
can get line heights here
### Document
* [x] Select code and run
* [x] Run current line
* [x] Clear previous overlapping runs
* [ ] Support running markdown code blocks
* [ ] Action to export session as notebook or output files
* [ ] Action to clear all outputs
* [ ] Delete outputs when lines are deleted
## Other missing features
The following is a list of missing functionality or expectations that
are out of scope for this PR.
### Python Environments
Detecting python environments should probably be done in a separate PR
in tandem with how they're used with LSP. Users likely want to pick an
environment for their project, whether a virtualenv, conda env, pyenv,
poetry backed virtualenv, or the system. Related issues:
* https://github.com/zed-industries/zed/issues/7646
* https://github.com/zed-industries/zed/issues/7808
* https://github.com/zed-industries/zed/issues/7296
### LSP Integration
* Submit `complete_request` messages for completions to interleave
interactive variables with LSP
* LSP for IPython semantics (`%%timeit`, `!ls`, `get_ipython`, etc.)
## Future release notes
- Run code in any editor, whether it's a script or a markdown document
Release Notes:
- N/A
Note:
- We have disabled all tests that rely on Postgres in the Linux CI. We
only really need to test these once, and as macOS is our team's primary
platform, we'll only enable them on macOS for local reproduction.
- We have disabled all tests that rely on the font metrics. We
standardized on Zed Mono in many fonts, but our CoreText Text System and
Cosmic Text System proved to be very different in effect. We should
revisit if we decide to standardize our text system across platforms
(e.g. using Harfbuzz everywhere)
- Extended the condition timeout significantly. Our CI machines are slow
enough that this is causing spurious errors in random tests.
Release Notes:
- N/A
---------
Co-authored-by: Conrad Irwin <conrad.irwin@gmail.com>
#### Lazily loading channels
I've added a new RPC message called `SubscribeToChannels` that the
client now sends when it first renders the channels panel. This causes
the server to load the channels for that client and send updates to that
client as channels are updated. Previously, the server did this upon
connection.
For backwards compatibility, the server will inspect clients' version,
and continue to do this work immediately for old clients.
#### Optimizations
Running collab locally, I realized that upon connecting, we were running
two concurrent transactions that *both* queried the `channel_members`
table: one for loading your channels, and one for loading your channel
invites. I've combined these into one query. In addition, we now use a
join to load channels + members, as opposed to two separate queries.
Even though `where id in` is efficient, it adds an extra round trip to
the database, keeping the transaction open for slightly longer.
Release Notes:
- N/A
Rust recently got the ability to check for typos or errors in `cfg`
attributes: https://blog.rust-lang.org/2024/05/06/check-cfg.html
This PR fixes the new warnings.
- gpui can be run with `RUSTFLAGS="--cfg gles"`, make this explicit in
`[workspace.lints.rust]`
- `cfg!(any(test, sqlite))` was just a bug, it should be
`feature(sqlite)`
- the `languages` crate had a `#[cfg(any(test, feature =
"test-support"))]` function without ever declaring the `test-support`
feature
- the `MarkdownTag` enum had a `cfg_attr` for serde without actually
having serde support
Now the only warnings when building are unused fields
`InlayHover.excerpt`, `SavedConversationMetadata.path` ,
`UserTestPlan.allow_client_reconnection` and `SyntaxMapCapture.depth`.
Release Notes:
- N/A
Co-Authored-By: Mikayla <mikayla@zed.dev>
In a fit of ill-advisedness I called these things remote projects;
forgetting that remote project is also what we call collaboratively
shared projects.
Release Notes:
- N/A
---------
Co-authored-by: Mikayla <mikayla@zed.dev>
Co-authored-by: Bennet <bennetbo@gmx.de>
This introduces semantic indexing in Zed based on chunking text from
files in the developer's workspace and creating vector embeddings using
an embedding model. As part of this, we've created an embeddings
provider trait that allows us to work with OpenAI, a local Ollama model,
or a Zed hosted embedding.
The semantic index is built by breaking down text for known
(programming) languages into manageable chunks that are smaller than the
max token size. Each chunk is then fed to a language model to create a
high dimensional vector which is then normalized to a unit vector to
allow fast comparison with other vectors with a simple dot product.
Alongside the vector, we store the path of the file and the range within
the document where the vector was sourced from.
Zed will soon grok contextual similarity across different text snippets,
allowing for natural language search beyond keyword matching. This is
being put together both for human-based search as well as providing
results to Large Language Models to allow them to refine how they help
developers.
Remaining todo:
* [x] Change `provider` to `model` within the zed hosted embeddings
database (as its currently a combo of the provider and the model in one
name)
Release Notes:
- N/A
---------
Co-authored-by: Nathan Sobo <nathan@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Conrad Irwin <conrad@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Antonio <antonio@zed.dev>
@ConradIrwin This is a followup for #9035 as agreed.
Release Notes:
- Fixed mention notifications are updated when channel message is
updated. And mention notifications are removed when message is removed.
---------
Co-authored-by: Bennet Bo Fenner <53836821+bennetbo@users.noreply.github.com>
This PR modifies the extension installation and update process to
respect version constraints (schema version and Wasm API version) to
ensure only compatible versions of extensions are able to be installed.
To achieve this there is a new `GET /extensions/updates` endpoint that
will return extension versions based on the provided constraints.
Release Notes:
- N/A
---------
Co-authored-by: Max <max@zed.dev>
This PR extends the extension API with some additional features to
support auto-updating extensions:
- The `GET /extensions` endpoint now accepts an optional `ids` parameter
that can be used to filter the results down to just the extensions with
the specified IDs.
- This should be a comma-delimited list of extension IDs (e.g.,
`wgsl,gleam,tokyo-night`).
- A new `GET /extensions/:extension_id` endpoint that returns all of the
extension versions for a particular extension.
Extracted from #9890, as these changes can be landed and deployed
independently.
Release Notes:
- N/A
Co-authored-by: Max <max@zed.dev>
* Store extensions versions' wasm API version in the database
* Share a common struct for extension API responses between collab and
client
* Add wasm API version and schema version to extension API responses
Release Notes:
- N/A
Co-authored-by: Marshall <marshall@zed.dev>
This PR provides some of the plumbing needed for a "remote" zed
instance.
The way this will work is:
* From zed on your laptop you'll be able to manage a set of dev servers,
each of which is identified by a token.
* You'll run `zed --dev-server-token XXXX` to boot a remotable dev
server.
* From the zed on your laptop you'll be able to open directories and
work on the projects on the remote server (exactly like collaboration
works today).
For now all this PR does is provide the ability for a zed instance to
sign in
using a "dev server token". The next steps will be:
* Adding support to the collaboration protocol to instruct a dev server
to "open" a directory and share it into a channel.
* Adding UI to manage these servers and tokens (manually for now)
Related #5347
Release Notes:
- N/A
---------
Co-authored-by: Nathan <nathan@zed.dev>
Co-authored-by: Antonio <antonio@zed.dev>
Resurrected this from some assistant work I did in Spring of 2023.
- [x] Resurrect streaming responses
- [x] Use streaming responses to enable AI via Zed's servers by default
(but preserve API key option for now)
- [x] Simplify protobuf
- [x] Proxy to OpenAI on zed.dev
- [x] Proxy to Gemini on zed.dev
- [x] Improve UX for switching between openAI and google models
- We current disallow cycling when setting a custom model, but we need a
better solution to keep OpenAI models available while testing the google
ones
- [x] Show remaining tokens correctly for Google models
- [x] Remove semantic index
- [x] Delete `ai` crate
- [x] Cloud front so we can ban abuse
- [x] Rate-limiting
- [x] Fix panic when using inline assistant
- [x] Double check the upgraded `AssistantSettings` are
backwards-compatible
- [x] Add hosted LLM interaction behind a `language-models` feature
flag.
Release Notes:
- We are temporarily removing the semantic index in order to redesign it
from scratch.
---------
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Thorsten <thorsten@zed.dev>
Co-authored-by: Max <max@zed.dev>
This should significantly reduce database load on redeploy.
Co-Authored-By: Max <max@zed.dev>
Co-Authored-By: Nathan <nathan@zed.dev>
Release Notes:
- Reduced likelihood of being disconnected during deploys
Co-authored-by: Max <max@zed.dev>
Co-authored-by: Nathan <nathan@zed.dev>
This reduces the server time to compute the hash from 40ms to 5µs,
which should remove this as a noticable chunk of CPU time in production.
(An attacker who has access to our database will now need only 10^54
years of CPU time instead of 10^58 to brute force a token).
Release Notes:
- Improved sign in latency by 40ms.
Marshall Bowers
Max BrunsfeldandMarshall
Richard Feldman
Conrad IrwinandMax
Piotr Osiewicz
Kyle Kelley
Mikayla MakiandConrad Irwin
Jakob Hellermann
Kirill BulatovandConrad Irwin
Bennet Bo Fenner
Remco SmitsandBennet Bo Fenner
