This PR makes the account age-related fields required in
`LlmTokenClaims`.
We've also removed the account age check from the LLM token issuance
endpoint, instead having it solely be enforced in the `POST /completion`
endpoint.
This change will be safe to deploy at ~8:01PM EDT.
Release Notes:
- N/A
This PR defers the account age check to the `POST /completion` endpoint
instead of doing it when an LLM token is generated.
This will allow us to lift the account age restriction for using Edit
Prediction.
Note: We're still temporarily performing the account age check when
issuing the LLM token until this change is deployed and the LLM tokens
have had a chance to cycle.
Release Notes:
- N/A
This PR cleans up the LLM token creation a bit.
We now pass in the entire list of feature flags to the
`LlmTokenClaims::create` method to prevent having a bunch of confusable
`bool` parameters.
Release Notes:
- N/A
This reverts commit 9ef0501853 due to a
panic.
```
{
"thread": "main",
"payload": "9 is not a valid char boundary in path \"crates/…/LiveKitBridge/\"",
"location_data": {
"file": "crates/file_finder/src/file_finder.rs",
"line": 646
}
}
```
Release Notes:
- N/A
This PR makes progress on #7711 by identifying any common prefix of the
paths in the file finder's search results, and replacing the "interior"
of that prefix---every path segment but the first and last---with `...`,
when a heuristic indicates that the longest path would otherwise
overflow the modal.
The elision is not applied to any segment that contains a match for the
search query.
There may be more work to do on #7711 in the case of long result paths
that do not share a significant common prefix.
Release Notes:
- Improved display of long paths in the file finder modal
Co-authored-by: Max <max@zed.dev>
This PR removes the `POST /predict_edits` endpoint from the LLM service,
as it has been superseded by the corresponding endpoint running in
Cloudflare Workers.
All traffic is already being routed to the Cloudflare Workers via the
Workers route, so nothing is hitting this endpoint running in the LLM
service anymore.
You can see the drop off in requests to this endpoint on this graph when
the Workers route was added:
<img width="472" alt="Screenshot 2025-01-30 at 9 18 04 PM"
src="https://github.com/user-attachments/assets/fa60f7c8-2737-4329-88a3-17093bdb5a29"
/>
We also don't use the `fireworks` crate anymore in this repo, so it has
been removed.
Release Notes:
- N/A
This PR adjusts the billing logic to not write any records to
`billing_events` if:
- The user is staff, as we don't want to bill staff members
- Billing is disabled (we currently enable billing based on the presence
of the Stripe API key)
Release Notes:
- N/A
This PR adds usage-based billing for LLM interactions in the Assistant.
Release Notes:
- N/A
---------
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Richard <richard@zed.dev>
Co-authored-by: Richard Feldman <oss@rtfeldman.com>
This PR makes the `has_llm_subscription` and
`max_monthly_spend_in_cents` fields in the `LlmTokenClaims` required.
This change will be safe to deploy in ~45 minutes.
Release Notes:
- N/A
This PR adds a new `Cents` type that can be used to represent a monetary
value in cents.
This cuts down on the primitive obsession we were using when dealing
with money in the billing code.
Release Notes:
- N/A
This PR makes the `github_user_login` field required in the
`LlmTokenClaims`.
We previously added this in
https://github.com/zed-industries/zed/pull/16316 and made it optional
for backwards-compatibility.
It's been more than long enough for all of the previous LLM tokens to
have expired, so we can now make the field required.
Release Notes:
- N/A
This PR reworks our existing billing code in preparation for charging
based on LLM usage.
We aren't yet exercising the new billing-related code outside of
development.
There are some noteworthy changes for our existing LLM usage tracking:
- A new `monthly_usages` table has been added for tracking usage
per-user, per-model, per-month
- The per-month usage measures have been removed, in favor of the
`monthly_usages` table
- All of the per-month metrics in the Clickhouse rows have been changed
from a rolling 30-day window to a calendar month
Release Notes:
- N/A
---------
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Richard <richard@zed.dev>
Co-authored-by: Max <max@zed.dev>
This PR extends the LLM usage tracking to support tracking usage for
cache writes and reads for Anthropic models.
Release Notes:
- N/A
---------
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Antonio <antonio@zed.dev>
Add `/auto` behind a feature flag that's disabled for now, even for
staff.
We've decided on a different design for context inference, but there are
parts of /auto that will be useful for that, so we want them in the code
base even if they're unused for now.
Release Notes:
- N/A
---------
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
This PR adds a `GET /models` endpoint to the LLM service.
This endpoint returns the models that the authenticated user has access
to.
This is the first step towards populating the models for the hosted
service from the server.
Release Notes:
- N/A
- Cloudflare provides ISO-3166-1 country code for protectorates. Expand our allowlist to include the territories of countries on the allowlist (US, UK, France, Australia, New Zealand).
- Also include the country_code in the error message when we block.
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
This PR fixes an issue where active user counts were being computed
across _all_ measures instead of the per-minute measures.
We now compute them using the tokens per minute measure, as we're
concerned with usage in recent minutes.
Release Notes:
- N/A
This PR fixes an issue where the active user count spanned individual
models.
We now track the active user counts on a per-model basis.
Release Notes:
- N/A
This PR fixes the writing of LLM rate limit events to Clickhouse.
We had a table in the table name: `llm_rate_limits` instead of
`llm_rate_limit_events`.
I also extracted a helper function to write to Clickhouse so we can use
it anywhere we need to.
Release Notes:
- N/A
This PR reworks how we do checks for model names in the LLM service.
We now normalize the model names using the models defined in the
database.
Release Notes:
- N/A
This PR updates the LLM service to include the GitHub login on its
spans.
We need to pass this information through on the LLM token, so it will
temporarily be `None` until this change is deployed and new tokens have
been issued.
Release Notes:
- N/A
- db deadlock in GetLlmToken for non-staff users
- typo in allowed model name for non-staff users
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Joseph <joseph@zed.dev>
This PR adds the ability to revoke access tokens for the LLM service.
There is a new `revoked_access_tokens` table that contains the
identifiers (`jti`) of revoked access tokens.
To revoke an access token, insert a record into this table:
```sql
insert into revoked_access_tokens (jti) values ('1e887b9e-37f5-49e8-8feb-3274e5a86b67');
```
We now attach the `jti` as `authn.jti` to the tracing spans so that we
can associate an access token with a given request to the LLM service.
Release Notes:
- N/A
This PR makes it so Zed staff can use a separate Anthropic API key for
the LLM service.
We also added an `is_staff` column to the `usages` table so that we can
exclude staff usage from the "active users" metrics that influence the
rate limits.
Release Notes:
- N/A
---------
Co-authored-by: Max <max@zed.dev>
Marshall Bowers
Conrad Irwin
Cole Miller
Agus ZubiagaandThorsten Ball
Antonio ScandurraandThorsten
Marshall Bowers
Richard FeldmanandMarshall
Piotr Osiewicz
Peter TrippandMarshall Bowers
Max BrunsfeldandMarshall
