Commit Graph
26 Commits
Author SHA1 Message Date
Marshall Bowers c709b66f35 collab: Don't record billing events if billing is not enabled (#19102)
This PR adjusts the billing logic to not write any records to
`billing_events` if:

- The user is staff, as we don't want to bill staff members
- Billing is disabled (we currently enable billing based on the presence
of the Stripe API key)

Release Notes:

- N/A
2024-10-11 17:54:10 -04:00
Richard FeldmanandMarshall 12c9f0f723 Test some billing events logic (#19094)
Release Notes:

- N/A

Co-authored-by: Marshall <marshall@zed.dev>
2024-10-11 16:21:21 -04:00
Marshall Bowers f33b8abc72 collab: Sort LLM database ID types (#19083)
This PR sorts the order of the LLM database ID type declarations.

Release Notes:

- N/A
2024-10-11 13:57:48 -04:00
22ea7cef7a collab: Add usage-based billing for LLM interactions (#19081)
This PR adds usage-based billing for LLM interactions in the Assistant.

Release Notes:

- N/A

---------

Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Richard <richard@zed.dev>
Co-authored-by: Richard Feldman <oss@rtfeldman.com>
2024-10-11 13:36:54 -04:00
Marshall Bowers 817a41c4dc collab: Add a Cents type (#18935)
This PR adds a new `Cents` type that can be used to represent a monetary
value in cents.

This cuts down on the primitive obsession we were using when dealing
with money in the billing code.

Release Notes:

- N/A
2024-10-09 14:22:32 -04:00
f861479890 collab: Update billing code for LLM usage billing (#18879)
This PR reworks our existing billing code in preparation for charging
based on LLM usage.

We aren't yet exercising the new billing-related code outside of
development.

There are some noteworthy changes for our existing LLM usage tracking:

- A new `monthly_usages` table has been added for tracking usage
per-user, per-model, per-month
- The per-month usage measures have been removed, in favor of the
`monthly_usages` table
- All of the per-month metrics in the Clickhouse rows have been changed
from a rolling 30-day window to a calendar month

Release Notes:

- N/A

---------

Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Richard <richard@zed.dev>
Co-authored-by: Max <max@zed.dev>
2024-10-08 18:29:38 -04:00
Marshall Bowers 3f2de172ae collab: Set cached token values when initially creating lifetime usage records (#18865)
This PR fixes an issue where we weren't setting the cached token fields
when initially creating a lifetime usage record.

Release Notes:

- N/A
2024-10-08 13:16:17 -04:00
d55f025906 collab: Track cache writes/reads in LLM usage (#18834)
This PR extends the LLM usage tracking to support tracking usage for
cache writes and reads for Anthropic models.

Release Notes:

- N/A

---------

Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Antonio <antonio@zed.dev>
2024-10-07 17:32:49 -04:00
Richard Feldman caaa9a00a9 Remove Qwen2 model (#18444)
Removed deprecated Qwen2 7B Instruct model from zed.dev provider (staff
only).

Release Notes:

- N/A
2024-09-27 13:30:25 -04:00
91ffa02e2c /auto (#16696)
Add `/auto` behind a feature flag that's disabled for now, even for
staff.

We've decided on a different design for context inference, but there are
parts of /auto that will be useful for that, so we want them in the code
base even if they're unused for now.

Release Notes:

- N/A

---------

Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
2024-09-13 13:17:49 -04:00
Piotr Osiewicz e6c1c51b37 chore: Fix several style lints (#17488)
It's not comprehensive enough to start linting on `style` group, but
hey, it's a start.

Release Notes:

- N/A
2024-09-06 11:58:39 +02:00
Marshall Bowers 93a7682659 collab: Count active users based on the tokens per minute measure (#16911)
This PR fixes an issue where active user counts were being computed
across _all_ measures instead of the per-minute measures.

We now compute them using the tokens per minute measure, as we're
concerned with usage in recent minutes.

Release Notes:

- N/A
2024-08-26 15:04:55 -04:00
Marshall Bowers 0229d3ccac collab: Track active user counts independently for each model (#16624)
This PR fixes an issue where the active user count spanned individual
models.

We now track the active user counts on a per-model basis.

Release Notes:

- N/A
2024-08-21 17:19:47 -04:00
Max BrunsfeldandMarshall b5bd8a5c5d Add logic for closed beta LLM models (#16482)
Release Notes:

- N/A

---------

Co-authored-by: Marshall <marshall@zed.dev>
2024-08-19 11:09:52 -07:00
Max BrunsfeldandMarshall 1b1070e0f7 Add tracing needed for LLM rate limit dashboards (#16388)
Release Notes:

- N/A

---------

Co-authored-by: Marshall <marshall@zed.dev>
2024-08-16 17:52:31 -04:00
Marshall Bowers b4c22cc861 collab: Add ability to revoke LLM service access tokens (#16143)
This PR adds the ability to revoke access tokens for the LLM service.

There is a new `revoked_access_tokens` table that contains the
identifiers (`jti`) of revoked access tokens.

To revoke an access token, insert a record into this table:

```sql
insert into revoked_access_tokens (jti) values ('1e887b9e-37f5-49e8-8feb-3274e5a86b67');
```

We now attach the `jti` as `authn.jti` to the tracing spans so that we
can associate an access token with a given request to the LLM service.

Release Notes:

- N/A
2024-08-12 21:47:05 -04:00
Max BrunsfeldandMarshall dbcd06642c Track lifetime spending for each user and model (#16137)
Release Notes:

- N/A

Co-authored-by: Marshall <marshall@zed.dev>
2024-08-12 20:15:26 -04:00
Marshall BowersandMax f3ec8d425f collab: Use a separate Anthropic API key for Zed staff (#16128)
This PR makes it so Zed staff can use a separate Anthropic API key for
the LLM service.

We also added an `is_staff` column to the `usages` table so that we can
exclude staff usage from the "active users" metrics that influence the
rate limits.

Release Notes:

- N/A

---------

Co-authored-by: Max <max@zed.dev>
2024-08-12 15:20:34 -04:00
Max BrunsfeldandMarshall 33e120d964 Capture telemetry data on per-user monthly LLM spending (#16050)
Release Notes:

- N/A

---------

Co-authored-by: Marshall <marshall@zed.dev>
2024-08-09 16:38:37 -07:00
Max BrunsfeldandMarshall 8688b2ad19 Add telemetry for LLM usage (#16049)
Release Notes:

- N/A

Co-authored-by: Marshall <marshall@zed.dev>
2024-08-09 18:15:57 -04:00
Max BrunsfeldandMarshall 423c7b999a Larger rate limit integers (#16047)
Tokens per day may exceed the range of Postgres's 32-bit `integer` data
type.

Release Notes:

- N/A

Co-authored-by: Marshall <marshall@zed.dev>
2024-08-09 14:07:49 -07:00
Max BrunsfeldandMarshall 240b7c641c Fix llm queries (#16006)
Release Notes:

- N/A

---------

Co-authored-by: Marshall <marshall@zed.dev>
2024-08-08 17:21:38 -07:00
06625bfe94 Apply rate limits in LLM service (#15997)
Release Notes:

- N/A

---------

Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
2024-08-08 15:46:33 -07:00
Marshall Bowers a54e16b7ea collab: Add usages table to LLM database (#15884)
This PR adds a `usages` table to the LLM database.

We'll use this to track usage for rate-limiting purposes.

Release Notes:

- N/A
2024-08-06 18:40:10 -04:00
Marshall Bowers b19f85f9b5 collab: Remove unused parameter to run_database_migrations (#15883)
This PR removes the unused `ignore_checksum_mismatch` parameter to
`run_database_migrations`.

We were always passing `false`, which meant the behavior didn't need to
be parameterized.

Release Notes:

- N/A
2024-08-06 17:31:52 -04:00
Marshall Bowers 7f6d0919c9 collab: Setup database for LLM service (#15882)
This PR puts the initial infrastructure for the LLM service's database
in place.

The LLM service will be using a separate Postgres database, with its own
set of migrations.

Currently we only connect to the database in development, as we don't
yet have the database setup for the staging/production environments.

Release Notes:

- N/A
2024-08-06 17:18:08 -04:00