Commit Graph
12 Commits
Author SHA1 Message Date
Marshall BowersandMax f3ec8d425f collab: Use a separate Anthropic API key for Zed staff (#16128)
This PR makes it so Zed staff can use a separate Anthropic API key for
the LLM service.

We also added an `is_staff` column to the `usages` table so that we can
exclude staff usage from the "active users" metrics that influence the
rate limits.

Release Notes:

- N/A

---------

Co-authored-by: Max <max@zed.dev>
2024-08-12 15:20:34 -04:00
Max BrunsfeldandMarshall 33e120d964 Capture telemetry data on per-user monthly LLM spending (#16050)
Release Notes:

- N/A

---------

Co-authored-by: Marshall <marshall@zed.dev>
2024-08-09 16:38:37 -07:00
Max BrunsfeldandMarshall 8688b2ad19 Add telemetry for LLM usage (#16049)
Release Notes:

- N/A

Co-authored-by: Marshall <marshall@zed.dev>
2024-08-09 18:15:57 -04:00
Max BrunsfeldandMarshall 423c7b999a Larger rate limit integers (#16047)
Tokens per day may exceed the range of Postgres's 32-bit `integer` data
type.

Release Notes:

- N/A

Co-authored-by: Marshall <marshall@zed.dev>
2024-08-09 14:07:49 -07:00
Max BrunsfeldandMarshall 240b7c641c Fix llm queries (#16006)
Release Notes:

- N/A

---------

Co-authored-by: Marshall <marshall@zed.dev>
2024-08-08 17:21:38 -07:00
06625bfe94 Apply rate limits in LLM service (#15997)
Release Notes:

- N/A

---------

Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
2024-08-08 15:46:33 -07:00
Bennet Bo FennerandThorsten 3a52d6cc52 assistant: Limit model access for Zed AI users to Claude-3.5-sonnet (#15904)
This prevents users from accessing other models, such as OpenAI's GPT-4
or Google's Gemini-Pro.
Staff members can still access all models.

Co-authored-by: Thorsten <thorsten@zed.dev>

Release Notes:

- N/A

---------

Co-authored-by: Thorsten <thorsten@zed.dev>
2024-08-07 16:26:56 +02:00
Marshall Bowers a54e16b7ea collab: Add usages table to LLM database (#15884)
This PR adds a `usages` table to the LLM database.

We'll use this to track usage for rate-limiting purposes.

Release Notes:

- N/A
2024-08-06 18:40:10 -04:00
Marshall Bowers b19f85f9b5 collab: Remove unused parameter to run_database_migrations (#15883)
This PR removes the unused `ignore_checksum_mismatch` parameter to
`run_database_migrations`.

We were always passing `false`, which meant the behavior didn't need to
be parameterized.

Release Notes:

- N/A
2024-08-06 17:31:52 -04:00
Marshall Bowers 7f6d0919c9 collab: Setup database for LLM service (#15882)
This PR puts the initial infrastructure for the LLM service's database
in place.

The LLM service will be using a separate Postgres database, with its own
set of migrations.

Currently we only connect to the database in development, as we don't
yet have the database setup for the staging/production environments.

Release Notes:

- N/A
2024-08-06 17:18:08 -04:00
Marshall Bowers cf5f4dddf5 Authorize access to language model providers based on country (#15859)
This PR updates the LLM service to authorize access to language model
providers based on the requester's country.

We detect the country using Cloudflare's
[`CF-IPCountry`](https://developers.cloudflare.com/fundamentals/reference/http-request-headers/#cf-ipcountry)
header.

The country code is then checked against the list of supported countries
for the given LLM provider. Countries that are not supported will
receive an `HTTP 451: Unavailable For Legal Reasons` response.

Release Notes:

- N/A
2024-08-06 11:49:04 -04:00
8e9c2b1125 Introduce a separate backend service for LLM calls (#15831)
This PR introduces a separate backend service for making LLM calls.

It exposes an HTTP interface that can be called by Zed clients. To call
these endpoints, the client must provide a `Bearer` token. These tokens
are issued/refreshed by the collab service over RPC.

We're adding this in a backwards-compatible way. Right now the access
tokens can only be minted for Zed staff, and calling this separate LLM
service is behind the `llm-service` feature flag (which is not
automatically enabled for Zed staff).

Release Notes:

- N/A

---------

Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
2024-08-05 20:26:21 -04:00