Oleksiy Syvokon 99d7b2fa1d zeta2: Compute diff-aware chrF metric (#43485) ...

Zeta evals now include a character n-gram metric adapted for multi-edit diffs (“delta chrF”). It works as follows:

1. Reconstruct the original, golden (expected), and actual texts from unified diffs.
   - "original": the text before any edits
   - "golden": the text after applying the expected edits
   - "actual": the text after applying the actual edits

2. Compute n-gram count deltas between original→golden and original→actual.
   - n-grams are computed as in chrF (max n=6, whitespace ignored).

3. Compare these deltas to assess how well the actual edits match the expected edits.
   - As in standard chrF, classify n-grams as true positives, false positives, and false negatives, and report the F-beta score with beta=2.

Release Notes:

- N/A

2025-11-27 11:10:35 +02:00

..

acp_thread

multi_buffer: Remove redundant buffer id field (#43459)

2025-11-25 17:13:16 +01:00

acp_tools

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

action_log

multi_buffer: Remove redundant buffer id field (#43459)

2025-11-25 17:13:16 +01:00

activity_indicator

releases: Add build number to Nightly builds (#42990)

2025-11-24 13:34:04 +01:00

agent

agent_ui: Add support for deleting thread history (#43370)

2025-11-26 15:55:12 +00:00

agent_servers

acp: Support specifying settings for extensions (#43177)

2025-11-20 17:12:00 +00:00

agent_settings

agent: Add ability to set a default_model per profile (#39220)

2025-11-10 11:11:24 -03:00

agent_ui

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

ai_onboarding

ai onboarding: Add dismiss button to the sign in banner (#40660)

2025-10-20 08:35:28 -03:00

anthropic

Add support for Opus 4.5 (#43425)

2025-11-24 20:01:43 +00:00

askpass

askpass: Quote askpass script in askpass helper command (#43542)

2025-11-26 12:17:57 +00:00

assets

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

assistant_slash_command

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

assistant_slash_commands

Automate settings registration (#42238)

2025-11-07 22:27:14 +00:00

assistant_text_thread

Fix language server renaming when parent directory does not exist (#43499)

2025-11-26 19:34:03 +00:00

audio

Automate settings registration (#42238)

2025-11-07 22:27:14 +00:00

auto_update

remote: Do not include prerelease and build meta in asset queries (#43611)

2025-11-26 23:55:11 +00:00

auto_update_helper

auto_update: Ignore dir removal errors on windows (#42435)

2025-11-11 12:55:19 +00:00

auto_update_ui

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

aws_http_client

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

bedrock

Add support for Opus 4.5 (#43425)

2025-11-24 20:01:43 +00:00

breadcrumbs

svg_preview: Remove unnecessary dependency on editor (#43147)

2025-11-20 12:18:50 +01:00

buffer_diff

multi_buffer: Remove redundant buffer id field (#43459)

2025-11-25 17:13:16 +01:00

call

Bump Rust to 1.91.1 (#42561)

2025-11-12 20:27:04 +00:00

channel

releases: Add build number to Nightly builds (#42990)

2025-11-24 13:34:04 +01:00

cli

Fix zed cli in NixOS WSL instances (#43433)

2025-11-24 15:46:13 -06:00

client

releases: Add build number to Nightly builds (#42990)

2025-11-24 13:34:04 +01:00

clock

text: Improve panic messages with more information (#42072)

2025-11-06 09:16:45 +00:00

cloud_api_client

Use cloud for auto-update (#42246)

2025-11-10 23:00:55 -07:00

cloud_api_types

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

cloud_llm_client

edit prediction: Request trigger (#43588)

2025-11-26 20:34:29 +00:00

cloud_zeta2_prompt

Combine zeta and zeta2 edit prediction providers (#43284)

2025-11-24 22:17:48 -08:00

codestral

zeta2: Predict at next diagnostic location (#43257)

2025-11-21 16:39:08 +00:00

collab

collab: Add zed-zippy[bot] to the GET /contributor endpoint (#43568)

2025-11-26 16:53:05 +00:00

collab_ui

ui: Remove CheckboxWithLabel and improve Switch and Checkbox (#43343)

2025-11-22 22:26:26 -03:00

collections

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

command_palette

Automate settings registration (#42238)

2025-11-07 22:27:14 +00:00

command_palette_hooks

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

component

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

context_server

Add HTTP transport support for MCP servers (#39021)

2025-11-18 16:39:08 +00:00

copilot

Add Gemini 3 support to Copilot (#43096)

2025-11-25 12:15:55 -05:00

crashes

crashes: Print panic message to logs (#43159)

2025-11-21 17:31:57 +01:00

credentials_provider

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

dap

util: Fix zip extraction (#42714)

2025-11-19 12:34:41 +01:00

dap_adapters

Fix connect.host setting being ignored by debugpy (#43190)

2025-11-21 09:20:15 +00:00

db

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

debug_adapter_extension

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

debugger_tools

Revert "Keep selection in SwitchToHelixNormalMode (#41583)" (#42892)

2025-11-17 11:01:34 -07:00

debugger_ui

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

deepseek

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

denoise

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

diagnostics

multi_buffer: Remove redundant buffer id field (#43459)

2025-11-25 17:13:16 +01:00

docs_preprocessor

docs: Ensure macOS and Linux keybindings are escaped in HTML (#39802)

2025-10-23 14:13:35 -04:00

edit_prediction

zeta2: Predict at next diagnostic location (#43257)

2025-11-21 16:39:08 +00:00

edit_prediction_button

Combine zeta and zeta2 edit prediction providers (#43284)

2025-11-24 22:17:48 -08:00

edit_prediction_context

Automate settings registration (#42238)

2025-11-07 22:27:14 +00:00

editor

Account for greedy tree-sitter bracket matches (#43607)

2025-11-26 23:22:13 +00:00

eval

Fix Gemini 3 on OpenRouter (#43416)

2025-11-24 13:24:26 -05:00

explorer_command_injector

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

extension

releases: Add build number to Nightly builds (#42990)

2025-11-24 13:34:04 +01:00

extension_api

Windows: Fix issues with paths in extensions (#37811)

2025-09-11 13:56:06 -07:00

extension_cli

Fix categorization of agent server extensions (#41689)

2025-10-31 21:18:22 +00:00

extension_host

mcp: Fix source property showing up as undefined in settings (#43417)

2025-11-25 16:03:21 +00:00

extensions_ui

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

feature_flags

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

feedback

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

file_finder

Fix git features not working when a Windows host collaborates with a unix guest (#43515)

2025-11-26 16:56:34 +00:00

file_icons

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

fs

Fix language server renaming when parent directory does not exist (#43499)

2025-11-26 19:34:03 +00:00

fs_benchmarks

fs: Reintroduce benchmarks crate (#40689)

2025-10-20 10:30:06 +00:00

fsevent

Disable fsevents tests (#43218)

2025-11-20 22:17:50 -07:00

fuzzy

Fix git features not working when a Windows host collaborates with a unix guest (#43515)

2025-11-26 16:56:34 +00:00

git

git: Use correct file mode when staging (#41900)

2025-11-26 09:01:20 +00:00

git_hosting_providers

git: Fix support for self-hosted Bitbucket (#42002)

2025-11-09 21:37:22 -05:00

git_ui

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

go_to_line

multi_buffer: Typed MultiBufferOffset (#42707)

2025-11-19 22:00:58 +00:00

google_ai

Update google_ai.rs (#43034)

2025-11-19 05:41:24 +00:00

gpui

Make key repeat rate on Wayland more precise (2) (#43589)

2025-11-26 22:16:50 +01:00

gpui_macros

Reduce macro burden for rust-analyzer (#42871)

2025-11-17 12:31:00 +00:00

gpui_tokio

workspace: Handle non-cloneable items better (#41215)

2025-10-26 13:24:26 +00:00

html_to_markdown

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

http_client

http_client: Support GITHUB_TOKEN env to auth GitHub requests (#42623)

2025-11-24 12:51:45 +02:00

http_client_tls

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

icons

Add experimental Sweep edit prediction provider (#42927)

2025-11-18 01:00:26 +00:00

image_viewer

Automate settings registration (#42238)

2025-11-07 22:27:14 +00:00

inspector_ui

Add support for multi-word snippet prefixes (#42398)

2025-11-11 16:34:25 -05:00

install_cli

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

journal

multi_buffer: Typed MultiBufferOffset (#42707)

2025-11-19 22:00:58 +00:00

json_schema_store

Replace deprecated json.schemastore.org with www.schemastore.org (#42336)

2025-11-11 15:43:25 +02:00

keymap_editor

chore: Fix build graph - again (#42999)

2025-11-18 19:20:34 +01:00

language

zeta2: Compute diff-aware chrF metric (#43485)

2025-11-27 11:10:35 +02:00

language_extension

language: Move language server update to background when it takes too long (#43164)

2025-11-21 14:19:55 +05:30

language_model

Fix Gemini 3 on OpenRouter (#43416)

2025-11-24 13:24:26 -05:00

language_models

Add GPT 5.1 to Zed BYOK (#43492)

2025-11-25 14:17:27 -08:00

language_onboarding

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

language_selector

Fix keybind hints flickering in certain scenarios (#40927)

2025-10-22 19:52:38 +00:00

language_tools

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

languages

languages: Recognize .clangd as YAML (#43557)

2025-11-26 15:55:23 +01:00

line_ending_selector

Fix keybind hints flickering in certain scenarios (#40927)

2025-10-22 19:52:38 +00:00

livekit_api

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

livekit_client

gpui: Add to support check state to MenuItem (#39876)

2025-11-07 09:42:55 +00:00

lmstudio

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

lsp

Redact environment variables in server info view (#43436)

2025-11-24 22:05:16 +00:00

markdown

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

markdown_preview

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

media

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

menu

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

migrator

mcp: Fix source property showing up as undefined in settings (#43417)

2025-11-25 16:03:21 +00:00

miniprofiler_ui

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

mistral

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

multi_buffer

multi_buffer: Remove redundant buffer id field (#43459)

2025-11-25 17:13:16 +01:00

nc

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

net

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

node_runtime

Passthrough env to npm subcommands when using the system node runtime (#43102)

2025-11-24 11:08:45 -08:00

notifications

sum_tree: Implement recursive Sumtree::find, use it over Cursor::seek if possible (#40700)

2025-10-20 17:20:09 +00:00

ollama

ollama: Fix tool calling (#42275)

2025-11-11 16:10:47 +01:00

onboarding

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

open_ai

Add GPT 5.1 to Zed BYOK (#43492)

2025-11-25 14:17:27 -08:00

open_router

Fix Gemini 3 on OpenRouter (#43416)

2025-11-24 13:24:26 -05:00

outline

multi_buffer: Typed MultiBufferOffset (#42707)

2025-11-19 22:00:58 +00:00

outline_panel

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

panel

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

paths

Miniprofiler (#42385)

2025-11-12 15:31:20 +01:00

picker

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

prettier

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

project

Fix language server renaming when parent directory does not exist (#43499)

2025-11-26 19:34:03 +00:00

project_benchmarks

search: New recent old search implementation (#40835)

2025-11-13 14:56:40 +01:00

project_panel

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

project_symbols

releases: Add build number to Nightly builds (#42990)

2025-11-24 13:34:04 +01:00

prompt_store

prompt_store: Remove unused code (#41473)

2025-10-29 16:27:30 +00:00

proto

project: Send LSP metadata to remote ServerInfo (#42831)

2025-11-24 23:16:35 +02:00

recent_projects

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

refineable

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

release_channel

releases: Add build number to Nightly builds (#42990)

2025-11-24 13:34:04 +01:00

remote

multi_buffer: Fix up some anchor checks (#43454)

2025-11-25 13:41:19 +00:00

remote_server

Only show ssh logs when toggled (#43445)

2025-11-25 08:25:49 +00:00

repl

multi_buffer: Typed MultiBufferOffset (#42707)

2025-11-19 22:00:58 +00:00

reqwest_client

Add experimental Sweep edit prediction provider (#42927)

2025-11-18 01:00:26 +00:00

rich_text

Revert "sum_tree: Replace rayon with futures (#41586) (#41846)

2025-11-03 19:25:15 +00:00

rope

rope: Turn ChunkSlice::slice panics into error logs (#43538)

2025-11-26 12:43:30 +01:00

rpc

ACP Extensions (#40663)

2025-10-24 07:52:51 -04:00

rules_library

Fix Gemini 3 on OpenRouter (#43416)

2025-11-24 13:24:26 -05:00

scheduler

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

schema_generator

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

search

Support bracket colorization (rainbow brackets) (#43172)

2025-11-20 19:47:39 +00:00

session

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

settings

mcp: Fix source property showing up as undefined in settings (#43417)

2025-11-25 16:03:21 +00:00

settings_json

settings_ui: Show migration banner (#41112)

2025-10-27 10:13:26 -04:00

settings_macros

Fallible Settings (#42938)

2025-11-21 08:28:17 -07:00

settings_profile_selector

Automate settings registration (#42238)

2025-11-07 22:27:14 +00:00

settings_ui

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

snippet

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

snippet_provider

Add support for multi-word snippet prefixes (#42398)

2025-11-11 16:34:25 -05:00

snippets_ui

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

sqlez

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

sqlez_macros

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

story

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

storybook

ui: Remove outdated/unused component stories (#43118)

2025-11-20 01:52:13 -03:00

streaming_diff

Revert "sum_tree: Replace rayon with futures (#41586) (#41846)

2025-11-03 19:25:15 +00:00

sum_tree

multi_buffer: Remove redundant buffer id field (#43459)

2025-11-25 17:13:16 +01:00

supermaven

zeta2: Predict at next diagnostic location (#43257)

2025-11-21 16:39:08 +00:00

supermaven_api

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

svg_preview

svg_preview: Remove unnecessary dependency on editor (#43147)

2025-11-20 12:18:50 +01:00

system_specs

releases: Add build number to Nightly builds (#42990)

2025-11-24 13:34:04 +01:00

tab_switcher

Automate settings registration (#42238)

2025-11-07 22:27:14 +00:00

task

debugger: Allow users to include PickProcessId in debug tasks and resolve Pid (#42913)

2025-11-20 10:12:59 -05:00

tasks_ui

multi_buffer: Typed MultiBufferOffset (#42707)

2025-11-19 22:00:58 +00:00

telemetry

Fix telemetry (#42184)

2025-11-07 11:36:05 +00:00

telemetry_events

releases: Add build number to Nightly builds (#42990)

2025-11-24 13:34:04 +01:00

terminal

terminal: Update search match highlights on resize (#43507)

2025-11-26 03:45:54 +00:00

terminal_view

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

text

multi_buffer: Remove redundant buffer id field (#43459)

2025-11-25 17:13:16 +01:00

theme

Treat .h++ files as C++ (#42802)

2025-11-24 11:36:04 +02:00

theme_extension

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

theme_importer

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

theme_selector

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

time_format

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

title_bar

Make drag and double click enabled on the whole title bar on macOS (#41839)

2025-11-14 22:46:35 +00:00

toolchain_selector

Fix git features not working when a Windows host collaborates with a unix guest (#43515)

2025-11-26 16:56:34 +00:00

ui

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

ui_input

debugger_ui: Improve some elements of the UI (#43344)

2025-11-22 23:28:34 -03:00

ui_macros

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

ui_prompt

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

util

Fix git features not working when a Windows host collaborates with a unix guest (#43515)

2025-11-26 16:56:34 +00:00

util_macros

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

vercel

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

vim

Fix git features not working when a Windows host collaborates with a unix guest (#43515)

2025-11-26 16:56:34 +00:00

vim_mode_setting

Automate settings registration (#42238)

2025-11-07 22:27:14 +00:00

watch

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

web_search

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

web_search_providers

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

workspace

workspace: Fix broken main build after #43518 (#43570)

2025-11-26 17:21:22 +00:00

worktree

Fix language server renaming when parent directory does not exist (#43499)

2025-11-26 19:34:03 +00:00

worktree_benchmarks

Automate settings registration (#42238)

2025-11-07 22:27:14 +00:00

x_ai

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

zed

gpui: Unify track_scroll method to receive a reference type (#43518)

2025-11-26 18:03:42 +01:00

zed_actions

chore: Fix build graph - again (#42999)

2025-11-18 19:20:34 +01:00

zed_env_vars

Remove workspace-hack (#40216)

2025-10-17 18:58:14 +00:00

zeta

edit prediction: Request trigger (#43588)

2025-11-26 20:34:29 +00:00

zeta2_tools

Separate experimental edit prediction jumps feature from the Sweep AI prediction provider (#43481)

2025-11-25 10:36:45 -08:00

zeta_cli

zeta2: Compute diff-aware chrF metric (#43485)

2025-11-27 11:10:35 +02:00

zlog

cli: Allow opening non-existent paths (#43250)

2025-11-24 11:30:19 +02:00

zlog_settings

Automate settings registration (#42238)

2025-11-07 22:27:14 +00:00