Code block evals (#29619)

Add a targeted eval for code block formatting, and revise the system
prompt accordingly.

### Eval before, n=8

<img width="728" alt="eval before"
src="https://github.com/user-attachments/assets/552b6146-3d26-4eaa-86f9-9fc36c0cadf2"
/>

### Eval after prompt change, n=8 (excluding the new evals, so just
testing the prompt change)

<img width="717" alt="eval after"
src="https://github.com/user-attachments/assets/c78c7a54-4c65-470c-b135-8691584cd73e"
/>

Release Notes:

- N/A

This commit is contained in:

Richard Feldman

2025-04-29 18:52:09 -04:00

committed by

GitHub

parent 2508e491d5

commit d7004030b3

10 changed files with 536 additions and 134 deletions

Cargo.lock

Generated

View File

@@ -4993,6 +4993,7 @@ dependencies = [
  "language_model",
  "language_models",
  "languages",
  "markdown",
  "node_runtime",
  "pathdiff",
  "paths",