2 Commits

Include renames

Author	SHA1	Message	Date
Oleksiy Syvokon	76a78b550b	eval: Write JSON-serialized thread (#29271) ... This adds `last.message.json` file that contains the full request plus response (serialized as a message from assistant for consistency with other messages). Motivation: to capture more info and to make analysis of finished runs easier. Release Notes: - N/A	2025-04-23 15:22:19 +03:00
4 people	ce1a674eba	eval: Fine-grained assertions (#29246) ... - Support programmatic examples ([example](https://github.com/zed-industries/zed/blob/17feb260a0919e102ae4e220669c467885bf4b57/crates/eval/src/examples/file_search.rs)) - Combine data-driven example declarations into a single `.toml` file ([example](https://github.com/zed-industries/zed/blob/17feb260a0919e102ae4e220669c467885bf4b57/crates/eval/src/examples/find_and_replace_diff_card.toml)) - Run judge on individual assertions (previously called "criteria") - Report judge and programmatic assertions in one combined table Note: We still need to work on concept naming <img width=400 src="https://github.com/user-attachments/assets/fc719c93-467f-412b-8d47-68821bd8a5f5"> Release Notes: - N/A --------- Co-authored-by: Richard Feldman <oss@rtfeldman.com> Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com> Co-authored-by: Thomas Mickley-Doyle <tmickleydoyle@gmail.com>	2025-04-22 23:58:58 -03:00