oak-team/oak-gpui
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

language_models: Add support for images to Mistral models (#32154)

Browse Source 
Tested with following models. Hallucinates with whites outline images
like white lined zed logo but works fine with zed black outlined logo:

Pixtral 12B (pixtral-12b-latest)
Pixtral Large (pixtral-large-latest)
Mistral Medium (mistral-medium-latest)
Mistral Small (mistral-small-latest)

After this PR, almost all of the zed's llm provider who support images
are now supported. Only remaining one is LMStudio. Hopefully we will get
that one as well soon.

Release Notes:

- Add support for images to mistral models

---------

Signed-off-by: Umesh Yadav <git@umesh.dev>
Co-authored-by: Bennet Bo Fenner <bennetbo@gmx.de>
Co-authored-by: Bennet Bo Fenner <bennet@zed.dev>
This commit is contained in:
Umesh Yadav
2025-06-09 10:00:02 +00:00
committed by GitHub
co-authored by Bennet Bo Fenner Bennet Bo Fenner
parent 4ac7935589
commit 0bc9478b46
3 changed files with 257 additions and 92 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/src/ai/configuration.md
+4 -3
View File
@@ -302,7 +302,8 @@ The Zed Assistant comes pre-configured with several Mistral models (codestral-la
"max_tokens": 32000,
"max_output_tokens": 4096,
"max_completion_tokens": 1024,
"supports_tools": true
"supports_tools": true,
"supports_images": false
}
]
}
@@ -374,10 +375,10 @@ The `supports_tools` option controls whether or not the model will use additiona
If the model is tagged with `tools` in the Ollama catalog this option should be supplied, and built in profiles `Ask` and `Write` can be used.
If the model is not tagged with `tools` in the Ollama catalog, this option can still be supplied with value `true`; however be aware that only the `Minimal` built in profile will work.
The `supports_thinking` option controls whether or not the model will perform an explicit “thinking” (reasoning) pass before producing its final answer.
The `supports_thinking` option controls whether or not the model will perform an explicit “thinking” (reasoning) pass before producing its final answer.
If the model is tagged with `thinking` in the Ollama catalog, set this option and you can use it in zed.
The `supports_images` option enables the model’s vision capabilities, allowing it to process images included in the conversation context.
The `supports_images` option enables the model’s vision capabilities, allowing it to process images included in the conversation context.
If the model is tagged with `vision` in the Ollama catalog, set this option and you can use it in zed.
### OpenAI {#openai}