Add support for queuing status updates in cloud language model provider (#29818)
This sets us up to display queue position information to the user, once
our language model backend is updated to support request queuing.
The JSON returned by the LLM backend will need to look like this:
```json
{"queue": {"status": "queued", "position": 1}}
{"queue": {"status": "started"}}
{"event": {"THE_UPSTREAM_MODEL_PROVIDER_EVENT": "..."}}
```
Release Notes:
- N/A
---------
Co-authored-by: Marshall Bowers <git@maxdeviant.com>
This commit is contained in:
co-authored by
Marshall Bowers
parent
4d1df7bcd7
commit
04772bf17d
@@ -2371,6 +2371,7 @@ impl AssistantContext {
|
||||
});
|
||||
|
||||
match event {
|
||||
LanguageModelCompletionEvent::QueueUpdate { .. } => {}
|
||||
LanguageModelCompletionEvent::StartMessage { .. } => {}
|
||||
LanguageModelCompletionEvent::Stop(reason) => {
|
||||
stop_reason = reason;
|
||||
|
||||
Reference in New Issue
Block a user