More resilient eval (#32257)
Bubbles up rate limit information so that we can retry after a certain duration if needed higher up in the stack. Also caps the number of concurrent evals running at once to also help. Release Notes: - N/A
This commit is contained in:
@@ -682,11 +682,12 @@ mod tests {
|
||||
_: &AsyncApp,
|
||||
) -> BoxFuture<
|
||||
'static,
|
||||
http_client::Result<
|
||||
Result<
|
||||
BoxStream<
|
||||
'static,
|
||||
http_client::Result<LanguageModelCompletionEvent, LanguageModelCompletionError>,
|
||||
Result<LanguageModelCompletionEvent, LanguageModelCompletionError>,
|
||||
>,
|
||||
LanguageModelCompletionError,
|
||||
>,
|
||||
> {
|
||||
unimplemented!()
|
||||
|
||||
Reference in New Issue
Block a user