Even the best models encounter occasional failures. This guide explains how the Gateway reports errors and how you should respond.
Error categories
Errors can originate from the Gateway, the provider, or your request. The error response will include metadata to help you diagnose the issue.
Retry behavior
- Implement exponential backoff with jitter.
- Use the
Retry-After header when present.
- Avoid infinite retries by setting a max retry count.
Provider errors
Sometimes errors originate upstream. We will attempt to indicate this in the error response. The router should attempt to navigate you away from that provider if possible, should these errors keep happening.
Next steps
Last modified on July 26, 2026
