A request can be throttled by the selected provider or by an applicable workspace or key policy. Routing and fallbacks can reduce provider failures, but a request can still return 429.
Provider limits
BYOK requests use the limits of your provider account. Managed requests use shared provider capacity; that capacity is not an individual workspace allowance.
Phaseo forwards a validated Retry-After value when the routed provider supplies one. BYOK responses can also include X-Phaseo-Upstream-RateLimit-* headers. These quota headers are omitted for managed requests because they describe shared provider capacity. Other upstream headers are not forwarded.
Handle a throttled request
- Inspect the error code to distinguish temporary provider throttling from a policy or budget restriction that requires configuration changes.
- Respect
Retry-After when present. It can be a delay in seconds or an HTTP date; validate the value before scheduling a retry.
- Otherwise, use exponential backoff with jitter and a maximum delay.
- Bound retries by both attempt count and an overall deadline. Cancel abandoned work and release each failed response body before retrying.
- Reduce concurrency if throttling persists.
Do not automatically retry an ambiguous job submission or replay completed tool actions. A request may have reached the provider even if the client lost its response; retries can duplicate work and charges. For an accepted video or batch job, use its status endpoint rather than submitting it again.
Monitor throttling
Track 429 rates by endpoint and model, and inspect fallback frequency in request logs. Compare your application logs with routing settings and provider account quotas for BYOK traffic.
See budgets and guardrails for workspace spending and access controls. Last modified on September 6, 2026
