PhaseoPhaseoPhaseoPhaseo
ModèlesChatComparerApplicationsClassements
PhaseoPhaseo
Vérification du statutVérification du statutConsulter la page du statut
Le statut au niveau du composant est indisponible.

Explorer

  • Modèles
  • Chat
  • Fournisseurs
  • Applications
  • Classements
  • Outils

Ressources

  • Comparer
  • Guides de migration
  • Cours
  • Méthodologie
  • Blog

Communauté

  • Discord
  • GitHub
  • LinkedIn
  • Reddit
  • X

Développer

  • Documentation
  • Référence API
  • Démarrage rapide
  • SDK

Entreprise

  • À propos
  • Centre de confiance
  • Mission
  • Tarifs
  • Compatible avec
  • Remerciements
  • Assistance
  • Confidentialité
  • Conditions

Explorer

  • Modèles
  • Chat
  • Fournisseurs
  • Applications
  • Classements
  • Outils

Développer

  • Documentation
  • Référence API
  • Démarrage rapide
  • SDK

Ressources

  • Comparer
  • Guides de migration
  • Cours
  • Méthodologie
  • Blog

Entreprise

  • À propos
  • Centre de confiance
  • Mission
  • Tarifs
  • Compatible avec
  • Remerciements
  • Assistance
  • Confidentialité
  • Conditions

Communauté

  • Discord
  • GitHub
  • LinkedIn
  • Reddit
  • X

© 2025 - 2026 • Phaseo

Changer de langue: Français (France)
  • English (UK)
  • 简体中文
  • हिन्दी
  • Español (España)
  • Français (France)
  • Deutsch (Deutschland)
  • Português (Brasil)
  • 日本語
  • العربية

Aide :Problème·Assistance

Besoin d’aide avec Phaseo ?Signaler un problèmeoucontacter l’assistance

  • Who needs LLM rate limits and AI budgets
  • How rate limiting works on Phaseo
  • Rate limit policies for requests, tokens and concurrency
  • Rate limit per API key, user, team or end user
  • Exact limits when you need them
  • AI budgets per department that stop spending at 100%
  • Count each person once with primary department attribution
  • 429 responses and rate limit headers
  • Set up rate limits and budgets in the dashboard or the API
  • Good to know
  • Get started
  • Questions fréquentes

Sur cette page

  • Who needs LLM rate limits and AI budgets
  • How rate limiting works on Phaseo
  • Rate limit policies for requests, tokens and concurrency
  • Rate limit per API key, user, team or end user
  • Exact limits when you need them
  • AI budgets per department that stop spending at 100%
  • Count each person once with primary department attribution
  • 429 responses and rate limit headers
  • Set up rate limits and budgets in the dashboard or the API
  • Good to know
  • Get started
  • Questions fréquentes
Retour au blog

Phaseo Team2 nov. 20266 min de lecture

LLM rate limits and AI budgets for every key, person, team and end user

Rate limit LLM requests, tokens and concurrency per API key, team member, department or end user, and cap AI spend with department budgets that block at 100%.

LLM rate limits and AI budgets for every key, person, team and end user cover image

Ce contenu éditorial est actuellement disponible en anglais. L’interface est traduite ; l’article original est affiché afin d’en préserver le sens.

Most teams find out they needed a limit right after something used up the shared capacity. An agent loops overnight, a batch script retries every failure at once, or one heavy user of your app slows things down for everyone else.

Phaseo now has rate limit policies and department budgets. You decide how many requests, tokens and parallel calls each API key, team member, department or end user gets, and how much each department can spend before its requests stop. It works across the models and providers you call through Phaseo. Limits for keys, members and departments need no code changes, and end-user limits need your app to send an end-user ID, if it doesn't already. Nothing is limited until you set it up.

Rate limits for keys, members, guardrails and end users are available on every workspace. Department limits and budgets come with Self Serve Enterprise, alongside SSO, SCIM and directory management.

Who needs LLM rate limits and AI budgets

Teams building AI products

One customer running a bulk job shouldn't slow your app for the rest. Give every end user their own allowance, keyed on the user ID you already send.

Companies rolling out AI by department

Engineering, support and marketing can share one workspace, and each department gets its own pool of requests and its own budget. Included with Self Serve Enterprise.

Platform teams running agents

A per-key limit with a concurrency cap stops a runaway agent or batch job from using the capacity your production traffic needs.

Whoever owns the AI bill

When a department reaches 100% of its budget, Phaseo refuses its requests instead of only sending a warning.

How rate limiting works on Phaseo

Every request passes through the same checks before it reaches a model. If any check fails, Phaseo returns a 429 straight away. The request never reaches the provider, so it costs nothing.

  1. Request arrives

    From your app, an agent or a teammate's key.

    For any model or provider

  2. Rate limit policies

    Requests, tokens and concurrency for the key, member, department, guardrail and end user.

    429 rate_limit_exceeded

  3. Department budget

    Daily, weekly or monthly spend for the departments the request counts toward.

    429 key_limit_exceeded

  4. Model provider

    Routing and fallbacks as usual.

    Response, with rate limit headers

Each request is checked against your limits before it reaches a provider.

Rate limit policies for requests, tokens and concurrency

A policy is a named set of limits that you can reuse anywhere:

LimitWhat it countsGood for
Requests per minuteCalls over a rolling minuteFair access and abuse protection
Tokens per minuteInput and output tokens over a rolling minuteMatching real model load and cost
Concurrent requestsRequests in progress at the same timeLong streams, agents and batch scripts

Rate limit per API key, user, team or end user

You assign a policy to whatever it should limit. A department can share a pool of 600 requests a minute while each person in it is held to 60. When several policies apply, Phaseo counts the request against each of them, and the first one to run out refuses it.

key

API key

That key on its own.
member

Team member

One allowance across everything a person sends: all their keys and OAuth requests.
department

Department

One shared pool for everyone in the department.
guardrail

Guardrail

A separate allowance for each key the guardrail covers.
end_user_default

End user

A separate allowance for each of your own users, from the user field or X-Phaseo-User-Id header.

Fallbacks to another provider within one request count once, and responses served from cache don't count at all.

Exact limits when you need them

Request and token limits are counted in the background by default, so they add no latency. The trade-off is that a sudden burst can briefly go over the limit before the count catches up.

When a limit must hold exactly, mark the policy strict. Strict limits are checked before the request is sent, which adds a little latency. If the check can't complete within two seconds, Phaseo refuses the request instead of letting it through. Concurrency limits are always exact: a request holds its slot until its response has been sent in full, including the whole of a streamed response.

AI budgets per department that stop spending at 100%

On Self Serve Enterprise workspaces, give a department a daily, weekly or monthly budget, and Phaseo tracks its spend as requests complete. Owners and admins get a push notification at about 80% and again at 100%. Once a budget is used up, Phaseo refuses that department's API-key requests until the period resets, at midnight UTC, on Mondays or on the 1st of the month.

A department's spend includes the keys linked to it. Keys that aren't linked to a department count toward the departments of the person who created them, and OAuth requests count toward the departments of the signed-in person.

Count each person once with primary department attribution

People often belong to more than one department, especially when departments are synced from your identity provider through SCIM. By default their unlinked keys count toward every department they're in, so each of those departments sees the spend.

To count each person once, switch the workspace to primary department only under Settings → Notifications. Their spend then goes to their primary department alone. Department budgets and department rate limits follow the same setting, so the numbers agree everywhere. A key linked to a department, or a member's department override, still takes priority.

429 responses and rate limit headers

Responses to requests that a policy covers include the standard rate limit headers, so OpenAI-compatible clients and retry logic work unchanged:

x-ratelimit-limit-requests: 60
x-ratelimit-remaining-requests: 41
x-ratelimit-reset-requests: 60s

A refused request returns 429 with a Retry-After header and a body that names the limit that refused it:

{
  "error": {
    "code": "rate_limit_exceeded",
    "reason": "policy_requests_per_minute",
    "limit_scope": "department",
    "policy_id": "…"
  }
}

A department budget refusal uses key_limit_exceeded with the reason department_monthly_cost_budget_reached (or daily or weekly), the department's name and reset_at, so you can tell people when they can try again.

Set up rate limits and budgets in the dashboard or the API

Everything is in the dashboard under Settings → Rate limits and Settings → Budgets. To manage limits from code, use the management API. For example, a shared pool for a department with a monthly budget on top:

curl https://api.phaseo.app/v1/rate-limits/policies \
  -H "Authorization: Bearer $PHASEO_MANAGEMENT_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "Support team", "requests_per_minute": 600, "concurrent_requests": 20}'

curl https://api.phaseo.app/v1/rate-limits/assignments \
  -H "Authorization: Bearer $PHASEO_MANAGEMENT_KEY" \
  -H "Content-Type: application/json" \
  -d '{"policy_id": "<policy-id>", "target_type": "department", "target_id": "<department-id>"}'

curl https://api.phaseo.app/v1/department-budgets \
  -H "Authorization: Bearer $PHASEO_MANAGEMENT_KEY" \
  -H "Content-Type: application/json" \
  -d '{"department_id": "<department-id>", "interval": "monthly", "limit": 500}'

Rate limit and budget commands for the Phaseo CLI, and resources for the Terraform provider, come in their next releases. You'll then be able to review and version limits alongside the rest of your infrastructure.

Good to know

  • These limits come on top of each workspace's built-in allowance, which still applies.
  • By default, a policy lets requests through if its limits briefly can't be checked. A department budget does the same if its check is unavailable, so a department can briefly spend past its budget. Use a strict policy when a rate limit must never be exceeded.
  • Batch jobs, realtime sessions and OAuth requests aren't blocked by department budgets yet. Their spend still counts toward the department.
  • Budget notifications are based on usage that can lag by up to an hour, so a request can be refused before the 100% notification arrives.
  • Policy changes apply within about a minute. Department membership changes, including those from SCIM, can take up to about five minutes.

Get started

Open Settings → Rate limits to create your first policy, or Settings → Budgets to give a department a budget. The rate limits guide covers every field, error reason and header, and guardrails let you restrict which models and providers each key can use.

Questions fréquentes

LLM rate limiting caps how much traffic can reach a model in a given time, usually as requests per minute, tokens per minute or the number of requests in progress at once. It stops one caller from using up shared capacity or budget, so other callers' requests still get through.

Create a Phaseo rate limit policy and assign it as the end-user default. Every end user then gets their own allowance, identified by the user field or X-Phaseo-User-Id header on each request. Requests without an end-user ID aren't limited by that policy.

Requests per minute counts calls over a rolling minute. Tokens per minute counts input and output tokens over a rolling minute, so it tracks real model load. A concurrency limit caps how many requests can be in progress at the same time, which suits long streams and agents.

Yes, on Self Serve Enterprise workspaces, which include departments. Give a department a daily, weekly or monthly budget. Owners and admins get a push notification at about 80% and at 100%, and once the budget is used up, Phaseo refuses API-key requests that count toward that department until the period resets.

The request returns HTTP 429 with a Retry-After header. A rate limit returns the rate_limit_exceeded error code with the policy and scope that refused it. A department budget returns key_limit_exceeded with the department's name and when the budget resets.

Not by default. Request and token limits are counted in the background. Strict policies and concurrency limits are checked before the request is sent, which adds a little latency in exchange for exact limits.

Responses served from cache don't count. Fallbacks to another provider within one request count once.

Yes. Phaseo sends the standard x-ratelimit-limit-requests, x-ratelimit-remaining-requests and x-ratelimit-reset-requests headers, the matching -tokens headers for token limits, and a Retry-After header on 429 responses, which OpenAI-compatible clients and retry libraries already understand.

Créer un compte