Skip to main content
Set stream: true to receive output as server-sent events (SSE). Keep your Phaseo API key on your server and forward the output to your UI.

​
Send a streaming request

Check the HTTP status before reading the stream. A failure before streaming starts returns the normal JSON error response.

​
Read events

Use the event format for the endpoint you called: SSE events are separated by blank lines. A network chunk can contain part of an event or several events; do not parse each network chunk as standalone JSON. Buffer incomplete frames or use an SDK stream helper. Do not assume every endpoint sends [DONE]. The SDK helpers handle stream transport:
  • TypeScript: streamResponses(), streamChat(), streamMessages().
  • Python: stream_responses(), stream_chat(), stream_message().
  • Vercel AI SDK: streamText() with @phaseo/ai-sdk-provider.
See the TypeScript usage guide or Python usage guide for client setup.

​
Stream tool calls

Streaming with tools is accepted by the Gateway, but support depends on the selected model and provider. Accumulate tool-call argument deltas until the call is complete, then parse, validate, and authorise the arguments before execution. Never execute partial arguments. Follow tool-calling patterns for continuation requests and tool safety for validation and approval controls.

​
Cancel and recover

Pass an AbortSignal to your server-side HTTP client and cancel when the user stops or disconnects. Set a deadline appropriate to the workload. Cancellation does not guarantee that the provider stops immediately or that generated output is free. Render partial text progressively, but mark a response complete only after the endpoint’s successful terminal event. A closed connection alone is not proof of completion. If a stream fails after output begins:
  1. Mark the partial response as interrupted.
  2. Keep the request or generation ID, endpoint, and model in your logs.
  3. Offer an explicit retry. A new request can generate different output and incur another charge.
  4. Do not replay executed tools or append a retried response to the partial output automatically.
For failures before output begins, use bounded retries and respect rate-limit headers. If your application must validate the entire output before displaying it, use a non-streaming request or buffer the result on your server.
Last modified on September 6, 2026