Sort providers by price, latency, or throughput

Use this recipe when the model stays the same but you want the gateway to rank provider offers differently for one request.

Goal

Prefer the cheapest provider for one request.
Prefer the lowest-latency provider for interactive paths.
Prefer the highest-throughput provider for bulk generation.

1. Add `provider.sort` to the request

For text requests, set the sort you want directly on the provider object.

curl https://api.phaseo.ai/v1/responses \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemini-3.1-flash-lite",
    "input": "Give me one release-note bullet for the last deploy.",
    "provider": {
      "sort": "price"
    }
  }'

import Phaseo from "@phaseo/sdk";

const client = new Phaseo({ apiKey: process.env.PHASEO_API_KEY! });

const response = await client.generateResponse({
  model: "google/gemini-3.1-flash-lite",
  input: "Give me one release-note bullet for the last deploy.",
  provider: {
    sort: "price",
  },
});

console.log(response.output_text);

from phaseo import Phaseo

client = Phaseo(api_key="YOUR_API_KEY")

response = client.generate_response(
    {
        "model": "google/gemini-3.1-flash-lite",
        "input": "Give me one release-note bullet for the last deploy.",
        "provider": {
            "sort": "price",
        },
    }
)

print(response.get("output_text"))

package main

import (
  "context"
  "fmt"

  phaseo "github.com/phaseoteam/Phaseo/packages/sdk/sdk-go"
)

func main() {
  client := phaseo.New("YOUR_API_KEY", "https://api.phaseo.ai/v1")

  response, err := client.GenerateResponse(context.Background(), phaseo.ResponsesRequest{
    Model: "google/gemini-3.1-flash-lite",
    Input: "Give me one release-note bullet for the last deploy.",
    Provider: map[string]interface{}{
      "sort": "price",
    },
  })
  if err != nil {
    panic(err)
  }

  fmt.Println(response)
}

using PhaseoSdk;
using System.Collections.Generic;

var client = new Phaseo("YOUR_API_KEY");

var response = await client.GenerateResponse(new Dictionary<string, object>
{
    ["model"] = "google/gemini-3.1-flash-lite",
    ["input"] = "Give me one release-note bullet for the last deploy.",
    ["provider"] = new Dictionary<string, object>
    {
        ["sort"] = "price"
    }
});

Console.WriteLine(response);

<?php
require 'vendor/autoload.php';

use Phaseo\Sdk\Phaseo;

$client = new Phaseo(getenv('PHASEO_API_KEY') ?: 'YOUR_API_KEY');

$response = $client->generateResponse([
    'model' => 'google/gemini-3.1-flash-lite',
    'input' => 'Give me one release-note bullet for the last deploy.',
    'provider' => [
        'sort' => 'price',
    ],
]);

print_r($response);

require 'phaseo_sdk'

client = PhaseoSdk::Phaseo.new(api_key: ENV.fetch('PHASEO_API_KEY', 'YOUR_API_KEY'))

response = client.generate_response(
  model: 'google/gemini-3.1-flash-lite',
  input: 'Give me one release-note bullet for the last deploy.',
  provider: {
    sort: 'price'
  }
)

puts response

Supported routing sorts for this flow:

price
latency
throughput

2. Pick the sort that matches the workload

Use:

price when cost matters more than tail latency
latency for chat, copilots, and human-in-the-loop tools
throughput for high-volume generation or backfills

3. Understand what the gateway compares

When you send an explicit request-level sort, the gateway ranks candidates deterministically instead of using the normal balanced weighted shuffle. For text models:

price compares a common price basis across the eligible providers
if shared text meters exist, the gateway prefers matching input_text_tokens and output_text_tokens
latency uses the latest provider latency data
throughput uses the latest throughput measurements

4. Keep the provider pool realistic

Sorting works best after you narrow the candidate pool when needed. For example, sort only among one approved set of providers:

curl https://api.phaseo.ai/v1/responses \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemini-3.1-flash-lite",
    "input": "Summarize the incident in one sentence.",
    "provider": {
      "only": ["google-vertex", "google-vertex-eu"],
      "sort": "latency"
    }
  }'

import Phaseo from "@phaseo/sdk";

const client = new Phaseo({ apiKey: process.env.PHASEO_API_KEY! });

const response = await client.generateResponse({
  model: "google/gemini-3.1-flash-lite",
  input: "Summarize the incident in one sentence.",
  provider: {
    only: ["google-vertex", "google-vertex-eu"],
    sort: "latency",
  },
});

console.log(response.output_text);

from phaseo import Phaseo

client = Phaseo(api_key="YOUR_API_KEY")

response = client.generate_response(
    {
        "model": "google/gemini-3.1-flash-lite",
        "input": "Summarize the incident in one sentence.",
        "provider": {
            "only": ["google-vertex", "google-vertex-eu"],
            "sort": "latency",
        },
    }
)

print(response.get("output_text"))

package main

import (
  "context"
  "fmt"

  phaseo "github.com/phaseoteam/Phaseo/packages/sdk/sdk-go"
)

func main() {
  client := phaseo.New("YOUR_API_KEY", "https://api.phaseo.ai/v1")

  response, err := client.GenerateResponse(context.Background(), phaseo.ResponsesRequest{
    Model: "google/gemini-3.1-flash-lite",
    Input: "Summarize the incident in one sentence.",
    Provider: map[string]interface{}{
      "only": []string{"google-vertex", "google-vertex-eu"},
      "sort": "latency",
    },
  })
  if err != nil {
    panic(err)
  }

  fmt.Println(response)
}

using PhaseoSdk;
using System.Collections.Generic;

var client = new Phaseo("YOUR_API_KEY");

var response = await client.GenerateResponse(new Dictionary<string, object>
{
    ["model"] = "google/gemini-3.1-flash-lite",
    ["input"] = "Summarize the incident in one sentence.",
    ["provider"] = new Dictionary<string, object>
    {
        ["only"] = new[] { "google-vertex", "google-vertex-eu" },
        ["sort"] = "latency"
    }
});

Console.WriteLine(response);

<?php
require 'vendor/autoload.php';

use Phaseo\Sdk\Phaseo;

$client = new Phaseo(getenv('PHASEO_API_KEY') ?: 'YOUR_API_KEY');

$response = $client->generateResponse([
    'model' => 'google/gemini-3.1-flash-lite',
    'input' => 'Summarize the incident in one sentence.',
    'provider' => [
        'only' => ['google-vertex', 'google-vertex-eu'],
        'sort' => 'latency',
    ],
]);

print_r($response);

require 'phaseo_sdk'

client = PhaseoSdk::Phaseo.new(api_key: ENV.fetch('PHASEO_API_KEY', 'YOUR_API_KEY'))

response = client.generate_response(
  model: 'google/gemini-3.1-flash-lite',
  input: 'Summarize the incident in one sentence.',
  provider: {
    only: ['google-vertex', 'google-vertex-eu'],
    sort: 'latency'
  }
)

puts response

5. Verify the ranked outcome

When debugging, inspect the request in Gateway -> Usage and look for:

providers considered
ranked providers
the routing score factors for price, latency, or throughput

That makes it easy to confirm whether the gateway sorted the way you expected.

​Goal

​1. Add provider.sort to the request

​2. Pick the sort that matches the workload

​3. Understand what the gateway compares

​4. Keep the provider pool realistic

​5. Verify the ranked outcome

​Related guides