SDKs
Streaming
Some responses arrive over time rather than all at once: a token stream from a model, a log tail, a long export. Configure streaming per endpoint and the generated method hands the caller an async iterator instead of a single body.
The settings
Select the endpoint in SDK Studio and open its streaming settings.
| Setting | What it does | Example |
|---|---|---|
| Format | The wire format your endpoint uses | Server-sent events |
| Event field | The field surfaced from each chunk | data |
Formats
What callers get
const stream = await client.chat.complete({ prompt: "Hello" });
for await (const chunk of stream) {
process.stdout.write(chunk.text);
}Event field decides what each chunk carries. For SSE events shaped like this:
data: {"text": "Hel"}
data: {"text": "lo"}Set Event field to data and each iteration yields the parsed object.
Streaming and retries
Retries apply to establishing the request. Once the server has started streaming, the client has already handed chunks to the caller, and replaying the request would duplicate them. A stream that breaks surfaces the error to the caller, who decides whether to restart.
Design streamed endpoints so a caller can resume: an offset, a sequence number, or a resumable cursor in each chunk.
Timeouts
A per-request timeout bounds the whole call, which is rarely what you want for a stream that legitimately runs for minutes.
Give the endpoint its own Timeout that fits the stream, or 0 to disable it. Your project-wide default is meant for ordinary request/response calls. See Retries and timeouts.
Don't stream everything
Streaming costs the caller a loop and gives up a plain return value. Use it when the first byte matters, or when the response is unbounded. A 40 ms JSON response should stay a JSON response.