Streaming

Show output as it is generated instead of waiting for the complete response.

const stream = await client.chat.completions.create({
  model: "YOUR_MODEL",
  messages: [{ role: "user", content: "Hello" }],
  stream: true,
});
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? "");

Enable streaming

Set stream: true on Chat Completions or the equivalent streaming option for your selected operation. Each event contains a small delta; join content deltas in order.

Finish safely

Handle the final usage event and finish reason. If the connection closes, keep the request ID and retry only when your application can safely do so.

Tools while streaming

Tool-call arguments arrive in fragments. Accumulate them and parse the JSON only after the tool call is complete.

Need help?

Keep the request ID and contact Torve support without sharing your API key or private data.

Contact support