Streaming
Server-sent events on each protocol, usage and cost at the end of a stream, errors mid-stream, and cancellation.
Every chat model streams on all four protocols. Set stream: true on Chat Completions, Responses, or Messages, or call :streamGenerateContent on Gemini. The response is a stream of server-sent events in the format of the protocol you called. Streams end with token usage (on Chat Completions, when you ask for it), and Chat Completions, Responses, and Messages streams also carry the request's charge.
| Protocol | Events | Last event |
|---|---|---|
| Chat Completions | data: {chat.completion.chunk} | data: [DONE] |
| Responses | response.created, response.output_text.delta, … | response.completed, response.incomplete, or response.failed |
| Messages | message_start, content_block_delta, ping, … | message_stop |
| Gemini | data: {GenerateContentResponse} | the chunk carrying usageMetadata |
With an SDK
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.ohmygpt.com/v1",
api_key=os.environ["OHMYGPT_API_KEY"],
)
stream = client.chat.completions.create(
model="ohmygpt/mock",
messages=[{"role": "user", "content": "ping"}],
stream=True,
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)As on OpenAI, a Chat Completions stream reports usage only when you set "stream_options": {"include_usage": true}. Usage then arrives in one last chunk whose choices array is empty, so check choices before you index into it.
Raw frames
The end of a Chat Completions stream with stream_options.include_usage set:
data: {"id":"chatcmpl-ohmygpt-mock","object":"chat.completion.chunk","created":1793318400,"model":"mock","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: {"id":"chatcmpl-ohmygpt-mock","object":"chat.completion.chunk","created":1793318400,"model":"mock","choices":[],"usage":{"prompt_tokens":29,"completion_tokens":17,"total_tokens":46}}
: x-cost-credits: 0
data: [DONE]The end of a Messages stream:
event: message_delta
data: {"delta":{"stop_reason":"end_turn","stop_sequence":null,"stop_details":null},"type":"message_delta","usage":{"input_tokens":33,"output_tokens":17}}
: x-cost-credits: 0
event: message_stop
data: {"type":"message_stop"}If you parse the stream yourself, read it line by line, skip lines that start with :, and parse the JSON after data: . On Chat Completions, stop at data: [DONE].
Cost of a stream
Chat Completions, Responses, and Messages streams carry the charge in Credits as an SSE comment line, : x-cost-credits: N, right before the last event. SDKs skip comment lines, so read the raw stream to capture it.
For any request, streaming or not and on any protocol, you can also look the charge up with its X-Request-ID response header:
curl https://api.ohmygpt.com/v1/generations/$REQUEST_ID \
-H "Authorization: Bearer $OHMYGPT_API_KEY"{
"object": "request_receipt",
"id": "01a19b55-f2f8-787b-941c-7e3e0bda2b0a",
"status": "succeeded",
"model": { "id": "google/gemini-3.1-flash-lite" },
"usage": { "input_tokens": 16, "output_tokens": 3, "reasoning_tokens": 118, "cache_read_tokens": 0, "cache_creation_tokens": 0 },
"cost": { "list_credits": 48, "credits": 43 }
}Errors mid-stream
Errors before the first event come back as a normal HTTP error. If the model fails after the stream has started with 200, the stream ends with one error event in your protocol's format:
| Protocol | Final event |
|---|---|
| Chat Completions | data: {"error": {...}} |
| Responses | event: response.failed, with response.error.code and response.error.message |
| Messages | event: error, with {"type": "error", "error": {...}} |
| Gemini | data: {"error": {"code": ..., "status": ...}} |
The output received before the error is incomplete. Retry the request.
Cancellation
Close the connection to stop a stream. Generation stops, and you are charged only for the output already sent to you.