OhMyGPT

Streaming

Server-sent events on each protocol, usage and cost at the end of a stream, errors mid-stream, and cancellation.

Every chat model streams on all four protocols. Set stream: true on Chat Completions, Responses, or Messages, or call :streamGenerateContent on Gemini. The response is a stream of server-sent events in the format of the protocol you called. Streams end with token usage (on Chat Completions, when you ask for it), and Chat Completions, Responses, and Messages streams also carry the request's charge.

ProtocolEventsLast event
Chat Completionsdata: {chat.completion.chunk}data: [DONE]
Responsesresponse.created, response.output_text.delta, …response.completed, response.incomplete, or response.failed
Messagesmessage_start, content_block_delta, ping, …message_stop
Geminidata: {GenerateContentResponse}the chunk carrying usageMetadata

With an SDK

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.ohmygpt.com/v1",
    api_key=os.environ["OHMYGPT_API_KEY"],
)

stream = client.chat.completions.create(
    model="ohmygpt/mock",
    messages=[{"role": "user", "content": "ping"}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

As on OpenAI, a Chat Completions stream reports usage only when you set "stream_options": {"include_usage": true}. Usage then arrives in one last chunk whose choices array is empty, so check choices before you index into it.

Raw frames

The end of a Chat Completions stream with stream_options.include_usage set:

data: {"id":"chatcmpl-ohmygpt-mock","object":"chat.completion.chunk","created":1793318400,"model":"ohmygpt/mock","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-ohmygpt-mock","object":"chat.completion.chunk","created":1793318400,"model":"ohmygpt/mock","choices":[],"usage":{"prompt_tokens":29,"completion_tokens":17,"total_tokens":46}}

: x-cost-credits: 0
data: [DONE]

The end of a Messages stream:

event: message_delta
data: {"delta":{"stop_reason":"end_turn","stop_sequence":null,"stop_details":null},"type":"message_delta","usage":{"input_tokens":33,"output_tokens":17}}

: x-cost-credits: 0
event: message_stop
data: {"type":"message_stop"}

If you parse the stream yourself, read it line by line, skip lines that start with :, and parse the JSON after data: . On Chat Completions, stop at data: [DONE].

Keep-alive

A model can go quiet for minutes: before its first token while it thinks, or while it builds a large tool call. Proxies and NAT devices between you and the API often close a connection that carries no bytes. To keep it open, Chat Completions, Responses, and Messages streams carry an SSE comment line after every 15 seconds of silence:

: keep-alive
: keep-alive
: keep-alive
: keep-alive
: keep-alive
: keep-alive
: keep-alive
: keep-alive
data: {"id":"chatcmpl-EjizVUFmGLjgYzu4TcCL30VobpoVy","object":"chat.completion.chunk","created":1794566413,"model":"openai/gpt-5.1",...}

SDKs skip comment lines. Chat Completions requests that send Go's default User-Agent (Go-http-client/…, as go-openai does) get none, because go-openai can't read an error that follows a comment line.

Headers that arrive in the stream

Four kinds of response headers are sent with the first output: the TEE proof receipt (X-VeriRelay-*), X-Output-Tokens-Capped when a low balance lowered the output limit, X-Parameters-Dropped when sampling parameters were left out, and Deprecation / Sunset for a model being retired. A stream that has sent a keep-alive line has already sent its headers, so these values then arrive as lowercase comment lines ahead of the first event instead:

: keep-alive
: x-output-tokens-capped: 12000
: x-verirelay-request-id: 36a420474e713c4ecbe0a6d5dfd79ed0
: x-verirelay-store-id: cFMrwmc4IeTQT14_R9y6gg
data: {"id":"chatcmpl-...","object":"chat.completion.chunk",...}

A stream that sent no keep-alive line before its first output carries them as response headers, as a non-streaming response does. X-VeriRelay-Store-Key is never written into a stream, because stream bodies are often forwarded or logged as they are: when it did not arrive as a header, look the request up to get it.

Cost of a stream

Chat Completions, Responses, and Messages streams carry the charge in Credits as an SSE comment line, : x-cost-credits: N, right before the last event. SDKs skip comment lines, so read the raw stream to capture it.

For any request, streaming or not and on any protocol, you can also look the charge up with its X-Request-ID response header:

curl https://api.ohmygpt.com/v1/generations/$REQUEST_ID \
  -H "Authorization: Bearer $OHMYGPT_API_KEY"
{
  "object": "request_receipt",
  "id": "01a19b55-f2f8-787b-941c-7e3e0bda2b0a",
  "status": "succeeded",
  "model": { "id": "google/gemini-3.1-flash-lite" },
  "usage": { "input_tokens": 16, "output_tokens": 3, "reasoning_tokens": 118, "cache_read_tokens": 0, "cache_creation_tokens": 0 },
  "cost": { "list_credits": 48, "credits": 43 }
}

Errors mid-stream

Errors before the first byte come back as a normal HTTP error. Once the stream has started with 200, whether with output or with a keep-alive line, a failure ends the stream with one error event in your protocol's format:

ProtocolFinal event
Chat Completionsdata: {"error": {...}}
Responsesevent: response.failed, with response.error.code and response.error.message. When nothing came before it but keep-alive lines, a response.created event comes first.
Messagesevent: error, with {"type": "error", "error": {...}}
Geminidata: {"error": {"code": ..., "status": ...}}

Any output received before the error is incomplete. Retry the request.

Cancellation

Close the connection to stop receiving a stream. To bound what a response can cost, set an output limit (max_tokens, max_completion_tokens or max_output_tokens) on the request.

Was this page helpful?

Still stuck? Live chat · [email protected] · Telegram group