Reliability
Automatic failover between providers, and the few things your client should handle.
When a model is offered by more than one provider, OhMyGPT gives you failover without any code on your side. If a provider fails before it sends any output, the same request goes to another provider. You make one call, get one answer, and pay only for the attempt that answered.
To see which provider answered, look the request up by its X-Request-ID.
Streaming
A stream switches providers only before its first output. Once output has started, the stream stays with its provider to the end. If generation stops partway, the stream ends with an error event that your SDK raises as an exception; send the request again to get a complete answer.
Timeouts
Reasoning models can think for minutes before their first token, and a stream can pause between chunks while the model works. Give your HTTP client, and any proxy in front of it, a read timeout of several minutes so these responses arrive in full:
client = OpenAI(
base_url="https://api.ohmygpt.com/v1",
api_key=os.environ["OHMYGPT_API_KEY"],
timeout=600,
)Stopping a request early
To stop generation partway, use streaming and close the connection: the request stops at the provider, and you pay for the output generated up to that point. A non-streaming request runs to completion once sent, even if your client disconnects.
If a model stays unavailable
When a call returns 5xx after OhMyGPT has tried the model's providers, retry with backoff as described in When to retry, or switch to another model. Because every model uses the same key and API, switching is a one-line change.