The Realtime API makes conversation a state-management problem

A new interaction model

OpenAI introduced the Realtime API in October 2024, enabling low-latency multimodal conversation through a persistent connection. The initial GPT-4o Realtime preview brought audio interaction into a dedicated API workflow.

Unlike a single request followed by a complete answer, a live conversation has interruptions, partial output, and an evolving session state.

Latency is only one part of the experience

A user may interrupt an answer, correct a name, or change their intent halfway through a sentence. The application needs to know which output was actually delivered and which actions remain pending. A fast response that ignores an interruption can feel less useful than a slightly slower system with clear turn handling.

These lessons also apply to interactive creative tools, where users revise instructions while background jobs are still running.

Keep actions explicit

Separate conversational suggestions from operations that change a record, submit a form, or spend a budget. Track action status independently from the spoken response, and define how a dropped connection is recovered. This is industry coverage of a voice API milestone; XMH.NET's service remains focused on image generation and editing workflows.