Executive Summary
A new WebSocket mode has been introduced for the Responses API to address latency bottlenecks in complex, multi-turn agentic workflows. As model inference speeds have dramatically increased (e.g., GPT-5.3-Codex-Spark exceeding 1,000 tokens/second), the overhead from repeated synchronous API calls became a significant performance issue. By establishing a persistent connection, this new mode caches conversation state on the server, eliminating redundant processing and reducing end-to-end workflow latency by up to 40% for developers building agents.
Key Takeaways
* Core Problem: In agentic systems like the Codex coding assistant, numerous back-and-forth API calls created significant latency, negating the speed benefits of faster underlying models.
* Technical Solution: The API now supports WebSockets to create a persistent connection. This allows the server to maintain a connection-scoped, in-memory cache of the conversation state.
* New Implementation: Instead of resending the entire conversation history, developers can now pass a `previous_response_id`. The API then retrieves the cached state, processes only the new input, and appends to the existing context, drastically reducing overhead.
* Performance Gains: The update delivers up to a 40% end-to-end speed improvement in agentic workflows. It enables the API to keep pace with models that can generate over 1,000 tokens per second.
* Availability & Adoption: The feature is now live following a successful alpha phase. Partners like Vercel, Cline, and Cursor have already integrated it and are reporting significant performance improvements.
Strategic Importance
This infrastructure upgrade is critical for ensuring the platform's API does not become a bottleneck for its increasingly powerful models. It makes the service more viable and performant for the growing market of sophisticated, multi-step AI agents.