Every large internet service eventually discovers that “please fetch this small piece of data” is not a small request when a billion people are asking. OpenAI calls the system handling much of that traffic Habitat, and its latest engineering account is a useful tour of what happens after an application outgrows ordinary nouns.
OpenAI says Habitat now serves more than 70 million requests per second across nearly 40 regions, supporting more than one billion weekly users and roughly 500 petabytes of data. The service began in Python. Two engineers, working with Codex and GPT-5.5, later rewrote its data plane in Rust; OpenAI reports that the Rust version is six times as CPU-efficient and 15 times as memory-efficient, and now handles 95 percent of production requests.
When “get” becomes a distributed-systems problem
Habitat sits between products and storage systems, routing requests while hiding some of the complexity underneath. At modest scale, that can sound like a thin abstraction. At tens of millions of requests per second, it becomes a place where a small inefficiency buys a very large electricity bill.
The rewrite numbers are striking, but they should not become a bumper sticker declaring that every Python service needs an emergency Rust transplant. OpenAI’s account describes a specific, mature bottleneck with enormous volume. Python helped the system move quickly early on; Rust became valuable when predictable performance and resource efficiency mattered at a different scale.
The involvement of coding agents is also notable, though the human scope is the stronger lesson. Two engineers could coordinate a rewrite because the boundary was understood, the performance target was measurable, and the new system could be introduced gradually. “Rewrite everything” remains a terrible plan. “Replace the hottest path and measure it mercilessly” is much more promising.
The signal
Consumer AI is usually discussed through models, features, and personalities. The less visible story is the enormous layer of storage, routing, caching, and failure handling required to make a response feel immediate. Reliability is produced by systems most users will never know exist.
Habitat is an oddly cozy name for infrastructure processing 70 million requests each second. At that point it is less a habitat than a small digital climate.
