700 Tokens Per Second: How Google's Chips Are Breaking Into Meta's Infrastructure

The headline number is 700 tokens per second. That is the reported speed for a single user on Google's seventh-generation Ironwood TPU, a figure that significantly outpaces the 452 tokens per second achieved on Nvidia's GB200 hardware in comparative tests. This performance gap highlights a new reality in the AI hardware market: Google’s custom chips are no longer just internal tools but are now a viable, high-speed option for external giants.

The core development is that Google’s Tensor Processing Units are now serving AI workloads in production at Meta. This was made possible through the release of SGLang, a new inference engine that runs natively on PyTorch rather than Google’s proprietary JAX framework. Meta’s PyTorch-TPU lead, Zongwei Zhou, confirmed that this setup is live in their production environment. He noted that the collaboration between Meta, Google, and the SGLang team has resulted in a "remarkably mature, PyTorch-native serving stack," allowing Meta to utilize the hardware without the usual friction of rewriting code.

This move signals a strategic shift for Meta, which has historically been wary of being tied to a single chip provider. Jana van Greunen, director of PyTorch engineering at Meta, stated that the company "cannot be constrained by a single hardware architecture." By adopting the new TorchTPU bridge, Meta can deploy its existing PyTorch workloads directly onto Google’s hardware. Google co-developed this solution specifically with Meta’s team, acknowledging that Meta did not want to migrate its entire operation to JAX.

However, this is not Meta's first foray into using Google's chips. The new system is described as a successor to the older PyTorch/XLA bridge. Reports from last year indicated that Meta research teams used TPUs through that older bridge between 2020 and 2023 before canceling those contracts. Industry analysts had previously suggested that Meta was interested in renting TPUs via Google Cloud in 2026, with potential purchases following in 2027. The current announcement confirms that those discussions have moved from talk to actual production deployment, though the exact physical location of these chips remains undisclosed.

The timing of this release is also significant. It arrived just one day after Amazon announced it was open-sourcing TorchNeuron, a similar native backend for its own Trainium chips. Both moves appear designed to weaken the software lock-in that has long favored Nvidia. For the performance benchmarks, a tuned version of vLLM served Moonshot AI’s 2.8-trillion-parameter Kimi K3 model. In tests using 16 Ironwood chips with speculative decoding, the setup achieved the 700-token speed, beating the Nvidia baseline. Even without that acceleration technique, the new kernels deliver roughly 1.4 to 2 times the throughput of the GB200 at small batch sizes.

Looking ahead, the SGLang-TorchTPU ecosystem plans to expand its model support, starting with DeepSeek V4.1 Flash and Reflection Beam. The teams aim to provide support for new open models on release day, a goal they hope to achieve by the first quarter of 2027. More details are expected at the SGLang Summit 2026, scheduled for November 12–13 in San Francisco. For now, the fact that Meta is running Google's chips in production suggests that the era of single-vendor dominance in AI infrastructure may be coming to an end.