Four TPU systems in one year. Google had never launched that many, and on November 9, 2026 it rounded out its eighth generation with two more: TPU 8tx, built for confidential inference, and TPU 8i Max, which puts two TPU 8i chips on a single package. The same day it open-sourced TPU Sync, a library that in Google's tests cuts KV cache transfer between nodes from several seconds to about 230 ms.
What was announced
In April, at Google Cloud Next, came TPU 8t (large-scale training) and TPU 8i (real-time reasoning), with up to 2.7x better training performance and 80% better inference price-performance, respectively, over the previous generation. The two new systems don't change the silicon, they change the configuration:
| System | Built for | What's different |
|---|---|---|
| TPU 8t | Training foundation models | Generation baseline |
| TPU 8i | Inference and reasoning | Boardfly network, up to half the inter-chip delay |
| TPU 8tx | Confidential inference and small-scale training | 8t silicon with 5th Gen AMD EPYC hosts, hardware isolation and Boardfly: up to 2x lower latency than 8t |
| TPU 8i Max | Very long contexts | Two 8i chips per package: double the compute, SRAM and HBM bandwidth; twice the silicon in the same rack |
The 8tx targets a specific problem: fine-tuning on private data, distilling or running reinforcement learning without data or weights leaving the protected environment, and without shuffling them between clusters. The 8i Max goes after the "memory wall": when serving huge context windows, the limit isn't compute but how many bytes per second you can pull out of HBM.
Google hasn't published FLOPS, HBM capacity, power, pricing or Regions for either system, so for now there is no way to compare them in absolute terms with other accelerators.
TPU Sync, the part you can use today
The hard numbers are in TPU Sync, a C++ library that moves tensors between TPU HBM, host DRAM, shared storage and the data center network using asynchronous DMA that overlaps with compute. It tackles three problems anyone serving LLMs runs into: tiered KV caching, disaggregated prefill/decode serving and prefix caching. It plugs into vLLM and SGLang (through TorchTPU), llm-d 1.0 on GKE and JAX, and ships as the tpu-sync-torch and tpu-sync-jax wheels.
The figures come from Google's internal tests in July 2026 on TPU v7x (Ironwood), not on the eighth generation:
- Prefill-to-decode KV cache transfer: median down from several seconds to about 230 ms on Qwen3 235B, a cut of more than 90%.
- Time to first token: 60% lower on average for 235B and 90% for 480B at low concurrency; up to 90% lower with prefix caching on 64K-context workloads.
- Up to 223 GB/s between TPU and host on each Ironwood host, against a 256 GB/s PCIe Gen5 ceiling.
Shared storage as a third tier and the RL weight-transfer module are still experimental.
What it means
The bigger picture is that Google no longer sells "the TPU" but a family tuned to each phase: training, reasoning, inference on sensitive data and long-context serving. TPU 8i Max and TPU Sync are two answers to the same problem: one packs more memory bandwidth into each rack, the other moves the KV cache around instead of recomputing it.
If you build on Google Cloud, the practical part today is the software. TorchTPU, the vLLM and SGLang backends and TPU Sync lower the cost of trying TPUs without rewriting PyTorch code, which was the real barrier to moving off GPUs. The two new TPUs matter mostly to teams with regulated data (8tx) or serving models with contexts of hundreds of thousands of tokens (8i Max); until pricing and Regions show up, they are a roadmap.
Sources
- Our newest TPUs: 8tx and 8i Max, built for the agentic era, Google Cloud Blog
- Introducing TPU Sync: Fast KV cache and weight transfer for inference and RL on TPU, Google Cloud Blog