Chip Wars Go Confidential: Google’s New TPUs Take On Nvidia’s Memory Wall

Google unveiled two artificial intelligence computing systems on Monday, introducing the TPU 8tx and TPU 8i Max as additions to its eighth-generation Tensor Processing Unit lineup and giving businesses four configurations for training and running AI models. Google says it serves nearly 100 TPU customers, ranging from AI labs to universities. Unlike Nvidia’s widely used graphics processing units, TPUs are purpose-built AI accelerators developed by Google for its own services and increasingly offered to outside companies, including on-premises customers.

The TPU 8tx focuses on protecting sensitive information while AI models use it. Although data is encrypted in storage and during transmission, processing generally requires decrypting it in memory. Google says the TPU 8tx uses hardware-level confidential computing to isolate customer data and proprietary model parameters even during operation. The system combines TPU 8t silicon with fifth-generation AMD EPYC processors and Google’s low-latency Boardfly network, and Google claims it can cut latency by up to half versus TPU 8t with almost no performance penalty from the added protection.

OpenAI plans to buy AI chips from Google, according to a person familiar with the matter who spoke with The Information. The reported purchase would loosen OpenAI’s dependence on Nvidia, which supplies most of the hardware behind the AI boom. Neither company has confirmed the plan, and the price, number of chips and delivery dates have not been disclosed. OpenAI is one of Nvidia’s biggest customers.

The TPU 8i Max addresses what Google calls the “memory wall,” a constraint for autonomous agents that need rapid access to data, codebases, documents and hours of video. Google combines two TPU 8i chips in one package, doubling computing power, on-chip SRAM and high-bandwidth memory throughput versus TPU 8i while packing twice as much silicon into the same rack footprint. “Our fourth system, TPU 8i Max, is built for a different problem: the memory wall,” Google chief technologist and senior vice president of AI and infrastructure Amin Vahdat wrote.

Google is also open-sourcing TPU Sync, software designed to move AI working memory between chips, servers and storage. The two new systems expand Google’s effort to offer AI infrastructure with greater data protection and capacity for memory-intensive workloads as computing shifts toward agent scale.