Google Splits AI Chips into Training and Inference Lines to Challenge Nvidia

Google has split its custom AI chip lineup into two specialized processors for the first time, unveiling the TPU 8t for model training and TPU 8i for inference workloads in a direct challenge to Nvidia's dominance in AI hardware.

The announcement, detailed in a CNBC report citing Google senior vice president Amin Vahdat's blog post, marks a strategic pivot as the company separates tasks previously handled by unified processors. "With the rise of AI agents, we determined the community would benefit from chips individually specialized to the needs of training and serving," Vahdat wrote, emphasizing the shift toward dedicated hardware for increasingly complex AI workflows.

The TPU 8t delivers 2.8 times the training performance of Google's seventh-generation Ironwood TPU at the same price point, while the TPU 8i achieves 80% better inference performance. Both chips leverage 384 megabytes of SRAM—triple Ironwood's capacity—to reduce latency in data transfers, a critical factor for real-time AI interactions. The architecture aims to "deliver the massive throughput and low latency needed to concurrently run millions of agents cost-effectively," as Alphabet CEO Sundar Pichai noted in a separate blog post.

Unlike previous iterations, the new chips reflect a broader industry trend: Amazon and Microsoft have also begun developing specialized AI silicon, though Google's move is the first to explicitly split training and inference into distinct product lines. Business Insider observed that this shift aligns with the "agentic era," where AI systems must handle multi-step reasoning rather than simple query responses.

Google's strategy hinges on efficiency gains but doesn't signal a full break from Nvidia. The company remains a major customer, with plans to offer Nvidia's upcoming Vera Rubin GPUs in its cloud later this year. "This is not a frontal assault on Nvidia," a Google Cloud executive told reporters, noting the chips will complement rather than replace existing infrastructure. The move echoes Amazon's approach with its Trainium and Inferentia chips, though Google's focus on inference—where latency directly impacts user experience—adds a new layer to the competition.

For developers, the physical reality of adopting these chips means fewer "wait states" during model deployment. Where once AI agents might stall for seconds while data shuffled between memory layers, the TPU 8i's SRAM-heavy design promises near-instantaneous responses—a difference felt in the click of a button that now feels instantaneous rather than delayed.

Analysts estimate Google's TPU business, combined with DeepMind, could reach $900 billion in value. Yet the path to overtaking Nvidia remains steep: the chipmaker still commands 92% of the data center GPU market, per IoT Analytics. As one analyst quipped, "Google's TPU could be bad news for Nvidia" (a prediction made in 2016 that didn't pan out).

Adoption is already underway: Anthropic has committed to using multiple gigawatts of Google TPUs, and all 17 U.S. Energy Department labs now run AI workloads on the chips. But the real test lies in whether enterprises will accept the trade-off of cloud lock-in for cost savings—a hurdle Nvidia avoids by keeping its GPUs ubiquitous across cloud providers. Whether these chips will chip away at Nvidia's lead remains uncertain, but Google's specialization bet is a clear signal that the AI hardware arms race is far from over.

Artūras Malašauskas is an AI Systems Integrator with 20+ years of production-grade web engineering experience. He has designed, shipped, and scaled enterprise Python/PHP systems for logistics, SaaS, and public-sector clients. For the past year, he has focused exclusively on AI integrations: deploying open-source LLMs, building generative media pipelines (image, audio, video), and engineering multi-agent workflows for real production environments. His standard: reproducibility, security, cost-efficient inference—no vaporware. He documents and evaluates emerging AI tooling, separating verified capabilities from marketing noise. Technical editor at: muza-ai.eu, ai-verslas.lt, ai-naujinos.lt Connect on LinkedIn