Google Open-Sources TorchTPU for PyTorch Applications on TPUs

Google has released TorchTPU as an open-source project, giving developers a way to run existing PyTorch applications on Google Tensor Processing Units (TPUs). The company says workloads can be moved through a configuration change without rewriting the application, creating a direct path from PyTorch-based development to TPUs and Google Cloud.

The announcement arrives one day after Amazon introduced TorchNeuron, its PyTorch implementation for the company’s Trainium chips and AWS. Both initiatives target a practical obstacle to wider adoption of non-Nvidia AI hardware: many AI applications are built with PyTorch features that were designed around Nvidia GPUs.

PyTorch portability becomes the competitive focus

PyTorch is an open-source toolkit used to build, train and run AI models. It was created by Meta, then Facebook, in 2016, when Nvidia GPUs were already the standard hardware for AI research. Meta transferred the project to the Linux Foundation in 2022. Some estimates put PyTorch use at 85% of AI applications, making compatibility with its ecosystem strategically important for cloud and chip providers.

TorchTPU is designed to reduce the work required to take an existing PyTorch application to Google’s accelerator hardware. That does not mean developers will automatically abandon Nvidia systems, but it removes a migration barrier for teams considering alternative infrastructure.

Google expands the surrounding toolchain

Google said it worked with Meta, Hugging Face, Inferact, the maker of vLLM, and RadixArk, the maker of SGLang, so TorchTPU features and code libraries operate with widely used AI development tools. The company has also introduced a TPU agent suite to help developers move applications to its chips and cloud, including assistance with resolving errors during the process.

The release follows Google Cloud's TPU 8t and TPU 8i chips and extends Google’s TPU strategy from new accelerator hardware to the software layer that shapes where AI applications can run. Google also updated LiteRT, its runtime for AI on phones, laptops and embedded devices, to work better with PyTorch applications.

Amazon is pursuing a parallel route

Amazon’s TorchNeuron similarly enables PyTorch applications to be ported to Trainium and AWS without rewriting them. Amazon also said it is contributing code to the main PyTorch project to separate features from Nvidia-specific hardware, with the aim of making them more hardware-neutral by default.

For businesses planning AI infrastructure, the immediate implication is to examine whether current PyTorch workloads can remain portable across accelerator platforms. Configuration-based migration and compatible tools may broaden deployment options, even though Nvidia’s established position in the PyTorch ecosystem remains significant.