Google and Amazon Target CUDA Lock-In with Automated PyTorch Migration Tools

Google and Amazon are releasing open source tools like TorchTPU and TorchNeuron to help developers port PyTorch apps to alternative AI chips without rewriting code.

Chips & Climate Desk
Updated 2d ago 2 min read
Google and Amazon Target CUDA Lock-In with Automated PyTorch Migration Tools

According to TechCrunch, Google announced that TorchTPU, its version of PyTorch, is now available as an open source project. With this release, Google joins Amazon in trying to break Nvidia's stranglehold on the artificial intelligence industry, where most applications are written by default to run on Nvidia graphics processing units.

PyTorch is a popular open source toolkit used by developers and startups to build, train, and run AI models. According to TechCrunch, PyTorch is used by 85 percent of AI applications. The framework was originally developed by Meta in 2016, and in 2022, Meta turned PyTorch over to the Linux Foundation, a nonprofit managing open source projects.

Google's TorchTPU allows applications to run on Google's Tensor Processing Units and makes it easy to move existing PyTorch apps to TPUs and Google Cloud. According to TechCrunch, developers can accomplish this with a simple configuration change and without rewriting their applications.

Amazon made a similar announcement by releasing TorchNeuron, its own version of PyTorch. According to TechCrunch, TorchNeuron allows applications to be ported to Amazon's Trainium chip and AWS with no rewriting. Amazon is also submitting code to the main PyTorch project to decouple various features from Nvidia chips, making it more open to any hardware by default.

Google collaborated directly with Meta, Hugging Face, vLLM maker Inferact, SGLang maker RadixArk, and others to ensure that TorchTPU features and code libraries work well with popular AI development tools. According to TechCrunch, the search giant also launched a dedicated TPU agent suite to help developers move apps to Google chips and cloud while fixing errors.

In addition, Google updated its LiteRT tool to work better with PyTorch AI apps on phones, laptops, and embedded devices. According to TechCrunch, these developments from Amazon and Google do not mean developers will automatically abandon Nvidia hardware, but both companies are working to end Nvidia's advantage.

Sources

  1. 01 Google has joined Amazon in attacking Nvidia’s stronghold with AI app developers — TechCrunch