Attend QCon Plus online conference (May 10-20) and find practical inspiration from software leaders. Register

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Avdi Grimm describes the future of development, which is already here. Get a tour of a devcontainer, and contrast it with a deployment container.

The panelists reflect on various microservices topics.

Monte Zweben proposes a whole new approach to MLOps that allows to scale models without increasing latency by merging a database, a feature store, and machine learning.

In this article we will be sharing our experience learned from 12 months of adopting certain management and organisational insights from the book Team Topologies. It explores how we identified areas of responsibility and assigned those into mostly customer facing domains which could be given to our teams. It shows how an inverse Conway manoeuvre can be used to improve the architecture.

The panelists discuss the security for the software supply chain and software security risk measurement.

Uncover emerging trends and practices from software leaders. Attend online on May 10-20, 2022.

Learn how cloud architectures achieve cost savings, improve reliability & deliver value. Register Now.

Understand the emerging software trends you should pay attention to. Attend in-person on Oct 24-28, 2022.

InfoQ Homepage News Google Announces General Availability of Cloud TPU VMs

Last year Google introduced Cloud TPU Virtual Machines (VMs), which provide direct access to TPU host machines in preview. Today, Cloud TPU VMs are generally available, including the new TPU Embedding API, which can accelerate ML Based ranking and recommendation workloads.

Google optimized the Cloud TPU VMs for large-scale ranking and recommendation workload with the GA releases. And in a Google Cloud blog post, the company claims that the Embedding API can help businesses lower costs associated with ranking and recommendation use-cases that commonly rely on deeply neural network-based algorithms that can be costly. 

Vaibhav Singh, outbound product manager, Cloud TPU, and Max Sapoznikov, product manager, Cloud TPU at Google, wrote in a Google Cloud blog post:

Embedding APIs can efficiently handle large amounts of data, such as embedding tables, by automatically sharding across hundreds of Cloud TPU chips in a pod, all connected to one another via the custom-built interconnect.

Furthermore, the TPU VMs GA releases support three key frameworks: TensorFlow, PyTorch, and JAX, available through three optimized environments for ease of setup with the respective framework. A respondent Zak on a hacker news thread on the GA release of Cloud TPU VMs stated:

In the previous Cloud TPU architecture, PyTorch and JAX users had to create a separate CPU VM for every remote TPU host and arrange for these CPU hosts to communicate indirectly with the TPU hosts via gRPC. This was cumbersome and made debugging difficult. With TPU VMs, none of this is necessary. Instead, you can SSH directly into each TPU host machine and install arbitrary software on a VM to handle data loading and other tasks with much greater flexibility.

  Source: https://cloud.google.com/blog/products/compute/introducing-cloud-tpu-vms

In addition, the TPU VMs also allow input data pipelines to be run directly on the TPU hosts. Users can use this functionality to create their own customer operations, such as TensorFlow Text, and are no longer restricted to the TensorFlow runtime release version. Moreover, local execution on the host with the accelerator also enables use cases such as distributed reinforcement learning.

The same respondent Zak on the Hacker News thread said:

With TPU VMs, workloads that require lots of CPU-TPU communication can now do that communication locally instead of going over the network, which can improve performance.

Currently, the Cloud TPU VMs available in various regions, and pricing details can be found on the pricing page. And lastly, customers can get more information and guidance through the documentation, concepts, quickstarts, and tutorials.

Becoming an editor for InfoQ was one of the best decisions of my career. It has challenged me and helped me grow in so many ways. We'd love to have more people join our team.

The industry’s only comprehensive Cloud Native Security Platform. Get a free trial now.

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

You need to Register an InfoQ account or Login or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

Real-world technical talks. No product pitches. Practical ideas to inspire you and your team. QCon Plus - May 10-20, Online. QCon Plus brings together the world's most innovative senior software engineers across multiple domains to share their real-world implementation of emerging trends and practices. Find practical inspiration (not product pitches) from software leaders deep in the trenches creating software, scaling architectures and fine-tuning their technical leadership to help you make the right decisions.

InfoQ.com and all content copyright © 2006-2022 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with. Privacy Notice, Terms And Conditions, Cookie Policy