Tenstorrent partners with Japan’s LSTC to create 2nm AI accelerator chips

Tenstorrent, an AI chip company headed by chip legend Jim Keller, has unveiled a key partnership with Japan’s Leading-edge Semiconductor Technology Center (LSTC).

Together, LSTC and Santa Clara, California-based Tenstorrent will construct a cutting-edge two-nanometer AI Accelerator, a new kind of AI hardware solution that promises to make a leap forward in AI performance, the companies said.

The collaboration leverages Tenstorrent’s world-class RISC-V and chiplet intellectual property, the parties said. The strategic alliance entails a multi-tiered partnership between Tenstorrent and LSTC, where Tenstorrent’s expertise in RISC-V central processing unit (CPU) technology and chiplet technology — which fuses together different chips into a single package — will be utilized for LSTC’s edge AI accelerator.

In addition to IP licensing, Tenstorrent will serve as a collaborative innovation partner, co-designing the chip that aims to redefine AI performance standards in Japan.

The collaboration expands further with the involvement of Rapidus Corp., a newly established Japanese semiconductor company. Rapidus is poised to develop state-of-the-art logic semiconductor technologies, focusing on achieving the world’s best cycle time reduction services. Tenstorrent will collaborate with Rapidus on both wafer processing and advanced packaging, aligning with their shared commitment to optimizing the total manufacturing process.

Tenstorrent, known for its high-performing RISC-V CPU technology, will deploy its Ascalon RISC-V CPU core technology to co-develop a RISC-V CPU chiplet for LSTC’s new edge AI accelerator. The strategic alignment of Tenstorrent and LSTC is founded on their shared vision for the future of silicon, emphasizing heterogeneous compute by combining RISC-V CPU and AI cores for seamless handling of diverse workloads.

Wei-Han Lien, chief architect of Tenstorrent’s RISC-V products, said in a statement, “The joint effort by Tenstorrent and LSTC to create a chiplet-based edge AI accelerator represents a groundbreaking venture into the first cross-organizational chiplet development in the semiconductor industry. The pioneering strategy harnesses the collective capabilities of both organizations to meet the increasing needs of AI applications at the edge.”

CEO Jim Keller

Led by renowned chip architect and CEO Jim Keller, Tenstorrent has a rich history of delivering commercially successful silicon products. Keller’s leadership, especially in AI, RISC-V CPU, and heterogeneous compute development, made Tenstorrent the ideal choice for LSTC.

Keller has helped assemble a team of high-performance CPU engineers from Arm, AMD, Intel, Tesla and Apple. Keller has lead teams at each of those companies.

As for the advent of generative AI in the past year or so, Bennett said the goal was always to have an architecture that could do inference and training, could do NLP, and other recommendation engines.

“Jim has been adamant about cost performance, specifically around not using technologies like HBM, and doing more with less memory and doing it closer to the memory.”

Much of Keller’s vision has been about avoiding some of the mistakes that other AI hardware makers are encountering and focusing on building solutions for the AI models of today. AI is changing fast, and so the architecture has to be flexible and it should embrace open standards. And it is betting on the combo of AI processing and RISC-V CPUs.

LSTC support

Tetsuro Higashi, chairman of LSTC, said in a statement, “Tenstorrent is the perfect partner for us in this Post-5G Project. As a next-generation semiconductor design technology, we will promote the development of edge AI accelerators dedicated to edge inference processing applications, including generative AI, through international collaboration.”

Atsuyoshi Koike, CEO of Rapidus, said in a statement, “We will cooperate not only in the front-end process but also in the chiplet (back-end process) and work on as a leading example of our business model that realizes everything from design to back-end process in a shorter period of time ever.”

David Bennett, chief customer officer of Tenstorrent, said in a statement, “It is with great pleasure that I also announce that Tenstorrent will be opening a high-performance compute design center in Japan to support not only this project and our customers, but also to help nurture and develop the future of Japan’s high-performance compute industry.”

This collaboration signifies a crucial step in Japan’s bid to reestablish leadership in high-performance compute design. With Tenstorrent’s expertise, LSTC’s vision, and Rapidus Corporation’s commitment, the venture is poised to shape the future of AI in Japan.

A different model

A few things help differentiate Tenstorrent, Bennett said. The company tries to do all of the necessary AI processing from inference to training to vision to language to multimodal itself. It uses open-source technologies. It uses a bottoms-up approach to its open metal stack. So openness is its first differentiator.

“What we’re hearing from customers is they want open standards. They want RISC-V, the ability to customize for workloads that they want to drive is important to them,” Bennett said. “They want AI that they can own and in particular AI hardware that’s open. An alternative to Nvidia.”

The second part is a belief in the combination of compute and acceleration. It uses a high-performance RISC-V CPU together with AI. And the company is figuring out a business model, where the IP goes into the AI chips, with chiplets mixed in to give some flexibility. Tenstorrent monetizes each step of the way.

“We’ve got to make the IP, we got to make the chips, we got to put them in systems, we got to sell the software on top of it,” he said. “So we’ve monetized every step of the way. And what that enables us to do is offer our technology to customers tech any way they want it.”

A market alternative

Primarily, the company is an Nvidia competitor. It makes PCI boards, or accelerator boards, and they go into workstations, servers, and a Galaxy system, which is an ultra-dense 32-chip rack. These are accelerators designed to run inference training — NLP or its language models, across the board.

Bennett said, “That’s the fundamental goal of the company. What sets us apart is a couple of things. Primarily, we have a very strong belief that the future of computers is a combination of high performance compute and AI. So acceleration, and the balance between them. It’s not dissimilar to what you know Nvidia is doing with Grace Hopper and those kinds of things,” Bennett said.

But TensTorrent’s designs are very different from GPUs. The chips are designed from the ground up to be better at AI. Inside its AI tech, Tenstorrent uses RISC-V compute cores, which allows for a layer of compute on top of traditional parallel processors.

“We can do some really interesting things that are important to AI,” Bennett said.

“We’re about to announce a big deal with the government where they’re purchasing our RISC-V IP to help power one of the AI accelerators that they’re building. We’ve been able to secure, you know, contracts with customers and bookings, partnerships. I think we’ve done a pretty good job there. And the pipeline is actually a fairly healthy mix of both IP and AI, hardware sales around the world, across the board.”

The company already has partnerships with Hyundai and Samsung. And it announced LG as a customer, which is looking at putting Tenstorrent tech into its TVs. Now it is announcing the deal with Rapidus in Japan.

“They’re purchasing our RISC-V IP. And we’ll be working closely with them to design their next generation AI accelerator,” Bennett said.

Tenstorrent has said that its high-performance RISC-V CPU will be the best in the market, equivalent to AMD’s Zen 5 performance coming out later this year.

The company is at work on its second-generation tech. And it has more than 400 people. It raised $221 million in July 2021. It raised more money since then via strategic investments from Samsung and Hyundai.