Tenstorrent has revealed two state-of-the-art inference-focused accelerator cards, the Grayskull E75 and E150, geared to boost PCI Express-based 64-bit x86 systems’ inference performance. As the demand for specialized AI hardware escalates, these devices arrive at a crucial time. Inference processing, vital for AI workflows, is in especial need of efficiency upgrades. The new Tenstorrent cards look to answer this call. Partnered with Tenstorrent’s software, they promise to propel machine learning acceleration to new heights, signaling a leap forward for high-performance AI endeavors. These accelerators illustrate the industry’s broader trend towards optimizing AI systems for both performance and power efficiency, aiming to facilitate a range of applications from data centers to edge devices. As AI continues its integration into various sectors, tools like Grayskull E75 and E150 are set to become fundamental in navigating the complex terrain of machine learning tasks.
Pioneering Hardware for Machine Learning Acceleration
The Grayskull E75 and E150 inference cards represent a quantum leap in the realm of machine learning hardware. The E75 card flaunts an impressive array of 96 Tensix processing cores operating at 1GHz, bundled with 96MB SRAM, combined with 8GB of external LPDDR4 memory. As for the E150 model, it pushes performance boundaries further with 120 Tensix cores clocked at 1.2GHz, expanding the SRAM to 120MB while retaining the 8GB LPDDR4 memory. These cores, based on the versatile RISC-V instruction set architecture, are optimized meticulously for machine learning operations. They feature an elaborate tensor array math unit, a SIMD unit, as well as additional hardware accelerators for network functions and compression/decompression tasks, ensuring a seamless and robust ML acceleration experience.
The differences in the power requirements of the two cards are noteworthy. The Grayskull E75 maintains efficiency with a single six-pin PCIe power connector. Meanwhile, the E150, with its heightened performance capabilities, necessitates an extra six-plus-two pin power connector. Prospective users should note that a compatible 64-bit x86 system, armed with Ubuntu 20.04 LTS, is essential for deployment. Sufficiently ample system memory and storage are also recommended to fully harness the potential of these accelerators, thereby enabling users to overcome the bottlenecks that often hamper machine learning tasks.
Advancing AI With User-Friendly Software
Tenstorrent’s hardware reveal pairs well with their two innovative software stacks: TT-Buda and TT-Metalium. TT-Buda enables swift deployment of common ML models, ideal for developers eager to utilize machine learning without intricate hardware knowledge. On the flip side, TT-Metalium is for those seeking deep performance tuning with its low-level access. Both stacks align with the open-source movement under the Apache 2.0 license, which champions transparency and community-driven innovation in AI.
This openness is expected to spur growth and customization in AI development. Meanwhile, Tenstorrent hints at their upcoming Wormhole accelerators, which hint at heightened network integration for AI hardware—though specifics are not yet revealed. These innovations, including the Grayskull E75 and E150 processors, position Tenstorrent at the forefront of the impending era where advanced AI processing is critical, underscoring their dedication to enhancing AI efficiency.
