The world of artificial intelligence (AI) has seen a significant shift in recent years, with advancements in hardware and software technologies. Two major players in this arena are AMD and Intel, each vying for dominance in the AI chip market. In this blog post, we’ll delve into the latest developments and compare the performance of AMD’s Ryzen AI 300 series and Intel’s Core Ultra 7 258V in large language model (LLM) tasks.
AMD’s Ryzen AI 300 Series: The New Champion
AMD has been making waves in the AI community with its Ryzen AI 300 series of mobile processors. These chips have been tested against Intel’s mid-range Core Ultra 7 258V in various LLM tasks, and the results are impressive. According to AMD’s in-house testing, the Ryzen AI 9 HX 375 outperforms the Core Ultra 7 258V by up to 27% in token generation speed using LM Studio, a popular desktop app for downloading and hosting LLMs locally[1].
The Ryzen AI 9 HX 375 was tested against several LLM models, including Meta’s Llama 3.2, Microsoft Phi 3.1 4k Mini Instruct 3b, Google’s Gemma 2 9b, and Mistral’s Nemo 2407 12b. The tests measured speed in tokens per second and acceleration in the time it took to generate the first token. The results clearly show that AMD’s chip excels in both speed and time to start outputting text.
It’s worth noting that Intel’s Core Ultra 7 258V is not on a fair playing field against the HX 375. The 258V has a max turbo speed of 4.8 GHz, while the HX 375 reaches 5.1 GHz. This difference in performance is significant and affects the overall outcome of the tests. However, even with this disparity, AMD’s Ryzen AI 9 HX 375 demonstrates superior performance in LLM tasks.
GPU Acceleration: The Key to Faster Performance
One of the key features of AMD’s Ryzen AI 300 series is its ability to utilize GPU acceleration. Tests performed using the Vulkan API in LM Studio showed that the Ryzen AI 9 HX 375 achieved up to 20% faster token generation speeds when GPU acceleration was enabled. This highlights the importance of dedicated NPU and iGPU in AI tasks, especially when it comes to on-demand program-level AI tasks[1].
AMD’s commitment to open innovation is evident in its support for various AI frameworks and libraries. The company continues to advance its ROCm open software stack, which now includes support for critical AI features like FP8 datatype, Flash Attention 3, and Kernel Fusion. This ensures seamless integration with popular generative AI models like Stable Diffusion 3 and Meta Llama 3[4].
Intel’s Struggles in the AI Chip Market
Intel, on the other hand, has faced significant challenges in the AI chip market. Despite its efforts to launch the Gaudi accelerator chips, Intel has fallen short of its modest revenue goal for 2024. This is a stark contrast to AMD’s upgraded sales forecast for its Instinct GPUs, which now stands at $5.5 billion for 2024[3].
Intel’s CEO, Pat Gelsinger, remains bullish on the company’s long-term strategy for Gaudi chips. However, the current performance indicates that Intel is still in the early stages of scaling up sales efforts. The company’s reliance on software ease of use and the transition from Gaudi 2 to Gaudi 3 have contributed to slower-than-expected sales[3].
Conclusion
The battle between AMD and Intel in the AI chip market is heating up. AMD’s Ryzen AI 300 series has demonstrated superior performance in LLM tasks, thanks to its advanced architecture and GPU acceleration capabilities. Intel’s struggles with its Gaudi accelerator chips highlight the challenges of entering a market dominated by Nvidia. As AI continues to evolve, it will be interesting to see how these companies adapt and innovate to stay ahead in the race for AI supremacy.