Skip to main content

Alibaba's RISC-V Chip Runs 27B Model Without GPU

In a significant leap for chip technology, Alibaba's DAMO Academy has announced that its Xuantie C950 processor successfully ran the Tongyi Qianwen 3.8-27B large language model natively. This marks the first time a RISC-V architecture chip has directly driven a model with 27 billion parameters, bypassing the need for dedicated graphics cards or simulation layers.

A Technical Marvel

The Xuantie C950 is a 64-bit server-grade processor built on TSMC's 5nm process. It packs up to 64 computing cores, interconnected via a high-speed AMBA CHI bus, with 8 cores per group. The chip can hit a maximum clock speed of 3.2GHz. Its hardware design includes an 8-instruction decoding width and a 16-stage pipeline, which boosts memory bandwidth by over four times compared to its predecessor, the Xuantie C920. It also integrates matrix and vector acceleration engines optimized for AI inference.

In benchmark tests, the C950's single-core performance exceeded 70 points in SPECint2006, setting a new record for RISC-V CPUs worldwide.

Performance That Speaks Volumes

When running the Tongyi Qianwen 3.8-27B model, the C950 achieves a decoding speed of 30 tokens per second, with the first token latency kept under 1.9 seconds. Traditionally, such a large model would require 16GB to 24GB of dedicated GPU memory. But the C950 handles the entire inference process using just the CPU and its on-chip acceleration engine—no external graphics card needed.

Why This Matters

RISC-V's open-source nature offers a strategic advantage. Unlike x86 or ARM architectures, which come with hefty licensing fees, Alibaba has full control over the Tongyi model family, its cloud infrastructure, and the Xuantie chip design. This vertical integration means future models can be deeply optimized for the hardware, and subsequent chips can be customized based on real-world data. It opens up new possibilities for diversifying AI infrastructure, making it more accessible and adaptable.

Key Points

  • Breakthrough: Alibaba's Xuantie C950 runs a 27B-parameter model natively on RISC-V, no GPU required.
  • Performance: 30 tokens per second decoding, 1.9s first token latency.
  • Hardware: 5nm process, 64 cores, 3.2GHz, with AI acceleration engines.
  • Implications: Could reduce reliance on expensive GPU infrastructure and foster innovation in AI hardware.