Skip to main content

RISC-V Gets Its Own Offline AI: deepin 25 Runs Large Models on SpacemiT K3

RISC-V Just Got a Taste of On-Device AI

Imagine a desktop computer that can run a large AI model without an internet connection, all on a chip that isn't made by Intel or AMD. That's no longer a distant dream. deepin 25, a domestic operating system, has just completed a deep adaptation with the SpacemiT K3, a next-generation RISC-V chip from Jindie Space. The result? For the first time, a RISC-V desktop platform can handle offline large model inference right out of the box.

What Makes the SpacemiT K3 Tick

The K3 isn't your average chip. It's built for the AI era, using RISC-V heterogeneous fusion computing. Inside, you'll find three types of cores working in harmony: 8 X100 general-purpose cores running up to 2.4 GHz for everyday tasks, 8 A100 AI cores with a custom matrix extension (IME) that churn out about 60 TOPS of AI power, and 2 RT24 real-time cores for system management. There's also a 3D GPU and a video encoding engine. With typical power consumption between 15 and 25 watts, it's efficient and versatile.

When running a 30-billion-parameter model, the K3 can maintain over 10 tokens per second—smooth enough for interactive use.

deepin 25's Secret Sauce

To make the most of this hardware, deepin 25's developers didn't just tweak a few drivers. They introduced a Next repository that fully enables RVA23 optimizations for the riscv64 architecture. The core toolchain now natively supports GCC 15/16, Binutils 2.46, Glibc 2.42, LLVM 21/22, and OpenSSL 3.5 with assembly-level optimizations. This higher baseline runtime environment sets the stage for tapping into the chip's full instruction potential.

But the real magic is in the AI stack. The built-in "Xiao Wu" assistant, through deepin-modelhub, automatically picks the best operator for the chip's architecture. And the popular GGML library, used by llama.cpp, now supports SpacemiT's instruction sets, accelerating vector and matrix computations. When you run a local model, the system intelligently dispatches tasks to the 8 A100 cores.

What This Means for You

If you're a developer or open-source enthusiast, the latest deepin 25 RISC-V image for K3 is already available in the deepin-ports image list. You can download and flash it through official channels. This breakthrough isn't just about running models offline—it's a big step toward a thriving domestic open-source hardware and software ecosystem. As the Next repository matures, RISC-V desktops could become serious contenders in the AI race.

Key Points

  • First offline large model inference on RISC-V desktop achieved by deepin 25 with SpacemiT K3.
  • SpacemiT K3 features 8 X100 general cores, 8 A100 AI cores (60 TOPS), and 2 RT24 real-time cores.
  • deepin 25's Next repository brings RVA23 optimizations and updated toolchain (GCC 15/16, LLVM 21/22, etc.).
  • Local AI stack integrates Xiao Wu and GGML, enabling efficient inference on A100 cores.
  • Image available now for developers to download and test.