Skip to main content

Acrab's GΞLIX 1 Chip Lets You Run 100B AI Models Locally

Acrab's GΞLIX 1: Running 100-Billion-Parameter AI Models on the Edge

A startup called Acrab has officially launched its first edge AI system-on-chip, the GΞLIX 1. Built on a 5nm process, this chip is designed to deliver a compelling metric: enabling large language models with up to 100 billion parameters to run locally, without sending data to the cloud.

Hardware: A Heterogeneous Powerhouse

The GΞLIX 1 uses a heterogeneous computing architecture, integrating a 20-core Arm CPU, a 3 TFLOPS GPU, a dedicated NPU, and audio/video processing engines into a single chip. The result is a total AI computing power of 650 TOPS. To put that in perspective, Qualcomm's Snapdragon X Elite NPU delivers about 45 TOPS, and Intel's Core Ultra series NPU sits around 40 TOPS. GΞLIX 1's AI performance is roughly ten times that of these competitors.

To satisfy the voracious memory demands of large model inference, the cache and memory subsystems are specifically designed for high throughput. The chip features an 8 MB L1 cache, shares a 768 GB/s L2 cache bandwidth, and connects to a 256-bit wide LPDDR5X memory running at 8533 MT/s. During large model inference, weight reading and KV cache access are highly sensitive to memory bandwidth, and this configuration provides the foundation for running a 100B model on the edge.

Performance: Focus on Prefilling

Acrab showcased performance during the prefilling phase—the process where the model fully encodes the entire input after receiving user input but before starting to generate output token by token. Prefilling speed directly determines the first token delay in long text input. Acrab accelerates the Transformer attention mechanism using vector hardware units and makes targeted optimizations for the prefilling path. In official tests, under the Gemma 26B A4B configuration with a 40k KV cache and 10k token input, the chip achieved 1416.8 tokens per second, which is 7.5 times faster than Apple's M4 Pro Mac Mini.

Software and Ecosystem

On the software side, Acrab released a complete software stack and an agent reference design, covering everything from underlying drivers to the upper application framework. The company also unveiled its first-party terminal device, Agent Box, which serves as a reference implementation of GΞLIX 1 for developers to evaluate and prototype.

Key Points

  • Chip: GΞLIX 1, 5nm process, 650 TOPS AI computing power
  • Architecture: Heterogeneous with 20-core Arm CPU, 3 TFLOPS GPU, dedicated NPU
  • Memory: 8 MB L1 cache, 768 GB/s L2 bandwidth, 256-bit LPDDR5X at 8533 MT/s
  • Performance: 1416.8 tokens/s in prefilling (Gemma 26B A4B), 7.5x faster than Apple M4 Pro
  • Software: Full stack including drivers, frameworks, and Agent Box reference design