Skip to main content

Xiaomi's CocktailASR-1: AI that picks out one voice in a crowd

Xiaomi's CocktailASR-1: AI that picks out one voice in a crowd

Picture this: you're at a bustling party, and somehow, you can focus on just one person's voice amid the laughter and clinking glasses. That's the classic "cocktail party problem"—easy for humans, but a nightmare for traditional speech recognition systems. Xiaomi just took a big step toward solving it.

The company has open-sourced CocktailASR-1, an industrial-grade target speaker speech recognition model. Unlike conventional ASR that tries to transcribe everything, CocktailASR-1 zeroes in on a specific speaker using a reference voiceprint. You feed it a short audio sample of the person you want to hear, and it picks out their words from a multi-speaker mix—filtering out other voices, echoes, and background noise.

Image

How it works

Under the hood, CocktailASR-1 uses an end-to-end large language model architecture. It chains together a D2V2 audio encoder, an adapter, and a large model decoder, all bundled into a single checkpoint. To use it, you simply concatenate the reference audio with the target audio, separated by one second of silence. This design makes it versatile: it handles both single-speaker and complex multi-speaker scenarios without switching models.

Performance that speaks for itself

The numbers are impressive. In simulated multi-speaker tests, CocktailASR-1 achieved word error rates (WER) of just 4.11% on LibriMix2mix and 2.90% on LibriSpeechMix2mix. Compare that to other well-known models: on LibriMix3mix, some competitors had WERs between 76% and 121%, while CocktailASR-1 hit only 12.29%. That's not an improvement—it's a different league.

Real-world meeting recordings? The model also shines on AMI SDM and AliMeeting Far, outperforming existing solutions. And in single-speaker settings, it beats rivals on LibriSpeech, WenetSpeech, and CommonVoice-zh.

Two features worth highlighting

Beyond accuracy, CocktailASR-1 brings two practical perks. First, it's great at rejecting negative samples. If the reference audio belongs to someone not in the current conversation, the model outputs empty text—no false triggers for smart speakers or in-car assistants. Second, it supports chain-of-thought reasoning: it can show you how it identified the speaker before giving the final answer. This doesn't boost accuracy much, but it makes the system easier to interpret and debug.

Open source and ready to use

CocktailASR-1 is now available on GitHub under the Apache 2.0 license. The dependencies are minimal—just torch, torchaudio, transformers, and soundfile—so you can load it from HuggingFace without hassle. Xiaomi's move signals a shift in speech tech: from capturing all sounds to locking onto one.

Key Points

  • Xiaomi open-sources CocktailASR-1, a target speaker speech recognition model.
  • It uses a reference voiceprint to isolate a specific speaker from mixed audio.
  • Achieves state-of-the-art WER on multiple benchmarks, far ahead of competitors.
  • Features negative sample rejection and chain-of-thought reasoning.
  • Available on GitHub under Apache 2.0 with simple dependencies.