Skip to main content

NVIDIA's Nemotron-3 Embed Series Tops Retrieval Benchmark

NVIDIA has just dropped its new Nemotron-3 Embed series of embedding models, and the 8B version is already making waves by topping the RTEB retrieval benchmark. These models are built for real-world tasks like RAG (retrieval-augmented generation), smart question answering, code retrieval, and even agent memory.

The series comes in three flavors: the Nemotron-3-Embed-8B-BF16 for maximum precision, the Nemotron-3-Embed-1B-BF16 as a lightweight option, and the Nemotron-3-Embed-1B-NVFP4—a 4-bit version tailor-made for NVIDIA's Blackwell architecture. All three use a Transformer encoder with bidirectional attention masking, handle up to 32,768 tokens, and support 34 languages. They're open-sourced under the OpenMDW-1.1 license. The 8B model is based on Mistral's Ministral-3-8B-Instruct-2512, while the 1B variants stem from Ministral-3-3B-Instruct-2512.

Image

On the RTEB benchmark, which includes 16 public tasks, the 8B-BF16 version scored an average NDCG@10 of 78.46—enough to claim the top spot. The 1B-BF16 version hit 72.38, a solid 10.4-point improvement over the previous llama-nemotron-embed-vl-1b-v2 baseline. As for the 1B-NVFP4, it only lost 0.38 points compared to its BF16 sibling, meaning it retains 99.5% of the precision while doubling throughput on Blackwell hardware.

How did NVIDIA squeeze a 1B model from a 3B base? They used compression rather than training from scratch. The team first pruned the 3B model down to 2B using neural architecture search, then distilled knowledge from the fine-tuned 8B teacher model using cosine distance and mean squared error losses, iterating down to 1.14B parameters. For the NVFP4 version, they added quantization-aware distillation with 512 calibration samples and 20,000 training samples to restore precision, especially for long inputs.

Image

Deployment-wise, the 8B and 1B BF16 versions work with Transformers and Sentence Transformers, while the 1B-NVFP4 only supports vLLM's embed interface. The NVFP4 model runs on Ampere, Hopper, Lovelace, and Blackwell, whereas the BF16 versions target Ampere, Hopper, and Blackwell. NVIDIA also released a NIM microservice for the 1B model, built with Rust, that matches or beats vLLM performance on GB200 and RTX PRO6000.

What can you do with these models? They're great for multilingual enterprise search, code retrieval (trained on datasets like SWE-bench), and agent memory (thanks to that 32K token context). For cost-sensitive setups, NVIDIA suggests a hierarchical RAG strategy: use the 1B-NVFP4 for high-capacity recall and the 8B for tough queries.

NVIDIA also provides code examples for local inference with Sentence Transformers and server-side deployment with vLLM. Just prefix queries with query: and documents with passage:, and after L2 normalization, the dot product of the embeddings equals cosine similarity.

Key Points

  • Top benchmark performance: 8B model leads RTEB with 78.46 NDCG@10.
  • Three versions: 8B BF16, 1B BF16, and 1B NVFP4 (4-bit for Blackwell).
  • Compression technique: 1B model derived via pruning and distillation from 3B base.
  • Broad language support: 34 languages, 32K token context.
  • Practical applications: RAG, code retrieval, agent memory, multilingual search.