Skip to main content

NVIDIA's New Nemotron Embed Model Tops Retrieval Benchmark

NVIDIA just dropped a new family of embedding models called Nemotron-3-Embed, and the biggest one is already making waves. The 8B version snagged the top spot on the RTEB retrieval benchmark, a widely used test for search and retrieval tasks.

These models are built for real-world applications like RAG (retrieval-augmented generation), smart question answering, code search, and even agent memory. Think of them as the brain behind finding the right information fast.

Three Flavors to Choose From

NVIDIA released three versions:

  • Nemotron-3-Embed-8B-BF16 – the heavy lifter, focused on precision.
  • Nemotron-3-Embed-1B-BF16 – a lighter, faster option.
  • Nemotron-3-Embed-1B-NVFP4 – a 4-bit version optimized for NVIDIA's Blackwell architecture.

All three are Transformer encoders with bidirectional attention masking. They handle up to 32,768 tokens per input and support 34 languages. The models are open-sourced under the OpenMDW-1.1 license, so developers can dig in.

How They Performed

On the RTEB benchmark's 16 public tasks, the 8B-BF16 model scored an average NDCG@10 of 78.46 – that's first place. The 1B-BF16 version scored 72.38, a solid 10.4 points better than the previous llama-nemotron-embed-vl-1b-v2 baseline. And the NVFP4 version? It lost only 0.38 points compared to its BF16 sibling, keeping 99.5% of the precision while doubling throughput on Blackwell hardware.

Clever Model Building

The 1B model wasn't just trained from scratch. The team used NVIDIA ModelOpt's neural architecture search to prune a 3B base model down to 2B, then distilled knowledge from the fine-tuned 8B teacher model using cosine distance and mean square error losses. They iterated until they hit 1.14B parameters. For the NVFP4 version, they added quantization-aware distillation, calibrating with 512 samples and training with 20,000 more to restore precision in long inputs.

Image

Deployment and Compatibility

Deployment varies by version. The 8B and 1B BF16 models work with Transformers and Sentence Transformers frameworks. The 1B-NVFP4 only supports vLLM 0.25.0's /v2/embed interface. Hardware-wise, the NVFP4 version runs on Ampere, Hopper, Lovelace, and Blackwell, while BF16 versions target Ampere, Hopper, and Blackwell. NVIDIA also released an optimized NIM microservice for the 1B model, built with Rust, that matches or beats vLLM performance on GB200 and RTX PRO6000.

Real-World Use Cases

These models shine in several scenarios:

  • Multilingual enterprise search – cross-language retrieval across 34 languages.
  • Code retrieval – trained on code datasets like SWE-bench.
  • Agent memory – the 32K token context lets agents handle longer conversation summaries.

For cost-sensitive setups, NVIDIA suggests a hierarchical RAG strategy: use the 1B-NVFP4 for high-capacity recall and the 8B for tough queries.

Image

Getting Started

NVIDIA provides complete code examples for local inference with Sentence Transformers and server-side deployment with vLLM. Queries and documents are distinguished by query: and passage: prefixes. After L2 normalization, the dot product of the embedding vectors equals cosine similarity – simple and effective.

Key Points

  • Top benchmark performance: 8B version ranks first on RTEB with NDCG@10 of 78.46.
  • Three open-source models: 8B BF16, 1B BF16, and 1B NVFP4 for different needs.
  • 34 languages, 32K context: Supports multilingual search and long documents.
  • Efficient compression: 1B model built via pruning and knowledge distillation.
  • Blackwell optimization: NVFP4 retains 99.5% precision with 2x throughput.