Skip to main content

Mistral AI's Leanstral 1.5: Making Math Proofs Cheaper and More Accessible

Mistral AI's Leanstral 1.5: Making Math Proofs Cheaper and More Accessible

Mistral AI has just dropped a new open-source model that could change how mathematicians and developers work with formal proofs. Called Leanstral 1.5, it's built specifically for Lean4, a language used to write and verify mathematical proofs. And the numbers are impressive.

What's Under the Hood?

Leanstral 1.5 is a mixture-of-experts model with 119 billion total parameters, but it only activates 6 billion per token. That means it's efficient—both in terms of compute and cost. It's released under the Apache-2.0 license, so anyone can use it, tweak it, or build on top of it.

Performance That Speaks for Itself

On the miniF2F benchmark, which tests formal mathematics, Leanstral 1.5 scored a perfect 100% on both the validation and test sets. That's a clean sweep. But it didn't stop there.

Facing the notoriously tough PutnamBench—a collection of problems from the William Lowell Putnam Mathematical Competition—the model solved 587 out of 672 Lean4 problems. That's an 87% success rate. For context, these are problems that stump even the brightest math students.

In the FATE benchmark series, which covers abstract algebra, Leanstral 1.5 achieved 87% on the master-level FATE-H test and 34% on the doctoral-level FATE-X test. Those are new records for this type of model.

Image

The Cost Advantage: A Game Changer

Here's where things get really interesting. Mistral AI claims that using Leanstral 1.5 to solve a PutnamBench problem costs an average of just $4. Compare that to Seed-Prover1.5, which costs over $300 per problem, or Aleph Prover, which ranges from $54 to $68. That's a 75x to 150x reduction in cost.

This isn't just about saving money—it's about accessibility. High-precision proof assistance has mostly been confined to well-funded labs. With Leanstral 1.5, smaller research groups and even individual mathematicians can afford to use formal verification tools.

More Than Just Math: Finding Bugs in Code

Leanstral 1.5 isn't just for proving theorems. It also shows strong bug-finding capabilities. In tests across 57 code repositories, the model identified 47 violations, 11 of which were confirmed as real defects. Notably, five of those vulnerabilities had never been reported on GitHub before. That's a big deal for software security.

What This Means for the Future

By open-sourcing Leanstral 1.5, Mistral AI is lowering the barriers to formal mathematical research. The model's efficiency and low cost could accelerate the adoption of formal proofs in mathematics and computer science. Researchers can now focus on the creative parts of their work—breaking new ground—while leaving the tedious verification to AI.

Key Points

  • Model: Leanstral 1.5, a 119B-parameter mixture-of-experts model (6B active) for Lean4.
  • Performance: 100% on miniF2F, 87% on PutnamBench, and record scores on FATE benchmarks.
  • Cost: $4 per PutnamBench problem vs. $300+ for competitors.
  • Bug Finding: Identified 47 violations in code repositories, including 5 previously unknown vulnerabilities.
  • License: Apache-2.0, fully open-source.