Skip to main content

OceanBase Data Agent Beats GPT, Claude on Global Benchmark

OceanBase Data Agent Beats GPT, Claude on Global Benchmark

OceanBase's Data Agent just did something no other submission has managed: it broke the 90% accuracy barrier on the international Data Agent Benchmark (DAB). The team's solution, internally called Scout, scored 90.62%, landing it at the top of the leaderboard—ahead of multiple entries built on overseas models like GPT, Claude Opus, and Claude Fable.

But here's the kicker: OceanBase pulled this off using a domestic database and the domestic large model GLM-5.2. That combination proved it can handle complex data analysis tasks just as well as—if not better than—the big international names.

Image

What Exactly Is DAB?

DAB was launched by UC Berkeley EPIC Data Lab and Hasura PromptQL. It's not your typical text-to-SQL test. Instead of just checking whether AI can turn a question into a SQL query, DAB throws AI into a messy, real-world data environment. The agent has to figure out what data matters, plan an analysis path, run queries, crunch numbers, and then double-check its own work.

The benchmark covers a wide range of fields—internet/local life, finance, biomedicine, intellectual property, corporate operations, government, media/entertainment—and works with databases like PostgreSQL, MongoDB, SQLite, and DuckDB. In other words, it tests the full stack: model + agent + data system. If any piece falls short, the whole thing crumbles.

How OceanBase Did It

So how did OceanBase's Scout submission pull ahead? The team built a closed-loop system that mirrors how a human analyst would tackle a tough problem.

First, it uses a tool called DataLens to build a data profile—identifying fields and relationships. Then, based on the task's complexity, it plans an execution path, selects and filters data, and runs calculations. Finally, it verifies the results through evidence tracking and answer validation. If something looks off, it adjusts and re-verifies. That's the "data understanding – planning – verification – repair" cycle.

This matters because it's not just about raw model power. It's about whether the entire system can work together to deliver reliable answers. And clearly, it can.

Why This Win Matters

For OceanBase, this is more than a benchmark trophy. It's a validation of a bigger shift: databases are evolving from simple storage engines into AI data platforms.

Think about it. In the past, databases stored, queried, and processed data. But AI agents are becoming new "users" of data. They don't just need access—they need to understand, associate, and analyze data, then verify whether the results make sense. The data infrastructure of the AI era is moving from "delivering data to AI" to "helping AI use data effectively."

That's exactly where OceanBase DataPilot comes in. The Scout submission's capabilities will be folded into this product, which aims to handle data understanding, task planning, analysis execution, and result verification. The goal? Let AI move from merely calling data to actually using it to complete tasks.

Key Points

  • OceanBase's Data Agent scored 90.62% on DAB, becoming the first submission to exceed 90% accuracy.
  • It used domestic tech: OceanBase database + GLM-5.2 model, beating solutions built on GPT, Claude Opus, and Claude Fable.
  • DAB tests real-world data skills, not just text-to-SQL—covering multiple industries and database types.
  • The system's closed-loop approach (understand, plan, verify, repair) proved crucial for complex tasks.
  • This signals a shift: databases are becoming AI data platforms, helping AI truly "use" data, not just query it.