Skip to main content

Microsoft's $120K AI Gamble: Copilot Runtime Goes Rust, 15x Faster

Microsoft's $120K AI Gamble: Copilot Runtime Goes Rust, 15x Faster

What if you could hand a massive codebase to an AI and come back weeks later to a faster, leaner version? Microsoft just did exactly that—and the results are eye-opening.

Microsoft used AI agents to port the GitHub Copilot runtime from TypeScript to Rust. The cost? About $120,000 in tokens and three weeks of a developer's time. The payoff? A 15.9x speedup and a 90% reduction in memory usage.

The Runtime You Didn't Know You Rely On

The Copilot runtime isn't just some backend service. It powers GitHub Copilot CLI, Copilot App, SDK, and Copilot cloud Agent. It's also embedded in VS Code, Visual Studio, Excel, Outlook, PowerPoint, and a bunch of Microsoft cloud services. Originally built with TypeScript and Node.js, it was fast to develop but started hitting walls as it scaled—startup speed and server density became bottlenecks.

From 7.55 to 120 per Second

Microsoft Distinguished Engineer Stephen Toub documented the whole journey. AI agents turned 430,000 lines of TypeScript into 800,000 lines of production-grade Rust, spanning 135 releases over 14.5 weeks. On average, the migration opened 1.3 pull requests per day. Toub reckons doing it manually would have taken years and millions of dollars.

The performance numbers are stark. In one benchmark, the TypeScript version handled 7.55 single-turn session lifecycles per second. The Rust version? 120 per second—a 15.9x speedup. Memory tells an even bigger story: for 10 clients, TypeScript consumed 1,383MB, while Rust used just 126MB. The Rust version completes everything in-process, eliminating the background subprocess that TypeScript needed for completions.

AI That Researches, Not Just Spits Code

Here's the twist: the AI agents spent far more time gathering information than writing code. Toub observed, "The public image of AI wildly spewing code is almost the opposite; on this scale, the work looks more like iterative investigation—checking the current state, forming hypotheses, making targeted modifications, and looping."

The trickiest file was session.ts, a 30,000-line monster touching every part of the runtime. Migrating it took 25 hours, including 56 minutes of reading documentation and 122 tool calls to clarify requirements. Then the agent spawned 15 sub-sessions, each with its own worktree, communicating with each other. A built-in orchestration skill found overlapping tasks and coordinated collaboration.

The Compiler Isn't a Safety Net

But this migration also exposed the pitfalls. Dozens of regressions appeared—things that worked before suddenly broke. Toub summed it up: "If passing the compiler means correctness, then that statement is only fit for a joke." The compiler doesn't know if function order is right or if task costs are acceptable. At RustConf, consultant Lisa Crossman warned against treating the compiler as an oracle: "Rust prevents agents from writing unsafe code, but it doesn't stop them from writing correct yet wrong programs."

A similar story unfolded at Bun, where author Jarred Sumner used Claude agents to move Anthropic's JS runtime from 535,000 lines of Zig to Rust. That cost $165,000 in tokens and passed 99.8% of existing tests on Linux x64 glibc. Yet Zig creator Andrew Kelley still called the code "garbage no one has reviewed."

What This Means for Engineering

"AI-driven large-scale rewriting" is becoming a verifiable reality: cheap, fast, but with a catch. Acceptance criteria must go beyond "compilation success" to ensure semantic correctness. The tools are powerful, but human judgment remains irreplaceable.

Key Points:

  • Microsoft spent $120K in tokens and 3 weeks to rewrite Copilot runtime from TypeScript to Rust.
  • Result: 15.9x faster and 90% less memory.
  • AI agents spent more time researching than coding, with iterative investigation.
  • Regressions and semantic errors require human oversight beyond compiler checks.
  • Similar AI-driven rewrites (e.g., Bun) show promise but face skepticism about code quality.