Skip to main content

Running a Software Factory Efficiently at Uber Scale

Uber Engineering (@udaykiran)

Original Article: https://x.com/ubereng/status/2093444169037762840
Post Author: @udaykiran / Uber Engineering

Introduction

AI tools are now embedded in every phase of software development at Uber. More than 70% of pull requests are attributed to local or cloud agents. Engineers have built over 3,600 agent skills across the software development life cycle, and executed more than 30K agent skill executions per day.

At the AI Engineer 2026 conference, we shared our vision for the Software Factory and the building blocks and managed agents we are building across the lifecycle. As we progress on that vision, a growing share of sessions aren’t initiated by humans, but by automated managed agents handling code review, self-healing CI failures, completing E2E PRs with visual validation, triaging on-call alerts, debugging incoming bugs, and handling a variety of code maintenance tasks with human reviews/escalations.

From February to Aug 2026, weekly active users across all agentic offerings across all our employees (engineers & non-engineers) grew 7x, and weekly agentic requests grew 9.4x. Meanwhile, our total AI spend has relatively stabilized since April due to optimizations across the board.

Since adoption, workload mix, and model upgrades are all continuously changing, isolating our own optimization gains means holding one model fixed, since behavior shifts with every upgrade and model family. We did that from February to July: cost per 1,000 model requests is down almost 34% from its peak, and cost per session is down 52% from its June peak.

This blog walks through how we think about our software factory: the four layers agent sessions run in, the cost equation we use to decompose spend, how we measure each term, and how we optimize those terms across every layer.

All pricing and vendor metrics in this comparison are based on publicly available information, with cost efficiency gains driven by routing our internal Uber workloads more intelligently within standard tier-pricing. While specific cost reductions we measure are unique to our environment and your mileage may vary depending on your codebase, team size, and agent workflows, the methodology of benchmarking real work and optimizing for accuracy and cost is universally applicable.

The Software Factory and Its Cost Equation

Four Layers of Agent Usage

We organize AI usage into four layers, from the most specialized to the most general. The higher the layer, the more control we have over cost, quality, and model selection.

  • Layer 4: Managed Background Agents (Autonomous / Event-driven) - uReview (AI PR reviews), CI self-healing, automated bug triage, dependency migrations.
  • Layer 3: Purpose-Built Interactive Workflows - Domain-specific coding workflows, incident remediation assistants.
  • Layer 2: Standard Interactive Agent CLI / IDE - Terminal/editor integration with standard internal tools and skills.
  • Layer 1: Raw Chat / Ad-hoc Prompting - General exploration and ad-hoc queries.

The Cost Equation

Across any of the layers above, we can decompose the cost of an agentic session into the following terms, which we could measure and optimize independently:

\[\text{Total Cost} = \text{Users} \times \frac{\text{Sessions}}{\text{User}} \times \frac{\text{Turns}}{\text{Session}} \times \frac{\text{Requests}}{\text{Turn}} \times \frac{\text{Tokens}}{\text{Request}} \times \frac{\text{Price}}{\text{Token}} \]

  • The first two terms (\(\text{Users} \times \frac{\text{Sessions}}{\text{User}}\)) represent adoption & engagement, which we want to keep growing across our overall user base.
  • The three middle terms (\(\frac{\text{Turns}}{\text{Session}} \times \frac{\text{Requests}}{\text{Turn}} \times \frac{\text{Tokens}}{\text{Request}}\)) provide opportunities for optimization: the work the agent does on its own behalf, on top of the request an engineer actually made. This is where most of our effort goes, including mechanisms that help agents plan faster, reduce unwanted turns or errors, optimize input tokens, and more.
  • The final term (\(\frac{\text{Price}}{\text{Token}}\)) is governed by model selection and provider tier routing.

How We Measure

Below is the full set of metrics we track weekly and monthly that enable us to forecast & plan our efforts short-term and long-term:

  • Active Users & Adoption Rate across engineering and non-engineering teams
  • Sessions per User & Agent Invocations per Session
  • Input, Output, and Reasoning Tokens per Request
  • Cache Hit Rates and TTL Effectiveness
  • Cost per PR, Cost per Session, and Cost per Resolved Incident
  • Quality Metrics (Precision, Recall, F1 for automated reviews, PR Acceptance Rate)

Optimization Levers

1. Optimizing Price / Token

The vendor sets the token price. We pick which model runs which workload. Across all our managed agents’ layers, we pick the model that’s most Pareto efficient for that workload (cost/completed task, output quality, and model reliability).

Benchmark-Driven Model Selection

Model selection happens in four steps, the same for every managed agent we run:

  1. Build a benchmark out of the agent’s real work.
  2. Run the agent on a harness that serves any model, frontier or open-weight, behind one interface.
  3. Move to whatever is Pareto optimal, and keep moving. The frontier shifts every few weeks.
  4. Continually refine workload performance by leveraging aggregated insights from managed agents to test and deploy model routing strategies.

For example, uReview handles AI code review for all pull requests. We built its benchmark from real pull requests with known bugs (graded easy, medium, hard). We score precision, recall, and F1 against those bugs, plus cost per review, latency, timeouts, and noise. Switching models improved our F1 while dramatically reducing cost/PR.

Additionally, our Uber SWE Benchmark runs frontier and open-weight models across thousands of real-world PRs from large monorepos to guide model selection across all SDLC-managed agents.

Default Model Selection

In interactive interfaces, token unit costs remain fixed; however, we strategically manage token distribution across models via two defaults:

  • Initial Session Model
  • Subagent Model Default: This is the most impactful lever. The proportion of sessions initiating subagents has steadily increased. Because subagents perform well-defined tasks with specified inputs that often do not require frontier-level reasoning, we default them to a weaker, more cost-effective model while still allowing manual overrides. The primary model handles task decomposition and evaluation while subagents execute the work.

2. Optimizing Tokens / Request

Every turn re-sends the full conversation history, project context, and tool results. Anything that reduces the per-request payload compounds across the session.

Standardized Defaults

  • Automatic compaction triggered at 400k tokens (even for 1M context window models): Balances model performance against cache bursts and repeated input token costs.
  • Reasoning effort defaulted to Medium: Output tokens (including internal reasoning tokens) are billed at multiples of the rate of input tokens on primary models. Medium reasoning hits an optimal balance between cost and quality for the majority of tasks.

Prompt Caching Strategy

Prompt caching economics: subsequent reads cost only 0.1x the standard input token rate. However, write premiums vary (5-minute cache costs 1.25x; 1-hour cache costs 2x).

  • Interactive sessions: Engineers often leave sessions idle for more than 5 minutes. We transitioned from the default 5-minute TTL to a 1-hour window, preventing cache invalidation and costly full-price context rebuilds.
  • Subagents: Retain a 5-minute cache TTL because their execution is short-lived and task-focused.

Executing MCP Tools via the Shell

Uber routes all Model Context Protocol (MCP) interactions through a unified gateway encompassing 1,000+ internal and third-party SaaS MCP servers.

Standard MCP pre-loads all tool schemas into every session context, adding 50K-70K tokens of schema overhead on every turn (for ~100 tools). To fix this context bloat:

  1. CLI tool resolution: Replaces direct schema injection by allowing the model to execute a shell command. All 1,000+ MCP tools are projected as CLI commands, resolving against the gateway dynamically at call time.
  2. Tool search: Scales to thousands of tools by allowing the model to search the catalog and load only required tools on demand.

Code-Mode

When tools call functions directly as shell commands, models can batch multiple actions within a single script.

  • Under standard MCP, executing a single SQL query requires submitting a request, polling status 2 to 5 times (each taking a full model turn with full context roundtrip), and retrieving output.
  • Code-mode streamlines this into a Python loop executed in a subprocess. Only the final summary returns to the model context.
  • Result: Even for minimal result sets, code-mode reduces token usage by over 50%. For bulk workflows, savings compound to more than 90%. Uber deployed 25+ pre-built code-mode skills for high-frequency MCP servers.

SaaS MCPs

Third-party SaaS MCP servers bundle large tool schemas (e.g., 49 tools / ~22K tokens for workspace suites). Uber routes SaaS MCPs through the central gateway, projects them as CLIs, and encapsulates common workflows into dedicated code-mode skills.

3. Optimizing Requests / Turn

An ungrounded agent fails slowly rather than cheaply, repeatedly sending an expanding context window to search one more location. Providing richer information upfront is the single most powerful lever to reduce search overhead.

Context Engineering & AI Context Graph

Across Uber’s codebase (hundreds of millions of lines of code) and thousands of tables, agents spend most turns locating information rather than writing code.

Uber engineered the AI Context Graph:

  • 24 million nodes and 80 million edges across 86 node types and 117 edge types.
  • Integrates data from 30+ internal systems: services, engineering teams, incident logs, PRs, architectural design docs, deployments, datasets, and historical query logs.
  • Enables natural language querying for all agents.

Benchmark comparison:

  • Grounded agent: Queried historical usage, located the exact table used by 50+ analysts, and answered in 38 seconds.
  • Ungrounded agent: Lacked visibility, spent 20 minutes inspecting service code, spawned 2 subagents, hit 3 errors, and falsely concluded the dataset was unqueryable.

Visibility & Education

  • The Status Line: Real-time terminal counter tracking active spend per harness and across all harnesses.
  • Visibility and Spend Tiers: Harness pool budget tiers, Slack alerts at 50/80/100% thresholds, streamlined manager approval flows, and interactive cost coaching tips.
  • Session Analysis Dashboard: Inspects session artifacts across local/cloud sandboxes, detecting 16 distinct anti-patterns (suboptimal routing, context bloat, cache expiration, prompt initialization overhead) with precise remediation steps.

What’s Next?

  1. Growing the fleet of managed agents: Systematic roadmap for every new agent (target metrics, benchmarks, Pareto-optimal model).
  2. Dynamic Model Routing: Expanding benchmarks across languages, repositories, and modalities.
  3. Deepening Context Graph Integration: Unlocking graph query capabilities for more autonomous agents.
  4. Real-time Developer Guidance: Shifting from batch anti-pattern detection to continuous trace monitoring.
  5. Continuous Skill Improvement: Auto-recording execution papercuts and generating skill updates directly from traces.

Conclusion

Managing AI coding expenses is a tractable engineering challenge. By eliminating wasted, zero-value token consumption rather than relying solely on lower unit prices or tool downgrades, Uber scaled usage 7x while cutting unit costs across all metrics and maintaining high quality.

The core strategic shift is moving from interactive developer workflows to fully managed agents. Transitioning SDLC workloads into managed environments grants full control over model routing, execution harnesses, and operational spend.

Acknowledgments

Core team & contributors: Abhishek Bhatia, Adam Huda, Aditya Patel, Alok Srivastava, Ameya Ketkar, Anil Purohit, Atakan Kandemir, Ben Chou, Brandon Barker, Danielle Yim, Deepanshu Mehndiratta, Gaurav Gill, Israel Marban, Jason Varbedian, Karen Xu, Lei Shi, Mager Mager, Meghana Somasundara, Peng Liu, Preet Inder, Qiushen Wang, Rush Tehrani, Shesh Patel, Shiven Tripathi, Shubham Gupta, Stas Khalup, Ting Chen, Tse-Shi Wang, Ty Smith, Vikram Hullukunte, Weiqiang Wang, Will Bond. Leadership: Johannes Gehrke, Mattie Toia, Sumanth Sukumar, Praveen Neppalli Naga.

Comments