Embedded Files
# The +20% Illusion: Why Studies on AI Speed Are Hopelessly Outdated
## The Problem Is Not AI. The Problem Is That Studies Are Stuck in 2025.
It's August 17, 2026.
A 2025 Stanford study showed that developers *feel* they're working faster, but objectively tasks are only completed ~20% faster. The industry built a standard narrative from this: "AI is a useful tool, but not a revolution. Autocomplete on steroids."
This is a misconception. Not because the study is bad. But because **it measures a reality from two years ago**, when AI was autocomplete, not an autonomous agent.
---
## Why 20% Was True. Back Then.
Two years ago the landscape looked like this:
- AI generated **~10% useful code**. Anthropic themselves admitted this. The other 90% was garbage that had to be read, discarded, and rewritten.
- There were **no autonomous agents**. No Claude Code, Codex, MCP tools, skills, feedback loops.
- The workflow was: **typed a question in chat → copied the response → pasted into IDE → fixed manually → committed**.
- AI had no access to the codebase, couldn't run tests, couldn't see the result.
In that world, a ~20% speedup was real. Because AI was **an autocomplete reference manual**, not an executor. You were still writing code by hand. AI was just making suggestions.
---
## What Changed. Radically.
Today the landscape is different:
- **Autonomous agents** (Claude Code, Codex, MiMo Code) work in the terminal, read the codebase, run tests, fix bugs, make commits.
- **MCP tools and skills** let agents use external tools: browsers, databases, APIs, CI/CD.
- **The feedback loop is closed**: the agent checks results itself via tests, linters, visual comparison.
- **Memory and context**: agents maintain project context, write to memory.md, load it after compaction.
- **Sub-agents**: you can launch a context-free agent to verify a skill.
In this world, AI is not a reference manual. AI is **the execution layer**. You formulate intent and control the result. The agent writes the code.
---
## 10x on Routine Tasks for Professionals. Not 20%.
Even on ordinary routine tasks, a professional who knows how to work with agents sees a **10x** speedup. Not 20%. Ten times.
Here's why:
**Before (manual):**
- Write a CRUD endpoint: 2 hours
- Write tests: 1 hour
- Set up CI: 30 minutes
- Debug an integration: 2 hours
- **Total: ~5.5 hours**
**Now (with an agent):**
- Formulate the task and criteria: 5 minutes
- Agent writes code, tests, CI: 10 minutes
- Agent runs tests and fixes issues: 10 minutes
- You review the result: 5 minutes
- **Total: ~30 minutes**
This is not a subjective feeling. This is concrete math. And it works **only if**:
- The feedback loop is closed (agent checks results itself)
- There are clear completion criteria (goal)
- The agent has access to tools (MCP, browser, tests)
- You don't skimp on tokens and don't prevent the agent from iterating
If even one of these is missing — you get that 20%. Because you're still working in the "copy from chat, paste manually" paradigm.
---
## 100x on Impossible Tasks. The Next Level.
10x on routine is the baseline. But the real quantum leap happens on tasks **that were previously impossible**.
Example: automating Google Sites through Playwright. 40 actions for content management. DOM is obfuscated, classes like `e2cuDd-fmcmS-v0zI3c-LgApSb`, no proper API.
**Without AI:** you need to know Python, Playwright, reverse-engineer the DOM manually, write and stabilize 40 actions. For someone without this stack — 3–6 months. Or hire a specialist.
**With AI:** 18 hours. Analyzed the DOM, wrote a skill with Python actions, abandoned MCP in favor of direct .py scripts, automated 40 actions.
This is not a speedup. This is **crossing a competence boundary**. The task didn't exist in the backlog because it was impossible. Now it exists and is closed in two days.
No study measuring "time to complete an existing task" will capture this shift. Because the task wasn't in the sample.
---
## Why the Gap Is Growing. Leverage vs Crutch.
Stanford confirms: the gap between top performers and average developers is widening. But the interpretation "tops type prompts faster" is wrong.
The gap is growing because **tops work in the autonomous agent paradigm, while averages work in the chat-copy-paste paradigm**.
**Crutch (2023 paradigm):**
- Copies code from chat into IDE
- Expects AI to get it right the first time
- Doesn't provide a feedback loop
- Skimps on tokens
- Manually breaks down sessions
- Measures efficiency in lines of code
- Gets +20%
**Leverage (2025 paradigm):**
- Formulates intent, the agent executes
- Closes the feedback loop: agent checks results itself
- Tests skills via context-free sub-agents
- Automates session auditing: AI analyzes AI
- Solves compaction via memory.md
- Measures cost/quality ratio and tasks without rework
- Gets 10x on routine and 100x on impossible tasks
The gap is not in typing speed. The gap is in **understanding that AI is not autocomplete — it's the execution layer**.
---
## Why Studies Don't See Reality
Three reasons:
**1. Outdated tool sample.** Studies measure work with autocomplete and chatbots. They don't include autonomous agents with MCP, skills, and feedback loops. That's where 10x happens.
**2. Outdated task sample.** Studies take tasks the developer already knows how to do. They don't include tasks that only became possible with AI. That's where 100x happens.
**3. Outdated metric.** They measure time to complete an existing task. But the right metric is **the class of solvable tasks and the cost/quality ratio**. If in 18 hours you did something that previously required hiring a specialist for 3 months, that's not "+20% to speed." That's a different economics.
---
## What to Do to Be in the "Leverage" Category
Five specific actions:
**1. Stop working in the chat-copy-paste paradigm.** If you're copying code from chat into IDE — you're in 2023. Move to autonomous agents: Claude Code, Codex, MiMo Code. The agent works in the terminal, reads code, runs tests, fixes bugs. You control.
**2. Close the feedback loop.** The agent shouldn't just produce code. It should check the result itself: tests, browser, linters, before/after comparison. Without this you'll always be in the "crutch" category.
**3. Automate session auditing.** No manual log review. A script parses sessions, feeds them to an AI auditor, you get a report: where the prompt failed, where the tools failed. Humans read only the conclusions.
**4. Test skills via sub-agents.** Wrote a skill — launch a sub-agent with an empty context. Couldn't do it — the skill is bad. This is the only way to ensure the skill works not just in your head.
**5. Formalize Judgment.** Domain expertise, taste, architectural decisions — all of this is extracted from the expert's head in 2–3 hours and embedded into domain-specific instructions. After that, AI starts producing solutions accounting for this context automatically.
---
## Conclusion
The Stanford study is not wrong. It simply describes a world where AI = autocomplete on steroids. This world was relevant two years ago, when AI generated 10% useful code, there were no autonomous agents, and people copy-pasted responses from chat.
Today is a different world. AI = execution layer with MCP, skills, and feedback loops. In this world even routine tasks are 10x faster for professionals. And impossible tasks become possible in days instead of months.
The gap between tops and averages is not growing because tops type faster. It's because tops already live in the world of autonomous agents, while averages still copy code from chat and wonder why they only get +20%.
And this gap won't close on its own. It closes only through a paradigm shift: **AI doesn't speed up what you do manually. AI does for you what you used to do manually. And makes possible what you couldn't do at all.**
Page updated
Google Sites
Report abuse