The Developer's Bargain: You Pay, and You're the Product

In the summer of 2025, a research group called METR ran the most careful experiment anyone had done on AI coding tools. Sixteen experienced open-source developers, 246 real tasks in large, mature repositories they knew intimately, every session screen-recorded, everyone paid properly. Half the tasks allowed AI assistance; half didn’t.

The developers using AI finished 19% slower. And when asked afterward, they estimated they’d been about 20% faster.

Read that again, because it’s not a story about AI being bad at code. It’s a story about you being bad at knowing what AI does to you. A 39-point gap between what developers felt and what the clock said — among experienced professionals, on their own projects, while being recorded. That gap is the real subject of this article. If you’re just joining, this is part two of a series about the three parties in every AI coding session; this one is about the party holding the keyboard.

The tool that feels better than it measures

Let’s be fair to the tools first, because the 19% number comes with an expiration date.

METR themselves revised the picture in February 2026: with 57 developers and 800+ tasks, the measured slowdown shrank — to around -18% for the original cohort and roughly -4% for newly recruited developers, with confidence intervals wide enough to include zero. The models got better, the harnesses got better, people learned to use them. And a well-known 2023 experiment found developers 55% faster with Copilot — on a from-scratch HTTP server, the kind of greenfield task AI eats for breakfast. Context decides everything: blank file, big speedup; million-line legacy codebase, much murkier.

But here’s what didn’t shrink: the perception gap. In METR’s May 2026 survey, technical workers retrospectively rated AI as doubling the value of their work — while the measured estimates hovered near zero, and METR’s own staff, who know these numbers, gave the lowest estimates of any group surveyed. Felt gains: 2x. Measured gains: somewhere between “modest” and “we can’t detect them.”

Why does the feeling run so far ahead of the clock? Because the tool changes which seconds you experience. The AI compresses the boring part — typing out the thing you already knew how to write — and hands you back a review problem instead. Reviewing feels like progress. Debugging the almost-right suggestion feels like progress too, right up until you check the clock. Stack Overflow’s 2025 survey found that “almost right, but not quite” is now the single biggest frustration with AI tools, cited by 66% of developers. The near-miss is the most expensive kind of wrong: close enough to accept, wrong enough to cost you an hour three days later.

And there’s one more number from METR’s follow-up, buried in the methodology notes, that says more than any benchmark: between 30% and 50% of the participating developers declined to submit tasks they’d have to complete without AI. Professional software engineers, refusing their own tickets unless the assistant came along. METR reported it as a selection-bias problem. It’s also the most honest measurement of dependency anyone has published.

Confidence is the failure mode

If the tools merely felt faster than they were, that would cost you some planning accuracy. The deeper problem is that the same distortion applies to quality.

A Stanford team ran a security experiment that should be taped to every monitor: developers with an AI assistant wrote measurably less secure code than developers without one — and were more likely to believe their code was secure. Both halves matter. The assistant didn’t just introduce vulnerabilities; it introduced confidence in them. The developers who did best were the ones who trusted the tool least and interrogated its output hardest.

Microsoft Research and Carnegie Mellon found the same shape in a 2025 study of knowledge workers: the more people trusted generative AI, the less critical thinking they applied to its output. Confidence in the tool and scrutiny of the tool move in opposite directions. Psychologists have called this automation bias for decades — pilots do it with autopilots, radiologists with diagnostic aids. Developers turn out to be exactly as human.

Put the two findings together and you get the failure mode of AI-assisted development in one sentence: the tool is most dangerous precisely when it feels most trustworthy. Nothing about that is fixed by better models — a better model that’s wrong less often earns more trust, which buys its remaining errors a softer review.

What you actually emit

So far this is a story about you and your code. Now add the third party in the room.

Every accept is a thumbs-up. Every reject is a thumbs-down. Every time you take a suggestion and then fix it, you’ve supplied the correction alongside the mistake — a before-and-after pair, annotated by a professional, in full context. GitHub’s own data puts suggestion acceptance around 30%, which means a working developer renders hundreds of expert judgments a day. In machine learning, that’s called labeled preference data. It’s the scarcest, most valuable ingredient in training better models, and the industry’s name for the person producing it is “user.”

For years the polite fiction was that your coding sessions stayed between you and the autocomplete. Then, between late 2025 and mid 2026, the individual tiers flipped one by one. Anthropic’s consumer terms update asked Free, Pro, and Max users to choose whether their chats and coding sessions feed training — with five-year retention if you allow it. GitHub moved Copilot Free and Pro to training on interaction data by default — prompts, snippets, and those accept/reject signals — from April 2026, opt-out. Cursor’s data-use page is admirably direct: with Privacy Mode off, it may use your code and editor actions “to train our models.” Windsurf’s current terms are the bluntest of all: the free tier trains on your data with no opt-out — opting out is a feature of the paid plans.

Notice what’s not covered by any of those toggles. Engagement telemetry — what you accepted, dismissed, how long you hovered, your edit patterns — typically lives under separate, longer-lived retention rules; GitHub’s Copilot trust documentation has put engagement-data retention on the order of two years, well beyond the life of any single suggestion. And even deletion has an asterisk: during the New York Times lawsuit, a court ordered OpenAI to preserve all output logs — overriding user deletions and its own 30-day policy — for months. Your provider’s retention promise is subordinate to any court with jurisdiction over it.

If you work at a company that bought the business tier, most of this doesn’t apply to your work account — enterprise contracts exclude training, which is the tell we covered in part one. The developers actually paying with their data are the ones on personal plans: freelancers, open-source maintainers, side-project builders, and everyone whose employer never got around to buying seats.

The skill ledger and the quiet repricing

There’s a longer-term entry in the bargain, and it’s the one that compounds.

GitClear, which analyzes code at scale, tracked 211 million changed lines between 2020 and 2024 and found the fingerprint of assisted coding everywhere: copy-pasted code rising year over year until it exceeded refactored code for the first time in the dataset’s history, and rework-within-two-weeks climbing steadily. The pattern is exactly what you’d expect when generating is cheap and understanding is expensive: more code, touched less deeply, revised sooner.

That’s the codebase’s ledger. Yours is the skill you don’t practice. Writing code you could have written builds nothing, fine — but reading code you didn’t write, deciding whether it’s correct, is a skill fed by having written a lot of code yourself. Every generation of tools has traded some low-level fluency for leverage, and mostly that trade has been worth it. The difference this time is that the thing being outsourced — judgment about what code should exist — is the thing your review job depends on. The skill you’re resting is the skill you’re being retained for.

There’s a subtler entry on the same page: the expertise you are building is increasingly tool-shaped. Months of learning which prompts work, what your assistant handles well, where it lies to you — that’s real skill, hard-won, and almost none of it transfers. It’s intuition about one vendor’s model, and it resets every time the vendor swaps the model under you, which the subscription terms let them do at any time. Developers used to accumulate mastery of languages and systems that outlived any employer. Mastery of a hosted model has the shelf life of someone else’s deployment schedule.

And the market is already repricing the rungs below you. Stanford economists tracking payroll data for millions of workers found employment for 22-to-25-year-olds in AI-exposed occupations — software development prominently among them — down around 13% relative to less-exposed peers, with junior developer employment down roughly 20% from its late-2022 peak. Honest caveat: those years also contained brutal tech layoffs and high interest rates, so AI isn’t the sole cause. But the direction is consistent, and the mechanism is visible from your own desk: the tasks that used to justify hiring a junior are the tasks the assistant now does.

Here’s the loop, closed: you pay a subscription. You supply the judgments that train the next model. The next model absorbs more of the work that used to be the on-ramp into your profession. The gains show up in your employer’s velocity metrics and your provider’s revenue — and the case for paying people like you what they were paid last year gets a little thinner. Nobody designed that loop maliciously. You’re just the only party in it who isn’t compounding.

What to do now

None of this argues for abstinence — the leverage is real, and opting out entirely is its own career risk. It argues for using the tools the way the best-performing developers in the Stanford study did: deliberately, skeptically, on your terms.

1. Flip your training toggle, today. Claude consumer settings, GitHub Settings → Privacy, Cursor Privacy Mode. Two minutes each. If you’re on Windsurf’s free tier, know that there is no toggle — that’s part of the price.
2. Separate your accounts. If any client or employer code ever passes through a personal AI subscription, you’ve made a data-sharing decision on their behalf. Work code belongs on the work tier — the one with the enterprise carve-out.
3. Treat acceptance as authorship. The Stanford result flips once developers interrogate output instead of approving it. If you’d be embarrassed to explain a line in review, don’t ship it because the tab key was convenient.
4. Keep one workout unassisted. A regular task you do without the assistant — a code review, a debugging session, a small feature. Not nostalgia: it’s the calibration that keeps you able to judge the machine’s output, which is the job you’re actually paid for now.
5. Time yourself, occasionally. The METR gap lives in unmeasured feelings. One honest week of tracking where the hours went — including the debugging of almost-right suggestions — tells you more than any benchmark.
6. Know what your harness leaks. Telemetry, error reporting, local transcript caches — most tools document environment flags to reduce emission. Twenty minutes with the data-usage page of your daily driver is the cheapest security review you’ll ever do.

The bargain isn’t hidden anymore; it’s in the terms of service, dated and versioned. You trade attention, judgment, and training signal for genuine leverage. Some version of that trade is worth making — but make it with the meter running where you can see it. Next up: what the same bargain looks like from the CFO’s chair, where the subsidy is bigger, and so is the hook.