The Calculator and the Auditor: The Layer of Thinking You Can't Safely Offload

Jul 8, 2026

In part one I argued that cognitive debt is a design choice, not a property of the machine — and that the strongest evidence for it, Bastani et al. in PNAS, showed exactly that: the same GPT-4 harmed learning when it acted as a crutch and didn’t when it was built as a tutor. I left one thread hanging on purpose. I want to pull it now, because it goes somewhere the current research mostly doesn’t.

Here’s the number that hangs everyone up. In that study, the students who used the raw chatbot scored 17% worse on the exam than students who never had AI at all — once the tool was taken away. That’s the headline harm. And the very first honest reaction to it, the one I had and maybe you did too, is a good one:

Taking the tool away isn’t the real world. These tools are here now. Measuring people without them is measuring a situation that no longer exists.

That objection is half right. The half it gets wrong is instructive, and the half it gets right is the whole essay.

Why the “unfair test” objection is half wrong

Removing the tool is not a value judgment sneaking in the back door. It’s an instrument — an X-ray. Think about what happens if you only measure people while the tool is in their hands. During practice, the crutch group looked fantastic — 127% better than the control. If with-tool performance were your only yardstick, you’d conclude the raw chatbot was a triumph. You literally cannot distinguish “this tool is building real understanding” from “this tool is a beautiful mask over the absence of it” without, at some point, taking it away and looking underneath.

And here’s the proof that the X-ray is imaging something real rather than just penalizing tool-users for a missing crutch: the two AI groups diverged. Both had full tool access while learning. If removal were merely an artificial handicap, both would have been handicapped equally when it was pulled. Instead the crutch group cratered and the tutored group tested fine. Same removal, different results — which means the test isn’t measuring “do you have your tool,” it’s measuring what got built inside your head while you had it. That’s a legitimate thing to measure. The objection doesn’t sink it.

Why the objection is also half right — and the half that survives is the important one

But grant all that, and something still hasn’t been justified. The study treats internalized, tool-removed, transferable skill as the outcome that counts. That is a choice. A defensible one — but a choice, and it carries a buried assumption: that the valuable competence is the one you can perform alone, with the tool gone.

For a whole class of skills, that assumption is quietly false, and the calculator is the proof of concept.

You can still see the tail end of that panic in the culture. The fear in the 1970s was total: give kids calculators and arithmetic dies, nobody makes change, a generation goes innumerate. And you know what? The mechanism was real. Mental arithmetic did atrophy. Watch a cashier freeze when you hand them an extra nickel after they’ve already keyed in the amount — the register was going to tell them the change, and you just broke its script. That skill genuinely decayed.

But the verdict was wrong, because the world stopped requiring the skill in that form. We got spreadsheets. We got a calculator in every pocket. The competence didn’t vanish; it changed shape — from “compute it in your head” to “know what to compute, and know when the answer looks wrong.” Nobody in 1975 mourning long division could have pictured a person casually describing a problem in plain English and being handed the right formula. Which is now an ordinary Tuesday: you describe what you’re trying to do to an LLM, it tells you the Excel function exists, and you think huh, neat — and you don’t acquire the derivation, but you retain that the tool exists and roughly what it’s for. The next time, you reach for it faster.

That is a real competence, and it is arguably a better one than the thing it replaced. Call it what it is: discovery outrunning acquisition. Being able to map a fuzzy intention onto the right tool — even, especially, when you lack the vocabulary to search for it the old way — is a genuine skill, and for a lot of tasks it beats being able to derive the answer cold. The “unfair test” objection is pointing at this, and it’s correct: for skills the world will let you keep the tool for, measuring unaided performance is scoring a counterfactual that never happens.

So the pro-AI instinct is right. Discovery beats acquisition. Except.

The catch: some layers you can offload, and one you can’t

Here’s the disanalogy that bounds the whole thing, and it’s the load-bearing idea of this essay.

A calculator is bounded and checkable. You offload the multiplication, but the part of you that knows $90 change on a $10 bill is absurd — that part stays online. A calculator never once handed you a fluent, confident, wrong answer dressed up as a right one. Its failures are your typos, and you can feel them. The skill you offloaded (computation) and the skill you kept (knowing when the output is nonsense) are cleanly separable, and you kept the one that matters for safety.

That’s the tell. Not every layer of a task is the same, and they don’t fail together.

There’s a mechanical layer: retrieval, derivation, boilerplate, the rote first pass. Offloading it is mostly fine — that’s what tools are for, and it’s been fine since Thamus grumbled about the written word.

And there’s an evaluative layer — the auditor. The judgment that tells you whether the output you just got is any good. Whether the formula is real or invented. Whether the confident paragraph is also a false one. Whether the discovery you just made is a discovery or a hallucination wearing its clothes.

You can offload the mechanical layer and keep the auditor, and you come out ahead — that’s the calculator, that’s your Excel move, that’s discovery-beats-acquisition working exactly as advertised. But offload the auditor itself and the whole thing inverts, because now the skill you’d need to catch the tool’s mistakes is the skill you handed to the tool. An LLM, unlike a calculator, will absolutely serve you a beautiful wrong answer. Which means the evaluative layer isn’t optional garnish on top of AI use. It’s the thing that makes the offloading survivable.

And notice — this is the part I want to be honest about — your own Excel example secretly runs on a retained auditor. You can pull off “describe it vaguely, get the right formula, trust it” because you’ve spent decades around spreadsheets and can smell when the model hands you something plausible but off. You kept the judgment. Someone building from zero, offloading derivation and evaluation at the same time, can’t run the discovery move safely — they can’t tell a real formula from a confident fake, so every hallucination lands as truth. Discovery beats acquisition right up until you’ve offloaded the very faculty you’d use to judge the discovery. Then it’s just inheritance of whatever the model happened to say.

So here’s the problem with the studies

Hold that distinction next to the research and something uncomfortable shows up.

Nearly all of it — the MIT EEG study, Bastani’s exam, the correlational work — measures tool-removed individual performance. Can you do the thing when the AI is gone. That’s a fine instrument for the crutch-versus-scaffold question; it’s the X-ray from earlier, and it works. But it is close to silent on the question this essay says is the real one: is your auditor intact?

Those are different measurements. “Can you derive the formula unaided” tests the mechanical layer. “Can you tell when the model’s formula is wrong” tests the evaluative layer. A person can score terribly on the first and brilliantly on the second — that’s literally the competent modern professional, the one who offloaded the rote work and kept the judgment. And the current study designs would flag that person as harmed, because they underperform with the tool removed. The instrument can’t see the skill that actually protects them.

Which means the discourse has a blind spot shaped exactly like the thing that matters. We are rigorously measuring whether people can still do the mechanical layer alone, and mostly not measuring whether they can still audit the tool that does it for them. We’ve built a great yardstick for the wrong length.

What would a better instrument even look like? Something like: seed the AI’s output with plausible errors and measure whether the user catches them. Test performance with the tool but on genuinely novel problems where the tool’s fluent answer is subtly wrong. Measure calibration — does the user know when to trust the output and when to distrust it — rather than raw unaided recall. Measure the intent-to-tool mapping directly. None of that requires pretending the tool will someday vanish. It takes the tool’s permanence as given, and asks the question that actually follows from it: when you and the machine disagree, do you still have what it takes to be right?

Which is why guardrails matter more than “matter”

Part one landed on: guardrails matter, and whoever controls the design decides whether you get a crutch or a tutor. This is where that gets teeth. The right guardrail isn’t the one that merely slows down answer-copying. It’s the one that keeps your auditor in the loop — that hands you hints instead of conclusions specifically so the evaluative layer stays switched on and load-bearing. Bastani’s tutor arm didn’t just prevent cheating; it kept students doing the judging. That’s the mechanism worth protecting, and it’s the design spec worth demanding: not “does the tool withhold the answer,” but “does the tool keep me evaluating.”

And it’s why this loops back to who controls the thing. A tool optimized to be maximally satisfying will retire your auditor for you — friction is exactly what an engagement metric wants to sand off, and your judgment is friction. A tool you can shape, inspect, and steer can be built to keep you sharp on purpose. If the competence that keeps AI-use survivable is an intact evaluative layer, then the fight over who gets to design the tool is the fight over whether you’re allowed to keep it.

The honest caveat, again

Same disclaimer as part one, pointed at myself this time: I’m proposing a better measurement, not reporting one. As far as I can find, nobody has run the clean study — offload the mechanical layer, protect the evaluative one, and track judgment over years. Until someone does, “keep your auditor” is a well-motivated hypothesis, not a proven prescription.

But the argument doesn’t need the study to be useful, because it’s really a reframe. The panic asks will AI make us worse at doing things ourselves, and the answer is a shrug — sometimes, for skills that stopped mattering, and so what. The question worth its weight is will we keep the judgment to know when the machine is wrong — and that one, nobody gets to shrug at. It’s not about your ability to do the tool’s job. It’s about whether you can still tell when the tool didn’t.