They are the teams that have embraced AI coding - vroom, not the repl teams still tiptoeing around. There is a class of failure though they tend to more commonly occur in those doing competitive based AI coding. It is dangerous as it resembles good practice. Call it comprehension theater.
What it looks like
The scene is familiar. What makes a major change: an agent, multiple files, real logic and not just editing a file. The pull request is opened by a senior engineer. They scroll through it. Nothing jumps out as wrong. They approve.
On paper, everything was correct. The process was followed. A human was in the loop. Your review box is ticked and your dashboard remains green.
There is only one problem. Nobody actually understands the change.
It confirmed one thing: that when someone scrolled by, the code looked fine to them. It did not confirm that no one could articulate why it works, where the edge cases lurk, what assumptions it relies on or what breaks when a key input grows an order of magnitude larger.
The signature is real. The comprehension is theater.
3 Why it is worse than skipping the review
You could argue that a shallow review is better than no review at all. The opposite might be true for AI.
Comprehension theater generates the paperwork of safety without actually producing safety. And that paperwork is actively deceptive.
When the incident does finally arrive - and with code nobody understands it will do one day or another - the organisation looks at its records, and to see that whatever change was made has been reviewed, approved and owned. So right off the bat, I can tell this response is based on a false premise. No one actually understands a part of the system, but people trust it. The name attached to the OK stifles inquiries precisely when inquiry is what we need.
You have not only missed out on the issue. You shoved it behind a green checkmark.
Solution: reverse the direction of review
Interest regarding from which direction the interaction was directed is what leads to comprehension theatre. Today the human glances through the output of the machine and nods. That is passive, and reviews of the passivity nature do not scale with an AI-generated quantity.
So flip it. A machine may interrogate a human.
Before any change is merged, the agent quizzes the engineer on the code it generated:
What are the edge cases in this change?
What is this logic dependent on?
But what if this parameter is ten times larger than we expect?
What part would break first while under load, and why?
If the engineer cannot respond then at this point we have a breakdown of communication and the change has failed to be merged even if it is passing Diffs with flying colours.
This single modification accomplishes two major tasks.
First it comes from "I have looked at this" to "can I defend this". Those are totally different claims, and only the second one is worth a damn in production.
Second, it makes ownership honest. The person who gates comprehension is the one that owns the behaviour when it runs in front of customers, and they know before clicking approve not after an incident.
So, where to apply it - and not
This is not a demand to slow down all the things. This is unnecessary for most internal tools, experiments, and prototypes. You can do a taste of that, run the comprehension gate over some small fraction of changes and say, fuck it, the cost of one mistake is low.
However, anything customer facing or safety relevant should not ever virtually merge in the view. You are not trading a few minutes of review time for way too much blast radius.
Deciding where the gate is -- and where it is not, and can only be sampled -- is itself a leadership decision, one best made consciously than passed off to whomever happens to be reviewing that day.
Why it survives
If comprehension theater seems hard to root out, it is because it feels like maturity. The process exists. People follow it. The metrics look healthy. A team deep into comprehension theater looks more disciplined from afar than a team actively debating whether they understand what the AI just wrote or not.
That comfort is the trap. Having a green dashboard isn't proof of comprehension. It is merely evidence the ritual was carried out.
A question you could ask of your team, not as a performing review but rather as an exhaustive followup:
For the last major AI-generated change you merged, would the person who approved it be able to explain it now?
No comments:
Post a Comment