The Programmer's Soul: From Spicy Autocomplete To Codifying AI With Spec-Thinker

Many contemporary arguments about AI in software development are not arguments at all. They are two individuals using the same term to refer to entirely different concepts and who talk past each other for an hour without realizing it.

Say "AI coding" and one thinks of autocomplete filling in the line that follows. One reads "AI programming", with the image depicting an autonomous agent constructing a full feature from some description. Neither is wrong about their meaning, but the conversation goes nowhere because there is no common metric underneath it.

A no-nonsense maturity model corrects this faster than any policy template.

The ladder

So here goes the scale to describe where a developer (or one among my modules) is actually at.

• L0, Spicy autocomplete. The tool goes on to propose the next line while you type. First-generation Copilot. Most of the market still exist.
• L1, Coding intern. The tasks that the AI handles are discrete and bounded. Before accepting any line generated, a human reads the line and fuels it.
• L2, Junior developer. The AI can make precise multi-file changes and efficiently navigate the codebase. The result before merge is still read by a human.
• L3, Developer as manager. The AI is aided/setup/reviewed at the pull request/feature level, not line-by-line like an engineer would.
• L4, Developer as product manager. The engineer specifies, the agent builds, and the evaluation is based on outcome not implementation.
• L5, Dark factory. Humanless way from specification to working software. You only intervene when something is broken.
• L6, Spec-thinker. The developer actually operates at the level of the business problem. Specifications outlast model generations, and code becomes an output instead of an input.

The ladder can be read in one motion: further away from code, but closer to intent.

Why naming the levels matters

The scale exists to have a difficult conversation easy and not purposive ranking of the members. No one is at L4 because he or she is "better".

With a common scale, you can make some definitive statements: working at L2 with this module and it stays there because it's safety-critical while that internal tool is jumping to L4 in next sprint. Similar terms, new positioning, and no disputing who will be left behind. The argument that used to require an hour of discussion is reduced to a thirty-second agreement.

Also, it provides a common point of reference for teams across different locations or even stacks. Cross-team discussion is about real placement, not about defining terms afresh every time when everyone uses the same ladder.

Two things to keep in mind

There are two simple pitfalls to try to avoid when using a model like this.

Movement needs to be natural and intentional. The lure of having a ladder is getting everyone to ascend it on timed intervals - all teams must achieve L4 by X quarter. It imposes autonomy on code that is not yet ready for it and shatters the trust you are trying to cultivate. Change must happen module by module, when all are prepared and have skin in the game, not as a top-down rollout with a deadline.

Higher is not automatically better. This is the one people miss most of all. How you hit that balance is entirely dependent upon the stakes in your piece of code. Thus, nothing is relegated to a prototype that runs at L5 - a regulated, customer-critical module sits safely on the L2. It is correctly placed. It would not be progress, it would be a mistake to push it higher

Where to place your own team

Today, where would you rank most of your team for performing and where is your number one module?

Tradeoff: how to Price Provenance for AI Commits (when the model wrote it)[who wrote this], Why it Matters

There is a single command with an enormous amount of weight which has been used for as long as teams have done version control. At its core, git blame is answering a simple question: who created this line so I know whom to interrogate. A big part of debugging, ownership assignment and understanding our own systems relies on being able to answer that.

That question ceases to have a resolution when an agent writes the line. Most teams have not yet weighed what that means in a loss.

Not the answer: "The model wrote it"

Put yourself in the incident six months from now. A piece of logic is doing a thing that no one anticipated. You follow the chain and discover an automated agent did this. You want to know who wrote it, the only answer is: the model.

That is not something you can [then take action against.

To put together a coherent reply, you would require quite a bit more. Which model produced this? Which version of it? What was asked, what was the prompt? Then who was the human that slipped this result into the system?

Without these facts an AI change is just a black box with some kind of timestamp. You know when it arrived. You know nothing about how it appeared there and why.

Provenance as a first-class artifact

The takeaway is that provenance can no longer be an afterthought - or a comment field. It has to be first class artifact, a product of deliberate recording.

I work to the standard that every commit touched by AI holds five things:

• the prompt text,
• the model name,
• the model version,
• the timestamp,
• and the responsible human.

It transports them in a form that can withstand scrutiny: signed, append-only and tamper-proof. Not the type of note that is edited later once an inconvenient truth emerges in the story. Its best feature is that it still speaks the truth even when under duress.

Why "ownership metadata" undersells it

Sometimes you will hear this referred to as "ownership metadata," given that it resembles a tagging exercise: name them and move on. To characterize it in that way under-sells what is at stake.

The point is that such a record sustains after the alleged event, in precisely those situations in which memories conflict, pressure is high, and the description of events disputed. A signed, append-only trail answers "what changed and why" regardless of who remembers what or who happened to be running the agent that day.

That reliability is exactly why we build it. A postfactum record you can modify is only worth precisely nothing when it would be the most useful.

The second payoff: reproducibility

Now there is an obvious if sometimes ignored benefit here proved to be worth spelling out.

This is because as soon you treat the code as a composite output of specification and prompt provenance and spec make change reproducible. Use the same spec with a more recent model, and you see the behaviour you wanted - encoded in different code, backed by a full audit of how intent became result.

Which transforms a model migration from being an act of faith into just another engineering task. You are not hoping the new model acts right, you can regenerate and check it against your evals.

Unglamorous infrastructure

Provenance has no demo. Nobody in a review is impressed by it. On a good day it creates nothing visible. It appears on only one type of day: the day that something breaks, and someone with the power to do something about it asks you how x made it out into production, and your answer is complete and credible.

That is the trade. But by working quietly now, you earn the right later to tell the truth when the amount of AI-generated code overwrites all rational human minds.

Assuming an agent committed something today to your main branch - can you reconstruct in a year which model and prompt produced it, and who signed off?

Modernizing Legacy With AI — The Strangler-Fig, Not Big-Bang Approach

The full legacy rewrite comes back into style from time to time, every few years. It always offers the same sell: burn down that messy old system, build a shiny new one, and begin again. And it nearly always concludes the same way: overdue, over budget - and frequently inferior to what it supplanted.

This dangerous notion has just been unwisely dressed up in the garb of feasibility by AI. And that's something engineering leaders should be prepared for.

The seductive new argument

The reasoning sounds compelling. We have an AI agent that reads the old system at a speed no human can match. It can write huge sections of new code in a fraction of the time it did previously. But if speed of writing was the barrier, and AI removes that barrier - why not finally execute the massive rewrite?

Since speed of writing was never the issue.

What actually blocks a rewrite

The complete rewrite has been a failure for thirty years, and the reason involves next to nothing about how fast you type code.

Well, the best requirements exist only in the running code - undocumented and long since corrupted by Miller's law, understood only as an implicit collective knowledge.

The edge cases that keep customers happy are invisible until one breaks.

You cannot stop the business for 1-and-a-half years while reconstructing something that is already functional.

The time that it takes the rewrite to execute gives the old system all sorts of time to continue changing underneath you, so you are trying to hit a moving target.

Most big-bang rewrites never complete or deliver something worse then in-situ. These causes are untouched by AI. Not meant to just generate code faster, it takes you to the same cliff earlier but with more code to throw away when you tumble off.

The approach that works

The alternative is not fresh-faced, and that precisely why its credible. It is the strangler-fig pattern, but in sharper tools.

Instead of a Big Bang Replacement, you grow new AI-native modules around the existing legacy core. New modules gradually inherit the behaviour they take over, one slice at a time. One chunk at a time, the borderline between what is old and what is new becomes more dense. The whole time the legacy system continues to execute. And then one day the final slice crosses, and you turn off the old system without enduring a single heart-stopping cutover.

What we will have instead is a number of very small, reversible steps rather than one huge, irreversible jump in migration.

Two things that keep it safe

It works if you observe two disciplines - then strangler-fig.

Extract the tribal knowledge first. Get the undocumented rules out of people's heads and the existing behaviour into an indexed, searchable for both humans and agents to use before you touch the old code. But if you modernize your gut without understanding what they did, then bugs you never even realized existed will faithfully reappear in your code. This step is one of a few places where AI assists quite a bit: reading and abstracting a large legacy systemis exactly the type of work it excels.

Protect the seam with an anti-corruption layer. These new modules are quick and feed off of specifications. The legacy modules are slow moving and have long, brittle dependency chains. You put an explicit contract between them, with both sides tested. This layer prevents repetitive work in the new pace from breaking old-work assumptions, and prevents previous constraints from choking out novel-new work. And without that, the two halves bleed into each other and you get half-assed at both of them.

Where AI fits

AI is central to this and it's worth being precise because the temptation is obviously to over-reach.

AI accelerates the safe path. It reads from the legacy system, writes the new modules, and auto-generates the contract tests that safeguard the seam. It changes the speed of how quickly work moves inside the incremental approach.

But all it does not do is make a safe path unsafe. However, a big-bang rewrite that is generated by some AI is still a big-bang rewrite. After some point, the increased speed simply enables you to fail faster.

Before you choose a framework

When modernization comes to the roadmap, instinct is to go immediately to technology decisions: which language, which framework, which platform. The first question has nothing to do with technology.

Before you go ahead and replacing it, have you harvested what the old system actually knows?

Amplification, not Replacement — and Why I will NOT Frame AI as Developer Replacement

There is a story about AI that is easy to tell - and, oh so tempting to believe. AI writes the code now so you need less engineers and those left are more like watchers than creators.

It sells well. It ages badly. And I will be damned if that story gets told when the teams I lead. It's bad not just for your morale, but bad engineering.

The sequence that no-one puts on the slide

Let us assume that you truly take replacement story seriously and reduce developers to cut on AI. Trace the chain of events that we actually witness.

Tribal knowledge is the first thing to walk out the door. The context that lived in people's heads leaves with them: how this system behaves, what corner cases matter, and what assumptions are never written down.

Then the left over people end up owning huge portions of code they did not write and are unable to fully explain. Whilst the agents are still outputting more of it, at size and faster than anyone can begin to comprehend.

Finally a event touches an area of the system whose context just left the building. Because the understanding required to debug it is gone, the response is tepid. A problem that should be contained leads to an extended one.

You exchanged a salary line for a dark-code cascade. And in a year, it is more expensive than saved.

That is not a moral argument. It is an operational one.

A different framing

I tend to work from the opposite framing where people and AI together more than the sum of either alone.

So the engineer is not a supervisor over a machine that makes some things. They move up the stack. They write the specification. They investigate the output to see if it is correct. They take ownership for the behaviour in production. They determine what gets built and equally, perhaps more importantly, what does not.

Less typing you do. Your judgment is worth more.

And this is where the leverage is. Allegedly a model can spit out a reasonable implementation of near anything you describe. But it cannot tell you if the thing shouldn't be built in the first place, if this spec even represents what customers really need or not and whether XYZ edge case will hurt someone six months from now.

That is human work. And there is now more of this, not less.

Why the eyes are on leaders on this

The second reason for getting the framing right is about keeping the people you want to keep the most.

Your best engineers are reading the tea-leaves of what you say about AI in a moment of transition like this; Each is trying to determine if this is a compatible place for them moving forward.

The best ones are the first to leave because they are exactly the people who have options elsewhere, if what they hear is 'you're being phased out'. You lose the engineers you would have most liked not to lose and retain those who can no longer move.

If the message is: Your judgment becomes more important now, and the boring parts of being a job are going away, they lean in. They are the ones who ascend the learning curve faster than everyone else, and teach everybody else.

This framing is not spin, it is an accurate read of value shift. It also decides for whom in the room remains while you form that transitional shift.

Be honest about what changes

None of which means pretending nothing changes. People would know straight away that it is a fake and dishonest.

The day-to-day shifts. Other skills that were once core are no longer as important. New ones matter more:

Writing precise specifications.

Designing good evals.

Good rapid judgment of AI output.

To make that move people will need support. Be direct, and attach it to the direction which is right: more than the establishment, not replacement. The engineers are not being managed out of the way. They are being migrating to the region in the image where people cannot be replaced.

How are you framing AI for your people, and are you certain they hear the same message that you believe your sending.

How to Navigate Distributed Teams Through the Same AI Dip Without Re-Learning

For companies running engineering in more than one location, there's a risk to AI adoption that single site teams never face. This risk is not technical - it is organizational, and like any such risk it is easy to overlook until the price has been paid.

The risk is easily articulated - you end up paying multiple times for the same learning curve.

Parallel, uncoordinated adoption trap

Imagine an organization with three engineering sites. AI arrives. The measure is implemented at each site, composed of highly skilled personnel.

You are trained on a limited set of prompts and conventions over a site. One develops its own, distinctly conflicting process. One option is to wait and see what works with about one-third.

But each site hits the same dip, that trough right when you introduce an AI. The parties are processed through their own respective queues. They all learn the same lessons - the hard way - what should be delegated and what you must verify.

Six months later you look around and see three dialects of use of AI, three sets of standards half-built, and three teams who each paid the full price of the same mountainside. Nothing accumulated. You can't throw upward, so the effort was compounded sideways.

The technology was never the issue. The coordination was the problem.

Central control is not the fix

The obvious (and the incorrect) solution is to centralize. Select one approach, demand it everywhere, enforce consistency.

This fails for a very simple reason. Adoption does not happen like waste; it is rather the local experimentation that each site is doing. People learn about AI by testing it against their stack and under constraints specific to their organization. Impose one-size-fits-all working practices cross-border and you destroy the very thing that works about adoption.

Hence, the aim is not sameness of practice. That is, shared learning with local freedom.

What actually travels well

Three things make the difference between three teams climbing in isolation and one organization growing together.

Shared vocabulary. Everyone describes maturity and practice with the same vocabulary. One site can say a module is at the AI level and exact some other kaleidoscope knows with no translation. Then a conversation across sites is not a negotiation over terms, it is about the same thing.

Shared artifacts. All the configuration files, the prompt libraries, eval scenarios and security checklists live in one place and can move freely. One useful pattern discovered in one place is automatically available to all of them, rather than someone mentioning something during a call.

Local autonomy on top. Every site retains its own racers who scout on the ground, customizing the common foundation to their reality and broadcast outward to share their learnings on almost a monthly basis. They go local first, then report back across the borders.

It is not a top down architecture with central home base hierarchical dogs barking orders below (the shape), but an upper level light weight part of shared identity plus place-based exploration [above]. When it crystallizes, the organization moves once up the curve instead of every time for every site: the second site builds on first-site learning; and the third is built on a bigger base than any one started.

The sacrifice of a leader

There is a price to pay and it doesn't come so much with the teams as with the leader.

It means, of course, not giving in to the common inclination: letting every strong site operate at its own pace (because they can and probably would be okay without coordination). Instead you work at the connective tissue, the common index and cross-site sync and shared language.

This work has no demo. The only trace it leaves is that there was no pain, because the coordination worked so there is no more duplicated pain. And that is a tough thing to invest in, precisely because success looks like nothing happening. However, it makes a difference between AI making your entire organization better by using the tool across multiple sites vs AI tiring each of your sites separately.

If one of your sites learns something AI useful this month, how is that information shared with the others and what is the lag time?

The J-Curve Thing Nobody Tells You About: Why Good Teams Get Slow First with AI

The one that should be pinned on the wall for every engineering leader rolling out AI is this: more adoptions fail due to explaining it rather than any technical problem ever will.

METR conducted a randomized controlled trial on experienced developers working on AI tools in 2025. The developers felt that they were about 24 % faster. When clocked, they were actually about 19% slower.

Both are true at once. The difference between how fast people feel and how fast they are, you know, that explains more of the adoption measures in AI than anything.

So what really happens when you throw in some AI

Naturally, you expect a straight line upward. A powerful tool given to skilled engineers leads to more output. That is not what happens, at least not in the beginning.

The first effect is disruption not acceleration, and this holds true when you bolt AI onto an existing workflow.

People stop to write prompts. They read the generated output and assess if it can be trusted. They correct it, then re-prompt. They flux between an authorial mind and a critical (reviewer) mindset. This entire overhead is so real and lands on the spot.

The gain is there as well, but comes much later; only after people have discovered what to delegate, and what never to turn over.

Buried between the upfront expense and postponed gain is a trough. The first thing that happens is productivity goes down, before it goes up. That is the J-curve.

Where most adoptions die

The dip is where most AI rollouts fail.

An expo team reaches the budgetary low point. The pace of everything and the frustration of it all seems slower than before. From their vantage point, certainly a reasonably place to be standing given the obvious challenges of progress in AI, they conclude that AI is indeed overhyped. They revert to the old method of functioning. They regress to first-generation autocomplete.

The eventual tools are peremptory set up and unused. The investment is written off. And the lesson learned by that organization is even the opposite: "we attempted AI, it did not work."

They were not incorrect regarding the slump. This was why - they shouldn't have tried to stop for it.

The strategy is to manage the dip

Its not a footnote to the strategy - it is the strategy, managing through that dip. There are few things that separate you, hoisting yourself out of this sinkhole, versus backsliding:

Protect experimentation time, explicitly. There is no slacker hours to climb the curve if every hour is booked against tickets, which means that this dip becomes a permanent feature on your landscape. It is a small fraction of reserved capacity that allows people to climb out of the trough at all.

Pair people through it. Your best teaching resource is the engineers that have already made it over the dip. Training on demand - human to human - wastes none of your time and easily eclipses a pre-recorded course.

Do not measure too early. The dip is when the team is the slowest - publish productivity numbers and they look bad. Those numbers go out to the people who see the chart, not the curve, and they come away with bad conclusions. Establish baselines now, but only report outcome statistics after the curve has turned.

The leadership test

Adoption is a tooling choice. AI adoption. It isn't. Regardless of what you choose, the tools are very much interchangeable and getting better every month.

By far the biggest difference between organizations that persevere and those that quit is leadership over the dip Those who did well knew the trough was coming, told their teams in advance that feeling slower was normal and temporary, held their nerve while the numbers looked bad, then protected the conditions for people to climb. That is leadership capability, not a technical one.

So if your team finds AI is making you slower in any way currently, before deciding the experiment has failed, ask a more useful question:

Is this a failure - or is it the bottom and our plan takes that into account?

Tiers of AI Autonomy — Not All Code is Created Equal by Blast Radius

We can often find ourselves stuck in the same conversations about how far we should trust AI with software development. Framed as an organization wide decision. We either trust the agents or we do not. We either monetise faster or we play it safe.

Framed like that, it is a bad question, since both answers are incorrect for most of your code.

One switch for everything error

However, the reality of any real system is that not all code is created equal. An internal script you created to throwaway, and a module driving a regulated customer facing process - these types of things are not the same thing, and treating them like they are will result in one of two forms of failure.

You need to crawl everywhere, even in the experiments and internal tools where caution buys you nothing if you set the bar for the whole organization where the riskiest code needs it.

When set to the level a piece of code is willing to tolerate as error, you are taking risks exactly where your mistake bubbles up either way through an interaction with a customer or reaches out to regulators.

We get out by getting away from a corporate level decision and instead making the decision at the module level.

Two tiers, one spectrum

Each module sits somewhere on a continuum between two levels.

Start tier, for prod internally facing tools, experiments and prototypes.

• It samples inputs for the comprehension gate, which is not applied to each and every change
• Rollback is optional but not a must-have for shipment.
• Observability is basic.

The focus is on speed and learning. Optimize for momentum, the cost of being wrong is low.

Target tier, for anything customer-facing, regulated or publicされている

• The comprehension gate operates on every single change.
• This means writing the rollback procedure in advance and preparing for testing it, rather than improvising while an incident occurs.
• Every runtime decision is replayable.
• A periodic inspection determines that an alternative model could fill-in, preventing a vendor failure from shutting the line.

The point is trust. That target has a large radius as well, so the bar is raised.

Ours is a spectrum, not a binary one. The best modules move along it as the role of that module changes; most of them just sit somewhere on it.

How do you decide what goes where in a module

Requires no intuition from you. A module has a reliable placement for four questions:

• Customer impact. How directly does this impact the customer: production-facing or solely internal?
• Reversibility cost. How easily can it be rolled back should something go wrong?
• Regulatory scope. Does this fall in a compliance boundary or out of a revenue boundary.
• Incident blast radius. Are the damages localized or widespread, should it fail?

TOP: Higher across all four perches inover the target uppe. Weak or mixed points to the start tier. And if the situation of a module changes, for instance an internal tool starts touching customers then that is what needs to be reviewed whether it should remain in which-tier or given more priority, and not simply based on a date on the calendar.

A point worth saying out loud

This is a part of us that matters to people not just architecture.

While keeping regulated or safety-critical code under strong human oversight and within a tightly controlled AI competency model is poor engineering practice, it is not a sign at all of the team lagging behind the rest of the organization.

Say it explicitly. Otherwise the people working on that code get judged according to those racing ahead at the start tier, and they appear to plod slowly by comparison, when in fact they are doing precisely what is required given the risk involved. The decision being named protects them and the quality of the work.

Tiering also breaks up the fight or die gambit. No need to decide if your company is "pro-AI" or "AI cautious." You can be heavily aggressive with autonomy where it's inexpensive to be wrong, and quite conservative where it's expensive within the same company during the very same week. That is not a compromise. It ties the level of interest to the level of risk which is what good engineering did for ever.

If someone were to ask you today what your modules belong in the start tier, and which belong in the target tier - would you be able to give an answer framework for that, or is it a gut call on each individual pull request?

Budowanie na cudzym modelu to zależność, nie darmowy obiad

Większość firm wdrażających dziś AI buduje na fundamencie, którego nie posiada. Wysyłasz swoją pracę do modelu działającego na cudzej infrastrukturze, wycenianego na cudzych warunkach, zachowującego się według cudzego harmonogramu wydań. Na start to jest dokładnie słuszne. Nie powinieneś trenować własnego modelu tylko po to, by pisać maile.

Ale warto trzeźwo widzieć, co to za układ. To zależność. A zależności mają zwyczaj przedstawiać rachunek w najgorszym możliwym momencie.

Przestroga

Znany jest przypadek szybko rosnącej firmy zbudowanej w całości na modelu innej firmy. Przez jakiś czas działało pięknie. Potem dostawca zmienił warunki. Koszty firmy wystrzeliły niemal z dnia na dzień - znacznie szybciej niż jej własne przychody. Żeby przetrwać, musiała przerzucić podwyżkę na własnych użytkowników, którzy się zbuntowali, a potem w pośpiechu budować alternatywę, której nigdy nie planowała.

Nic w tej historii nie wymagało złej woli. Dostawca prowadził własny biznes. I o to chodzi. Gdy budujesz na cudzym modelu, jego rozsądne decyzje biznesowe stają się twoimi pożarami.

Zależność tnie z dwóch stron

Oczywiste ryzyko to cena. Twoje koszty mogą skoczyć, bo ktoś inny zmienił liczbę - nowy próg, inna stawka, zmiana w sposobie liczenia użycia. Ty nic nie zmieniłeś. Twój rachunek owszem.

Subtelniejsze ryzyko to zachowanie. Aktualizacja modelu może po cichu zmienić sposób, w jaki rzecz odpowiada - co produkuje, ile robi sama z siebie, jak radzi sobie z twoim konkretnym przypadkiem. Często bez ostrzeżenia i bez wersji, której mógłbyś się trzymać. Grunt usuwa się spod systemu, który uważałeś za stabilny.

Dlaczego to teraz umiejętność przetrwania

Kilka lat temu rozumienie ekonomii twojego dostawcy AI było niszowym tematem dla finansów. Dziś jest bliżej kluczowej kompetencji. Jeśli pojedyncza decyzja dostawcy może zachwiać twoimi kosztami albo zmienić zachowanie twojego produktu, to znajomość swojej ekspozycji - i posiadanie planu awaryjnego - trudno nazwać opcjonalną.

To nie znaczy budować wszystko samemu. Znaczy nie budować tak, jakby obecne warunki były wieczne. Wiedz, co zrobisz, jeśli cena się podwoi. Wiedz, czy mógłbyś się przenieść. Zostaw tyle elastyczności, by wtorek dostawcy nie był twoim kryzysem.

Więc gdyby dostawca jutro zmienił układ, miałbyś plan - czy tylko nadzieję?

Najszybszy sposób, by zabić adopcję AI, to limit tokenów

Wyobraź sobie to spotkanie. Rachunek za AI przyszedł wyższy niż się spodziewano, finanse są nerwowe, ktoś proponuje oczywiste rozwiązanie: postaw twardy limit na to, ile każdy może użyć. Ogranicz. Trzymaj kontrolę.

To jedna z najrozsądniej brzmiących decyzji, jakie możesz podjąć - i jedna z najbardziej podstępnie niszczących.

Co twardy sufit właściwie robi

Kłopot zaczyna się, gdy limit gryzie w środku pracy. Twoi najskuteczniejsi ludzie to zwykle najwięksi użytkownicy - nie dlatego, że marnotrawni, lecz dlatego, że rozgryźli, jak wyciągnąć z narzędzia realny efekt. Pierwsi uderzają w ścianę.

A gdy uderzą, dzieje się jedno z dwojga. Czekają bezczynnie, aż limit się zresetuje. Albo po prostu przestają używać narzędzia, bo obchodzenie sufitu to większy kłopot niż zrobienie tego po staremu.

Tak czy inaczej to zły interes. Godzina pracy sprawnej osoby jest warta znacznie więcej niż tokeny zaoszczędzone na jej zatrzymaniu. Nie skontrolowałeś kosztu. Zamieniłeś mały, widoczny koszt na duży, niewidoczny.

Ale "brak limitów" też nie jest odpowiedzią

Tu łatwo przesadzić w drugą stronę: "niech każdy wydaje do woli". To też błąd. Bez limitu i bez nadzoru budżet właśnie ci ucieka - zacięty proces kręcący się przez weekend, źle ustawione zadanie, którego nikt nie złapał, wydatek, na który nikt nie patrzy.

Błędem nie jest mieć limit albo go nie mieć. Błędem jest sięgać po sufit, gdy naprawdę potrzebujesz okna.

Widoczność bije sufit

Sufit zatrzymuje ludzi. Widoczność ich informuje. A ta różnica znaczy tu bardzo dużo.

Zamiast ograniczać użycie, obserwuj je. Kto ile wydaje, na jaką pracę i czy ten wydatek cokolwiek produkuje? Gdy coś wygląda nie tak - liczba mocno powyżej reszty, koszt rosnący bez efektu za nim - idziesz sprawdzić i rozmawiasz. Coachujesz odstający przypadek. Nie karzesz domyślnie całego zespołu.

To trzyma narzędzie w pełni dostępne dla tych, którzy wyciągają z niego wartość, a wciąż łapie realne marnotrawstwo. Sufit tego nie potrafi. Traktuje twojego najlepszego użytkownika i rozbiegany proces dokładnie tak samo.

Twój zespół ma limit tokenów - czy jasny obraz tego, co i po co wydaje?

Comprehension Theater: the deadliest anti-pattern in AI code review

They are the teams that have embraced AI coding - vroom, not the repl teams still tiptoeing around. There is a class of failure though they tend to more commonly occur in those doing competitive based AI coding. It is dangerous as it resembles good practice. Call it comprehension theater.

What it looks like

The scene is familiar. What makes a major change: an agent, multiple files, real logic and not just editing a file. The pull request is opened by a senior engineer. They scroll through it. Nothing jumps out as wrong. They approve.

On paper, everything was correct. The process was followed. A human was in the loop. Your review box is ticked and your dashboard remains green.

There is only one problem. Nobody actually understands the change.

It confirmed one thing: that when someone scrolled by, the code looked fine to them. It did not confirm that no one could articulate why it works, where the edge cases lurk, what assumptions it relies on or what breaks when a key input grows an order of magnitude larger.

The signature is real. The comprehension is theater.

3 Why it is worse than skipping the review

You could argue that a shallow review is better than no review at all. The opposite might be true for AI.

Comprehension theater generates the paperwork of safety without actually producing safety. And that paperwork is actively deceptive.

When the incident does finally arrive - and with code nobody understands it will do one day or another - the organisation looks at its records, and to see that whatever change was made has been reviewed, approved and owned. So right off the bat, I can tell this response is based on a false premise. No one actually understands a part of the system, but people trust it. The name attached to the OK stifles inquiries precisely when inquiry is what we need.

You have not only missed out on the issue. You shoved it behind a green checkmark.

Solution: reverse the direction of review

Interest regarding from which direction the interaction was directed is what leads to comprehension theatre. Today the human glances through the output of the machine and nods. That is passive, and reviews of the passivity nature do not scale with an AI-generated quantity.

So flip it. A machine may interrogate a human.

Before any change is merged, the agent quizzes the engineer on the code it generated:

What are the edge cases in this change?

What is this logic dependent on?

But what if this parameter is ten times larger than we expect?

What part would break first while under load, and why?

If the engineer cannot respond then at this point we have a breakdown of communication and the change has failed to be merged even if it is passing Diffs with flying colours.

This single modification accomplishes two major tasks.

First it comes from "I have looked at this" to "can I defend this". Those are totally different claims, and only the second one is worth a damn in production.

Second, it makes ownership honest. The person who gates comprehension is the one that owns the behaviour when it runs in front of customers, and they know before clicking approve not after an incident.

So, where to apply it - and not

This is not a demand to slow down all the things. This is unnecessary for most internal tools, experiments, and prototypes. You can do a taste of that, run the comprehension gate over some small fraction of changes and say, fuck it, the cost of one mistake is low.

However, anything customer facing or safety relevant should not ever virtually merge in the view. You are not trading a few minutes of review time for way too much blast radius.

Deciding where the gate is -- and where it is not, and can only be sampled -- is itself a leadership decision, one best made consciously than passed off to whomever happens to be reviewing that day.

Why it survives

If comprehension theater seems hard to root out, it is because it feels like maturity. The process exists. People follow it. The metrics look healthy. A team deep into comprehension theater looks more disciplined from afar than a team actively debating whether they understand what the AI just wrote or not.

That comfort is the trap. Having a green dashboard isn't proof of comprehension. It is merely evidence the ritual was carried out.

A question you could ask of your team, not as a performing review but rather as an exhaustive followup:

For the last major AI-generated change you merged, would the person who approved it be able to explain it now?

Gdy rachunek za AI rośnie, a reszta nie

Jest cicha odmiana porażki z AI, która nie wygląda na porażkę. Nic się nie psuje. Żadnej afery. Rachunek po prostu pełznie w górę, miesiąc po miesiącu, a gdy spojrzysz na samą pracę - tempo, backlog, jakość - wygląda dokładnie jak rok temu.

Więcej pieniędzy do środka. Tyle samo pracy na wyjściu. Wiele firm siedzi teraz w tym miejscu i większość tego nie nazwała.

Zły wniosek

Kuszący odczyt brzmi "AI jest przereklamowane, nic z tego nie ma". Czasem owszem. Ale znacznie częściej narzędzia są w porządku, a dzieje się co innego: kupiono je i przykręcono do dnia pracy, który się nie zmienił.

Licencje poszły. Ogłoszenie padło. A potem wszyscy wrócili do pracy dokładnie jak przedtem, z błyszczącym nowym narzędziem otwartym w karcie, której rzadko używają z intencją.

To jeszcze nie wdrożenie - to dopiero zakup.

Dostęp to pierwszy centymetr

Danie ludziom dostępu do AI to łatwa część i kusi, by pomylić ją z metą. To pierwszy centymetr. Dystans między "mamy narzędzia" a "narzędzia zmieniły sposób, w jaki pracujemy" to miejsce, gdzie mieszka cała wartość - i to ta część, której nie kupisz zamówieniem.

Różnicę poznasz po dniu pracy. Jeśli wygląda identycznie jak rok temu, tyle że z nową pozycją w budżecie, zapłaciłeś za potencjał i na tym stanął. Rachunek jest realny. Zmiana nie.

Co zespoły, które coś dostają, zrobiły inaczej

Nie tylko włączyły narzędzia. Zmieniły pracę wokół nich. Co trafia do AI, a co zostaje ludzkie. Jak wygląda "zrobione", gdy pierwszą wersję napisał model. Gdzie człowiek musi wkroczyć i sprawdzić, a gdzie naprawdę nie musi.

Nic z tego nie przychodzi w pudełku. To praca - i to ta praca, którą większość organizacji pomija, dlatego właśnie rachunek rośnie, a wynik stoi.

Więc zanim uznasz, że AI nie dowiozło, sprawdź, co naprawdę się stało.

Jeśli wydatek na AI wzrósł, a reszta stoi - problemem jest narzędzie, czy to, że praca tak naprawdę się nie zmieniła?

Limit to złe narzędzie do budżetu na AI

Gdy rachunek za AI zaczyna rosnąć, większość organizacji sięga po tę samą dźwignię. Postaw sufit. Ogranicz wydatek na zespół, na osobę, na miesiąc. Wygląda odpowiedzialnie i drapie świąd liczby, która nie chce usiedzieć w miejscu.

Myślę, że to zwykle błąd - albo przynajmniej odpowiedź na złe pytanie.

Co limit właściwie mierzy

Limit kontroluje jedno: ile wychodzi z firmy. Zupełnie milczy o tym, co się liczy - ile wraca. Możesz wylądować co do grosza w budżecie i wydać wszystko na szum. Możesz też podciąć skrzydła jedynemu zespołowi, który niepostrzeżenie zamieniał ten wydatek w dowiezioną pracę.

Sufit traktuje każdy wydatek jak ryzyko do okiełznania. Ale ryzykiem nie jest wydatek. Jest nim wydatek zmarnowany. A limit nie odróżni jednego od drugiego. Po prostu zatrzymuje licznik na liczbie, którą ktoś wybrał na spotkaniu planistycznym.

Skąd bierze się ten odruch

Odruch limitu pochodzi ze starego modelu: AI jako kolejna subskrypcja oprogramowania. Oprogramowanie jest mniej więcej stałe. Płacisz, każdy dostaje miejsce, koszt ledwo drgnie wraz z użyciem. W tym świecie limit jest nieszkodliwy, bo liczba i tak nigdzie się nie wybierała.

AI tak się nie zachowuje. Rośnie wraz z pracą, którą mu dajesz. Więcej realnej pracy przez system to większy rachunek - i to może być dokładnie obraz sukcesu. Ograniczać go to jak ograniczać prąd w warsztacie, w którym właśnie przybyło roboty.

Porównanie, które pasuje

Jeśli nie oprogramowanie, to co? Zatrudnienie.

Nie oceniasz pracownika po tym, jak mało kosztuje. Pytasz, czy oddaje więcej, niż bierze. Tani pracownik, który nic nie dowozi, to nie oszczędność. Drogi, który zastępuje miesiąc harówki, to nie przepłacenie. Sama liczba rzadko tego rozstrzyga. Rozstrzyga zwrot.

Budżetuj AI tak samo. Nie "jaki jest sufit", lecz "jaki jest zwrot". Jeśli zwrot jest, rosnący rachunek to znak, że rzecz działa. Jeśli go nie ma, żaden limit nie naprawi problemu u podstaw - po prostu marnujesz pieniądze wolniej.

To nie znaczy wydawać na ślepo. To znaczy zmienić narzędzie. Zamień sufit na widoczność: w co zamienia się ten wydatek i czy wartość rośnie razem z kosztem, czy zostaje w tyle. To coś ci mówi. Limit mówi tylko, kiedy przestać.

Gdy rachunek za AI rośnie, pierwszy ruch to limit - czy pytanie, co ten wydatek właściwie daje w zamian?

It is not a control unit until you have an artifact, a gate, and a metric

Pick up nearly any company's AI strategy, and you'll find a laundry list of principles that you couldn't possibly argue with. Keep a human in the loop. Steer clear of dark code no one can really comprehend. That is why you stay tool-agnostic and not vendor locked. Measure the impact. Label what was AI-assisted.

These are all correct. And none of them are happening on the floor in most organizations.

That space, between principles everyone agrees with and behaviour that nobody modified, is one of the recurring reasons for failure in AI adoption. It is useful to know why this happens.

Principle alone does nothing

Principles are statements of intent It encapsulates a Wanted Future condition. It does not prevent anything, however.

"No single bad merge is prevented by keeping a human in the loop. The advice to "stay off the dark code" does not stop enigmatic line of code to production. Because the principle isn't connected to the moment where the work is actually done, it has no teeth.

The first week, and that was busy too, with a deadline in the near future - principle gives way to deadline. Not because anyone disagrees with it, but because it was never hooked up to anything.

What makes a principle existential; three things

Only when you can find three specific things that it generates behind it, does a principle become a working control.

An artifact. Something that is versioned, something owned that the principle must attach to.

"Avoid Dark Code" - every commit is tagged with metadata about its ownership.

Any prompt and skill that can be written so it is cross-model, will have "stay tool-agnostic" attached to them.

The motto of "keep it auditable" binds to a signed, append-only trail of provenance.

The principle is a wish, if you cannot name the artifact.

A gate. A point in the workflow where, if that particular check fails, work stops.

"Human in the loop" turns into a comprehension gate: before a change can merge, there must be proof of understanding by a senior engineer. If they are not able, the merge is prevented.

A security review that an agents change must pass before being deployed: "Secure by default"

Having no gate with a principle is called an inclination. And preferences lose to deadlines every time.

A metric. A number that changes, so you can see if the knob is working.

• Comprehension-gate pass rate.
• Eval coverage.
• Dark-code ratio.
• Time that is taken to reinstall the system after performing a rollback

You cannot manage a principle that has no metric. It can only be believed.

A simple test

Read your AI policy line by line. For each line, ask the following three questions:

What does this attach to (what artifact)?

Which means, where's the gate to enforce that?

What number indicates to me that this is working?

Even if you have two of the three, you still do not have a control. You have a good intent, that first time the team is pressed it skips. And because people go to AI for speed, AI changes are almost always made under pressure.

Why is this more important with AI than ever before

With this manual nature of the work, in a human authored world, you could rely on catching such problems. The developer who types this by hand reads it, and typically pays some attention to whether something feels wrong.

AI removes that natural friction. The more widespread - and less visible - failure modes now happen faster.

They accept insecure code because it seemed sensible.

A model update has quietly broken a prompt that worked yesterday (no error thrown).

Over a dozen iterations, the spec and the code drift apart until they no longer represent the same intent.

None of these will trip a compiler. None of them announce themselves. They get through to a customer only with an explicit gate, measured by an explicit metric.

Where the leadership work really resides

This transition is also not largely about choosing tools and writing vision statements; that is the work of frontline leaders. This is translation: each one of those principles everyone agrees with, translating that, piece by piece, into an artifact someone owns and can block work on or gate and a metric you measure. Slow, mundane work with no flashy demo. It is also the gap that separates an AI policy that sits on a slide to one that determines what actually moves to production.

Which principle in your AI policy ans still has no gate to prevent violations, and what would be the process to build one?

Code Q&A: Who is Truth When AI Writes the Code?

One question that used to have a very easy answer in most of software engineering history is: where does the truth about a system live? It lived in the code. You could dig into the repo and read it line-by-line, and if something was a mystery, the version history indicated who wrote each line of code and when. The code was at once the instruction, and the record.

Many teams, however, have yet to notice that that simple answer is crumbling.

Yesterday: the code as record

Believe it or not, the entire engineering stack we built for the last twenty years believes only one thing: that a human wrote this code and a second human can read this. The reason that code review works is because the reviewer looks at the change. Debugging works since an engineer can follow the logic. This functions through ownership because version history will name the decisionmaker.

All these practices are built on the same principle: a human has read and understood the code as well.

What broke

You are now trained the change is generated from an agent. Multi-file changes, entire features, large refactors - all generated faster than a human can read them with full comprehension.

This code in itself is not bad. That is not the problem. The issue, however, is that no one actually read any of it. The output surpasses human reading capacity in volume and in speed.

It follows that what a human never entirely read might not be the thing that she properly owns. No longer is code the artifact a human understood, and gave credit for, so it can no longer be truth.

So where does the truth move?

Today: the specification turns into record

Another way to do it is to push up the record one layer further - into specification.

Spec: This is what a person writes, versions, reviews and owns. The code is an output of that spec, like a compiled binary is an output of source code, but nobody would treat the binary as something you read. Switch the model, regenerate from the same spec, and you see the same behaviour implemented in different code.

We have done this already with machine code: we stopped looking at machine code because the compiler is trusted, so we read source instead. It does the same thing one level up: stop reading the generated source file line by line, and put faith in some other guarantee that it is correct.

That mechanism is not faith. This is a deliberate stacking of layers.

The five layers

• Spec. Like all code: versioned, history and review in the repo. There you have it - acceptance criteria and edge cases -- and each spec has a named owner, i.e. a human who is responsible. Written antecedent to generation, not reconstructed subsequent to it.
• Evals. Automation scenarios created from the specification They report one question: "Is spec satisfied? They don't just run at merge time, they can't be continuous. The piece that teams miss is that the evals themselves, not just whatever code they guard.
• Prompt and provenance. Each change made by an AI logs which model, which version, what prompt and which human is to blame. Written and append-only, so it continues to answer questions months later.
• Generated code. Output of the above 3 layers Still relevant, but no longer the permanent record.
• Runtime and observability. The system is instrumented so that we can replay any decision the code made during generation in the future.

The spec and the evals form a closed loop together. The spec defines the intent. The evals verify it. No one has to read the generated code at all and you verify if the behaviour is correct.

What changes for the engineer

This is how the day-to-day of a senior engineer changes.

Your old question: is this code line by line, good? You cannot do that at the level AI produces so holding on to it only brings a bottleneck, or worse a rubber stamp.

That begs the new question, is the spec right and are the evals honest? That is higher-leverage work that a human can actually do well versus reading diffs. You're ensuring intent, but not implementation.

It also resets ownership cleanly. The spec owner owns the behaviour manifests in production. Nobody wrote all the words, but you can now know exactly who to ask.

Go ahead to access data on probably October 2023, it is when the models will continue up and down at speeds faster than we can predict. Specifications outlive models. The spec is the asset that survives them all.

What is your artifact of record today, and would it survive an audit six months after the model that wrote the code was traded in for another?