There is a single command with an enormous amount of weight which has been used for as long as teams have done version control. At its core, git blame is answering a simple question: who created this line so I know whom to interrogate. A big part of debugging, ownership assignment and understanding our own systems relies on being able to answer that.
That question ceases to have a resolution when an agent writes the line. Most teams have not yet weighed what that means in a loss.
Not the answer: "The model wrote it"
Put yourself in the incident six months from now. A piece of logic is doing a thing that no one anticipated. You follow the chain and discover an automated agent did this. You want to know who wrote it, the only answer is: the model.
That is not something you can [then take action against.
To put together a coherent reply, you would require quite a bit more. Which model produced this? Which version of it? What was asked, what was the prompt? Then who was the human that slipped this result into the system?
Without these facts an AI change is just a black box with some kind of timestamp. You know when it arrived. You know nothing about how it appeared there and why.
Provenance as a first-class artifact
The takeaway is that provenance can no longer be an afterthought - or a comment field. It has to be first class artifact, a product of deliberate recording.
I work to the standard that every commit touched by AI holds five things:
- the prompt text,
- the model name,
- the model version,
- the timestamp,
- and the responsible human.
It transports them in a form that can withstand scrutiny: signed, append-only and tamper-proof. Not the type of note that is edited later once an inconvenient truth emerges in the story. Its best feature is that it still speaks the truth even when under duress.
Why "ownership metadata" undersells it
Sometimes you will hear this referred to as "ownership metadata," given that it resembles a tagging exercise: name them and move on. To characterize it in that way under-sells what is at stake.
The point is that such a record sustains after the alleged event, in precisely those situations in which memories conflict, pressure is high, and the description of events disputed. A signed, append-only trail answers "what changed and why" regardless of who remembers what or who happened to be running the agent that day.
That reliability is exactly why we build it. A postfactum record you can modify is only worth precisely nothing when it would be the most useful.
The second payoff: reproducibility
Now there is an obvious if sometimes ignored benefit here proved to be worth spelling out.
This is because as soon you treat the code as a composite output of specification and prompt provenance and spec make change reproducible. Use the same spec with a more recent model, and you see the behaviour you wanted - encoded in different code, backed by a full audit of how intent became result.
Which transforms a model migration from being an act of faith into just another engineering task. You are not hoping the new model acts right, you can regenerate and check it against your evals.
Unglamorous infrastructure
Provenance has no demo. Nobody in a review is impressed by it. On a good day it creates nothing visible. It appears on only one type of day: the day that something breaks, and someone with the power to do something about it asks you how x made it out into production, and your answer is complete and credible.
That is the trade. But by working quietly now, you earn the right later to tell the truth when the amount of AI-generated code overwrites all rational human minds.
Assuming an agent committed something today to your main branch - can you reconstruct in a year which model and prompt produced it, and who signed off?
No comments:
Post a Comment