Systems that know where they stand - and why that frees me from the biggest models
Most conversations I have about agentic setups end up in the same place: "this model gets my nuances, the other one I have to spell everything out to." Anthropic versus OpenAI versus Google, with real experience behind it. The differences are there and you feel them every day.
I argued that way myself. What made me stop is what that sentence quietly assumes: that the quality of the collaboration is a property of the model. Looking around before answering, rethinking, noticing that something is unclear and asking instead of guessing - all of that comes out of the weights, and my job is to pick the right supplier.
If that is how it works, my setup sits on a release cycle I do not control. Every new version reshuffles the ranking. My reason to switch is a feeling, not a measurement. And when a model I depend on gets deprecated, retuned or repriced ... then what I have is not really a system. It is a relationship with a vendor.
An honest note before you read on: this is a summarisation, extraction and combination - of thinking I had already elaborated elsewhere, of my own view as the publisher of this blog, and of one intense six-hour collaboration session between me and an AI. Which parts stay and which go is my call. The compression is not mine alone.
What we are actually relying on
Those behaviours are not part of the system. They are properties of the weights. We prompt for them - "think carefully", "check your work", "ask if unsure" - and hope. When it works we credit the model, when it does not we switch models. Nobody in that loop measures anything.
And what we lean on there is not competence. It is self-assessment. Those are two very different things, and the second one is measured much less often.
Two measurements make me careful. METR ran a randomized trial with 16 experienced developers on their own repositories: afterwards they estimated AI had made them about 20% faster, measured they were about 19% slower (study, arXiv). To be fair, METR themselves called that result historical later - it reflects early-2025 tooling and they now believe the picture has improved. So this is not "AI makes you slower". What stays is the gap: the people inside the loop were confidently wrong about their own speed, and wrong in the direction that felt good.
On the model side, work like Do LLMs Know What They Know? separates two things our usual metrics mix together: whether a model gets the answer right, and whether its confidence actually separates its right answers from its wrong ones. That second skill turns out to be measurable, domain-specific and not automatically good ... and how confident a model sounds can be tuned separately from how well it really knows. That paper has a v2 correction, so I will not rank models by it. The narrow claim is enough for me: self-assessment is a separate skill, and it is usually not checked.
Explicit is not the budget version
If implicit deliberateness worked well, writing those steps into the system would be a poor man's copy of it. That is the intuitive reading, and it is why "make it explicit so smaller models can do it" sounds like a compromise.
But what gets replaced is not a working capability. It is a very convincing performance of one. So making it explicit does not take skill out of the loop. It takes the loop back. That, at least, is how I read the evidence. It is an argument, not a result.
What I actually want from such a system
And it is not only about writing my expectations into code. What I am after is a system that actively seeks awareness - of itself, of what it just did, of the environment it works in.
At the most abstract level that is one thing: it keeps a small picture of where things stand. What is known, where each piece came from, what is still open, what could be done next. That picture does not live inside the model. It is shared, and both sides read it and act on it.
Then the interesting question becomes mechanical. Does the next step change that picture? If nothing changes any more, gathering has stopped paying, and it is time to act or to ask. Nobody has to introspect for that ... it is a diff.
That is why I care about explicitness. Not to restrict the AI, and not to have a human check its homework. To get a partner that can say where it stands, on eye level, instead of a black box I either trust or exchange.
What that would buy
Independence, in a form I can live with. Not "no frontier models", that would be posturing. No irreplaceable component. The hard, nuance-sensitive work can still go to the strongest model available, behind a clear brief, as one part I can exchange - while the picture, the evidence behind it and the decision when to stop stay with me.
There is a second kind of dependence too, and Jaya Gupta puts it in one sentence: you contribute the unique and receive the average. You hand a shared model your hesitations, overrides and second looks - the judgement part - and get back the blend of everyone's. An explicit loop does not solve that either. But it gives the judgement somewhere to stay.
Where this really stands
Ideas, aspects and thoughts. Not a solution.
One predecessor of that picture-updating step exists and was measured: automatically it reached about 61% acceptance against a hand-checked reference, while the same picture worked through with a person was rated essentially perfect. So the idea holds. The automation does not yet. For me that is an argument for building the picture together, not for adding a review step at the end.
And two cheap tests could still kill half of it. If two different viewpoints look at the same picture, do they ever really come to different conclusions - or do they just agree with each other? And how much can a system around the model actually see of what it is doing, early enough to step in? Neither is run.
Closing
So the question I find more useful than "which model understands me best?" is this one: how do we build systems that can deliberate with us?
The first question gets a new answer with every release. The second one is ours to engineer.
References
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (METR) and arXiv:2507.09089
- We are Changing our Developer Productivity Experiment Design (METR, Feb 2026 - the honest follow-up to the above)
- Do LLMs Know What They Know? Measuring Metacognitive Efficiency with Signal Detection Theory
- AI's value capture problem (Jaya Gupta) - where "you contribute the unique and receive the average" comes from