On the Seam: Working Drafts · Part 3 of a three-part series
Building Cognitive Legitimacy in the Agentic Enterprise
If distrust is what wakes up under autonomy, then comprehensibility is what puts it back to sleep. The case for treating cognitive legitimacy as the load-bearing wall of agent adoption — and building it into the architecture, not the slide deck.
In the last piece I argued that two of my own research findings may invert under autonomy: distrust and compliance, both inert when a human supervises generative AI, may switch on hard the moment an agent acts on its own. That was the diagnosis. This is the part where I say what to do about it if I am right, because a diagnosis that ends in alarm is not useful to anyone running a business.
Before I go further, I owe you a caveat, and it is not a small one. My empirical work measured generative AI — tools that suggest, with a human reviewing every output. The claim that distrust and compliance reverse under genuine autonomy is a suspicion I am reasoning toward from mechanism, not a result I have measured. I believe the argument is sound and the mechanism is well-supported, but I have not yet run the study that would confirm it, and I want to be honest that this is a hypothesis I am building on, not a finding I am reporting. Everything that follows should be read in that light: this is where I think the evidence points, offered so it can be tested and argued with, not asserted as settled.
The short version: if distrust is the thing that wakes up under autonomy, then the work of the agentic enterprise is to build legitimacy faster than autonomy raises distrust. And the dimension of legitimacy that does the most work, the one most underbuilt and most misunderstood, is cognitive legitimacy — the sense that a system makes sense, that its behavior is comprehensible, that a person can form a correct expectation of what it will do before it does it.
I want to make the case that cognitive legitimacy is not a soft, communications-department concern. It is the load-bearing wall of agent adoption, and most organizations are building everything except it.
What cognitive legitimacy means when the system acts
Suchman's framework gives us four kinds of legitimacy, and in my research three of them drove advocacy for generative AI: cognitive, pragmatic, and normative. Pragmatic legitimacy is usefulness — does it pay off. Normative is appropriateness — should we be doing this. Cognitive is comprehensibility — does it make sense.
For a tool that suggests, cognitive legitimacy means understanding the output. You read the summary, it tracks, you trust the next one a little more. The unit of comprehension is the result, and you have the result in front of you.
For an agent that acts, the unit of comprehension changes. You are no longer judging an output you can see. You are forming an expectation about a process you cannot watch — a sequence of decisions, tool calls, and actions the agent will take across time, on your behalf, without checking in. Cognitive legitimacy is no longer “do I understand what it produced.” It is “can I predict what it will do.”
This is why post-hoc explainability — the dominant paradigm for generative AI trust — is necessary but no longer sufficient. An explanation of why the agent did something arrives after the action is already a consequence. What builds cognitive legitimacy for an actor is not a better explanation afterward. It is predictability beforehand: bounded behavior, legible intent, and a track record that lets a person's expectation converge on what the agent actually does.
Why this is the dimension that reduces distrust
Recall the mechanism from the last piece. Distrust under autonomy is a live prediction about what an unwatched system will do. It is fear of the gap between what you authorized and what actually happens when you are not looking.
Cognitive legitimacy closes that gap directly. The more predictable an agent's behavior, the smaller the space of unpleasant surprises, and the less room distrust has to operate. You cannot argue someone out of distrusting an unpredictable actor; the distrust is rational. You can only make the actor predictable enough that the distrust has nothing to feed on. This is why cognitive legitimacy, almost alone among the dimensions, works on the distrust pole rather than the trust pole. Pragmatic legitimacy builds trust by showing payoff. Cognitive legitimacy reduces distrust by removing surprise.
That distinction matters because, as I argued before, I suspect the binding constraint on agent advocacy is distrust, not trust. If that is right, the dimension that reduces distrust is the one that releases the constraint. Cognitive legitimacy is not one lever among four. Under autonomy, it is the lever closest to the thing actually holding adoption back.
Building it into the architecture, not the slide deck
Here is where I part company with the way legitimacy usually gets discussed. Legitimacy is treated as a perception to be managed — something you establish through communication, framing, change management. That is half the picture, and for an acting system it is the less important half. The comprehensibility of an agent is a property of how it is built, not how it is described.
Concretely, cognitive legitimacy for an agent is manufactured by four architectural choices, and none of them are marketing.
First, bounded authority. An agent whose tool access is scoped to least privilege is an agent whose behavior is predictable by construction — it cannot surprise you in domains it cannot reach. The boundary is not just a security control; it is a comprehensibility control. A person can form an accurate expectation of an agent that can only do a few things far more easily than one that can do anything.
Second, legible intent. An agent that exposes its plan before it executes — that says what it intends to do and waits, or at minimum declares its reasoning in a form a human can inspect — lets expectation form ahead of action. This is the difference between an agent that acts and reports, and an agent that proposes and proceeds. The second is dramatically more comprehensible, and the cost is latency you can choose to pay where stakes are high.
Third, observable behavior. Memory, state, and action history that a person can actually examine turn an opaque process into a legible one. Not a log nobody reads — an interface that makes the agent's behavior over time inspectable, so that comprehension can be built from evidence rather than faith.
Fourth, reversibility. An agent whose actions can be undone is an agent whose mistakes are survivable, and survivable mistakes are the raw material of learned predictability. You cannot build an accurate expectation of a system you were too afraid to ever let run. Reversibility lowers the cost of the first observations enough that the track record can start accumulating.
Return to the invoice agent from the previous piece to see how these compose. Bounded authority means the agent can pay approved vendors up to a threshold and can do nothing else — it cannot create new vendors, cannot exceed the limit, cannot touch payroll. Legible intent means that before each payment it states the invoice, the matched purchase order, and the amount, in a form the finance team can read at a glance. Observable behavior means every match and payment is recorded in an interface the team actually opens, not a log buried in a system nobody queries. Reversibility means a payment can be clawed back within a window if the match was wrong. None of these makes the agent more capable. Every one of them makes it more comprehensible — and a finance team that can see the boundary, read the intent, inspect the history, and undo the error is a team whose distrust has nothing left to grip. That is cognitive legitimacy doing its actual job, and you will notice it is built entirely from architecture, not from a single sentence of reassurance.
The orchestration problem hiding underneath
There is a second-order issue I have to name, because it is where I see the most expensive mistakes being made right now. Comprehensibility does not compose. You can build four agents that are each individually legible and wire them into a workflow whose collective behavior no one can predict. The emergent behavior of a multi-agent system can diverge sharply from what any single agent's design would suggest.
This is the agentic version of a familiar enterprise pathology. In my consulting work I watch teams chase local wins — a useful agent here, a clever automation there — each of which earns pragmatic legitimacy on its own. What accumulates is a sprawl of agents that nobody architected as a system, and the system-level comprehensibility is zero. Each agent makes sense. The whole makes none. And distrust, correctly, attaches to the whole.
Cognitive legitimacy at enterprise scale is therefore not just a property of individual agents. It is a property of the orchestration layer — the part of the architecture that decides which agents exist, what they may do, how they interact, and who owns each one. An organization that lets agents proliferate bottom-up, optimizing each for local usefulness, is manufacturing exactly the system-level unpredictability that drives the distrust it will later struggle to overcome. The paved road — a governed platform through which agents are provisioned with identity, scope, and oversight built in — is not bureaucratic friction. It is how comprehensibility survives scale.
What I would tell a leader starting now
If you are standing up agentic capability this year, the sequence that follows from all of this is specific.
Build the containment and observability layer before you build the agents. The instinct is to prove value first and govern later, but under autonomy that order manufactures distrust you then cannot undo. The architecture that makes agents comprehensible — scoped authority, legible intent, inspectable behavior, reversibility — is the architecture that makes them adoptable. It is not a tax on the work. It is the work.
Measure distrust directly, not just trust. Most adoption dashboards track confidence and satisfaction — the trust pole. If the constraint is distrust, you are flying blind on the variable that matters. Ask people not only whether they trust the agent's competence, but whether they would let it act unsupervised, and watch the gap between those two numbers. That gap is your distrust, and cognitive legitimacy is how you close it.
I will say a little more about that gap, because it is the most practical instrument in this whole series. Two questions, asked of the same person about the same agent. One: how confident are you that this agent is competent at its task. Two: how comfortable are you letting it act without your review. For a generative tool, those two numbers move together — if you think it is good, you are happy to use it, because you are reviewing the output anyway. For an agent, I suspect they come apart, and the size of the gap between them would be a direct readout of the distrust that autonomy has woken. A small gap would mean cognitive legitimacy is doing its work and the agent is ready for more autonomy. A large gap would mean competence is not your problem; comprehensibility is, and no amount of demonstrating accuracy will close it. You close it by making the agent more predictable, more bounded, more legible — not more capable. I know of no cheaper diagnostic to try for whether an agent is actually ready to be trusted with autonomy than watching those two numbers converge or refuse to — and it is a hypothesis you can test on your own floor before you take my word for any of it.
And govern the orchestration, not just the agents. The legitimacy of any single agent is necessary but not sufficient. The thing your workforce ultimately advocates for or refuses is the system, and the system's comprehensibility is decided by who controls what gets deployed and how the pieces interact. Own that layer deliberately, or sprawl will own it for you.
I will end where the series began. The story we keep telling about agentic AI is a story about speed — agents that act faster than people, enterprises that move faster than competitors. But the constraint was never the speed of action. It was the speed at which legitimacy can be built relative to the distrust that autonomy unleashes. The enterprises that win the agentic transition will not be the ones with the most autonomous agents. They will be the ones whose agents are the most comprehensible — the ones that made their machines legible enough to trust appropriately, and bounded enough to distrust safely.
That is the seam I keep working: the place where what the technology can do meets what an organization can legitimately let it do. It is not finished. It is offered.
Dr. Trey Harper writes on trust, legitimacy, and the architecture of the agentic enterprise at treyharper.com and in the LinkedIn newsletter On the Seam: Working Drafts. The views expressed here are entirely my own and do not represent the policy or position of my employer or any customer.