WELCOME
A Human in the Loop Is Not Enough
Why meaningful AI oversight requires more than a human sign-off.
We're often told that keeping a human in the loop is essential for responsible AI. It sounds reassuring. AI does the work, a person reviews it, and human judgment remains in control.
But what happens when that person doesn't have enough information to question the result, time to investigate it, or a practical way to change it?
This issue explores a question I've been thinking about: Are we designing AI-assisted work to preserve human judgment, or merely the appearance of it?
FEATURE
A Human in the Loop Is Not Enough
The difference between having responsibility and being able to exercise it.
Imagine an AI tool synthesizes dozens of customer interviews into a polished summary. A researcher reviews it, agrees that the findings look reasonable, and approves the recommendations.
The workflow includes human oversight. But did the researcher have access to the underlying evidence? Could they identify where AI might have misinterpreted something? Did they have enough time to investigate an unexpected finding?
If not, what exactly did their approval establish?
This is the limitation of treating human-in-the-loop as a universal safeguard. Having a person involved doesn't automatically mean meaningful judgment is taking place.
I've been exploring a related problem through Synthetic Competence: when the quality of AI-assisted output exceeds our ability to independently understand, evaluate, or produce it.
But the responsibility doesn't rest entirely with individuals.
We can encourage people to think critically, question AI, and retain ownership of decisions. Yet if the tools and workflows around them make those things difficult, we're asking people to exercise capabilities without giving them the conditions to do so.
Three recent articles explore different dimensions of this challenge.
THREE READS: DESIGNING FOR MEANINGFUL JUDGMENT
1. Make the work understandable, not just editable
New tool shows how humans can learn with AI - University of Sydney, via Tech Xplore
Researchers developed Treellis, a 3D modeling interface that lets people see and modify the individual structures behind AI-generated designs instead of working only through prompts.
In a comparison with a conventional prompt-based tool, Treellis users explored how their models were assembled and made small, reversible changes to improve them. The findings are specific to 3D modeling, not evidence of lasting learning across other AI applications.
What interests me is the design principle: An interface can help people develop understanding, or simply make it easier to obtain a result.
We should ask whether people can inspect, explain, and repair an AI-generated outcome, rather than assuming that the ability to approve or regenerate it gives them meaningful control.
2. Build the evidence into the workflow
Research-in-the-Loop: How UXR Adapts to Development at AI Speed - Tony Alicea · Nielsen Norman Group
As AI accelerates product development, Tony Alicea proposes integrating research requirements directly into the development cycle. Instead of treating evaluation as something that happens after release, teams specify what evidence they need while defining what to build.
I particularly appreciate his distinction between measuring what we anticipated and discovering what we didn't know to look for. Instrumentation can tell us what happened. It doesn't necessarily explain why, establish causation, or reveal needs outside the questions we chose to measure.
The faster we can build, the more deliberate we need to be about learning. Otherwise, we risk accelerating decisions without improving the evidence behind them.
It's a useful direction for UX researchers and product leaders, although the proposed operating model still needs validation.
3. Give people the capacity to intervene
Interdependent conditions for successful human–AI delegation - Felipe M. Affonso and Mark Steyvers · Trends in Cognitive Sciences
This review identifies four interconnected conditions for effective human-AI delegation: understanding AI's capabilities, communicating uncertainty reliably, maintaining the human capacity to monitor, and structuring tasks so they can be checked.
The important insight is that these conditions depend on one another. Improving transparency won't accomplish much if people lack the expertise or time to evaluate what they're seeing.
The authors offer a conceptual framework, rather than a newly tested solution. But it raises an important organizational question.
Are we giving people accountability for AI-assisted decisions without giving them the resources and authority to exercise it?
A human approval step can become little more than a rubber stamp when the person responsible cannot independently assess or meaningfully challenge the result.
ONE THING TO TRY: The Meaningful Oversight Check
Think of a low-risk AI-assisted workflow you use, design, or manage. It might involve summarizing research, reviewing generated code, creating recommendations, or evaluating a decision.
Ask three questions:
- Can I understand it? Can I trace the important claims or recommendations to evidence, and recognize where uncertainty remains?
- Can I challenge it? Do I have the knowledge, independent information, and time to identify a plausible error or alternative interpretation?
- Can I change it? Can I correct, reject, or reverse the output without creating unreasonable friction?
If one of these is difficult, consider whether the problem is a lack of individual skill, a limitation of the AI, or something in the workflow itself.
Try changing one condition and see whether it makes review more meaningful. Not every task needs intensive oversight, but consequential decisions deserve more than a checkbox.
UNTIL NEXT TIME
The goal isn't to slow everything down or insist that humans manually verify every AI-generated detail. It's to be deliberate about where judgment matters, and to create the conditions for people to exercise it.
That means looking beyond individual responsibility to the design of our tools, workflows, and organizations.
I'd be interested to hear: Where have you seen human oversight become more of a formality than a meaningful opportunity to question a decision? And what would make it better?
Hit reply and share your thoughts. I'd love to learn from your experiences.
Until next time,
Lyndon
Don't just keep humans in the loop. Keep judgment within their reach.