The Judgment Problem: AI, Expertise, and the Dissolution of Knowledge

Sitting in a risk review at a big financial institution not long ago, I watched something that just won't leave me alone. A junior analyst had run a counterparty exposure question through an AI system before the meeting. The output looked perfect. It was structured right, hedged correctly, fluent in the language of credit risk. He walked the committee through it with this quiet confidence, like he had done the work right.

Two things happened.

The senior risk officers in the room couldn't immediately spot the flaw in the analysis. But they knew something was wrong.

There were just a few seconds of silence. That specific kind of quiet that settles in rooms like that. Then, one senior officer found the thread. She pulled it, and the whole thing unraveled in maybe ninety seconds.

I’ve been stuck on those ninety seconds ever since.

The public debate about AI and expertise mostly gets stuck on the wrong question. Will machines eventually beat human judgment? Will doctors, lawyers, engineers just vanish? Those questions have drama, sure. But they’re probably decades away, maybe impossible to answer.

The real shift, the one that’s happening right now, isn't about capability. It’s about perception. Large language models aren't replacing expertise. They are just dissolving the social distance that made expertise seem clear to other people. That’s a different, and frankly, harder problem.

What expertise actually is tends to get flattened in these conversations. A senior risk officer doesn’t just have more data than a junior analyst. She has something else. Years of seeing actual failure. She has learned pattern recognition heuristics that let her navigate uncertainty. She knows which numbers are structurally suspicious before she can even say why. That knowledge lives below the surface. It built up slowly, through feedback loops demanding real stakes, real consequences. You can’t pull it out of a database. It was never stored that way.

Transformer models work on a different level entirely. They learn the surface stuff. The vocabulary. The hedging. The way a professional judgment is usually structured. Because they are trained against what people expect, they get better at satisfying expectations. They optimize for plausibility. Plausibility isn't correctness. But when you’re under real pressure, without deep domain knowledge yourself, the two start to blur at first contact.

I want to spend a minute on the mechanical clock. I think that’s more useful than the usual comparisons we make.

Before standardized timekeeping, time was negotiated locally. The guild master’s sense of the working day held real authority. Not because he was more trustworthy. Because he controlled the measurement. The power imbalance collapsed not because workers suddenly got smarter. It happened because the infrastructure underneath everything shifted.

What followed wasn't smooth. It was a century of social re-organization. Ugly places. Labor conflicts. Institutional collapses that nobody saw coming when the first clock tower went up. The people who built it were solving a coordination issue. The destabilization was just a side effect. It took generations to absorb it. And the costs just get dumped unevenly on the people who couldn't see it coming.

When I looked across financial services and tech governance, something came up from talking to the senior people. The anxiety wasn't about automation in the usual sense. It was sharper. Harder to name. It was about the difficulty of justifying why human judgment should matter more than a system that spits out coherent, confident analysis, and does it cheaper.

Nobody I spoke with felt their expertise was shrinking. What they described, in different ways, was a loss of legibility. The expertise itself was fine. The audience for it was just getting harder to hold onto.

This is the institutional mess without any clean answer. Licensing. Credentials. Quietly.

The failure mode we ignore isn't the dramatic one. A system that spits out plausible-sounding expert analysis most of the time, but is wrong in ways that are structurally impossible to spot, isn't dangerous because of catastrophic errors. Those show up. It’s dangerous because those plausible outputs pile up across millions of decisions. They shift behavior. They erode respect long before anyone has the data to name what’s actually happening.

What actually survives as deep knowledge isn't the credential. It’s something harder to teach. Harder to automate. The capacity to question a confident answer. To notice the assumptions an explanation is quietly carrying past you. To ask the question that exposes the system’s limits, not just its center. These skills are learnable. They aren't what we select for, and they aren't what AI systems are built to develop in the people using them. Those two facts are worth sitting with.

The officer who pulled that thread did it because she spent twenty years learning to distrust fluency. She had been wrong in that exact register early on. She remembered what plausible-but-wrong felt like from the inside. That memory was the tool. You don't learn that from a training set. You learn it by being held accountable for outcomes over a long stretch of time, in an environment that won't let you quietly move on from your mistakes.

The real question that keeps me awake isn't whether AI will get smarter. They will. It’s whether we’ll manage to build the next generation of experts in a way that still allows for that kind of formation. Or if we’ll just optimize the pipeline for speed and fluency. And some years from now, we’ll realize we’ve produced professionals who are good at working with AI outputs, but terrible at knowing when to stop trusting them.

That isn't a technology problem. It’s a judgment problem. And judgment, so far, is the one thing that doesn’t transfer.

Aditya Vikram Kashyap is currently Vice-President at Morgan Stanley, New York. He’s a technology leader, known for enterprise AI and building ethical cultures. These thoughts are mine alone. They don’t necessarily reflect News18’s position.