How to Decide: Gary Klein on Expertise, Intuition, and the Limits of AI

You make hundreds of decisions a day. Most of them invisibly. A few of them under real pressure, with incomplete information and no clear right answer.

So how do the people who do this for a living like firefighters, surgeons, military commanders, and get it right when the stakes are highest?

That's the question Dr. Gary Klein has spent his entire career answering. Not in a lab. In the field. With people whose next call might be life or death.

Gary is a cognitive psychologist, a Senior Scientist at MacroCognition LLC, and the Chief Scientist at ShadowBox LLC. He's one of the founding figures of naturalistic decision making, the study of how people actually decide in the real world, under time pressure and uncertainty. He built the Recognition-Primed Decision model, which has been incorporated into Army and Marine Corps doctrine. He created the PreMortem method of risk assessment, endorsed by Nobel Prize winners Daniel Kahneman and Richard Thaler. 

He's the author of several influential books, including Sources of Power, The Power of Intuition, Streetlights and Shadows, Snapshots of the Mind, and Seeing What Others Don't, a fascinating deep dive into how insight actually works.

Malcolm Gladwell put it simply: "No one has taught me more about the complexities and mysteries of human decision-making than Gary Klein."

In this conversation, we get into everything from how Gary personally works through a tough decision to when you should, and shouldn't, trust your gut. We cover the value of first-person expertise, the difference between knowledge and knowing, how to use a pre-mortem, and why more information doesn't necessarily mean better decisions. Then we spend time on AI: what happens when people start outsourcing their thinking, and what might get lost in the shuffle.

I also ask him to audit my use of his framework for managing uncertainty  because there's a lot of that going around right now.

Some highlights from the episode:

• 02:35 The White House Situation Room (and why he can't talk about it)

• 05:17 Writer's block, pen and paper, and how Gary structures his thinking

• 07:37 Walking through a real decision: the medical scenario

• 10:53 Intuition: when to trust it, when to question it

• 13:00 Pattern matching, mental simulation, and the Recognition-Primed Decision model

• 18:00 The AI concern: outsourcing decisions and eroding expertise

• 18:42 The pre-mortem: how it works and why Nobel Prize winners endorsed it

• 22:35 The 80/20 of decision making: build experience and frame the problem

• 27:12 AI and the younger generation: old fogey worry or real risk?

• 31:49 Why curiosity about failure is the thing AI can't replicate

• 33:06 Tacit knowledge: the invisible layer AI can't scrape

• 39:07 Five sources of uncertainty — and tools for managing them

• 42:36 Wrapping up: the cognitive dimension and what makes humans indispensable

We go from the mechanics of expert decision making to a surprisingly urgent question: in an age of AI, what happens to the skills you never knew you were building?

Enjoy!