Today, I am joined by my friend, Mike Lauria, to interview Gary Klein, PhD. Dr. Klein is a masterful cognitive psychologist. He is known for many groundbreaking works, including: the Recognition-Primed Decision (RPD) model to describe how people actually make decisions in natural settings; a Data/Frame model of sensemaking; a Management by Discovery model of planning to handle wicked problems; and a Triple-Path model of insight. He has also developed several research and application methods: The Critical Decision method and Knowledge Audit for doing cognitive task analysis; the PreMortem method of risk assessment; the ShadowBox method for training cognitive skills. He was instrumental in founding the field of Naturalistic Decision Making.
The Books
Sources of Power
This is the one that got Mike and I started as Klein Fanboys
Streetlights and Shadows
The absolute best compilation of Dr. Klein's decision-making concepts that are directly applicable to medicine
Seeing What Others Don't
Next up on my reading list
Recognition Primed Decisionmaking
Sites and Links
Articles Mentioned in the Show
- Kahneman D, Klein G. Conditions for intuitive expertise: a failure to disagree. Am Psychol. 2009 Sep;64(6):515-26.
- Can We Trust Best Practices? Six Cognitive Challenges of Evidence-Based Approaches. Journal of Cognitive Engineering and Decision Making
Additional Related Stuff
- Effect of availability bias and reflective reasoning on diagnostic accuracy among internal medicine residents.
- Effects of reflective practice on the accuracy of medical diagnoses. Going fast might not induce more error, it's about experience and if you have the patterns to recognize:
- Disrupting diagnostic reasoning: do interruptions, instructions, and experience affect the diagnostic accuracy and response time of residents and emergency physicians? Slowing down doesn't help. Slow is just slow. Smooth is FAST, and smooth is about economy of cognitive resources and movements
- The relationship between response time and diagnostic accuracy.
- The etiology of diagnostic errors: a controlled trial of system 1 versus system 2 reasoning.
- The Checklist Manifesto: How to Get Things Right
- Descartes' Error: Emotion, Reason, and the Human Brain
Now on to the Podcast:
Podcast: Play in new window | Download (Duration: 53:19 — 48.9MB) | Embed
Subscribe: Apple Podcasts | Android | Google Podcasts | Spotify | RSS | More
Scott Weingart
Latest posts by Scott Weingart (see all)
- EMCrit 265 – ECPR 2.0 (ECMO CPR) - February 5, 2020
- EMCrit – Midlines Part 2-Explosion with Rory Spiegel - February 1, 2020
- EMCrit 264 – Case Discussion of Combined CCB and BB Overdose - January 26, 2020
Two comments: First, I was surprised to hear in a podcast that discussed, in part, cognitive errors: “yes that’s perfect that’s exactly what I want to hear more of this confirms everything I already believe, confirmation bias ho!!” I’m not sure if that’s exactly the quote but that’s what I remember. I enjoyed the irony. Second, this was a great podcast with great concepts about improving our cognitive skills and decision-making. Unfortunately, emergency medical education is not set up in such a way as to allow our expert educators to provide the feedback that is much needed. And I believe it would take a drastic paradigm shift. I would like to illustrate this with an analogy: my sport was Olympic taekwondo (coming to NBC August 17 by the way!). How did we train? In front of a coach, we, almost exclusively, practiced in a very sports specific and performance based way so that we could be provided with immediate feedback. The drills and practice were as realistic and approximate to competition conditions as was possible. This was invaluable. We did not sit around talking about our sport or our kicks or watching power points or having lectures about strategy. We… Read more »
This is one of the most enlightened and thoughtful
comments I’ve seen – long winded and all! Completely agree with Chad’s take on this. Imagine the rapidly accumulated experience and reinforcement from doing such a thing and the ever expanding number of ‘patterns’ the resident could draw on. In my opinion it’s this type of learning experience that contributes to a doc’s intuition.
MEZZO Podcast #179 – Progressive- & Deep-House #zene #djmix #letöltés #edm #trance #house… + ?lave olarak Out now! #MEZZO #Podcast #179 by @070MENNO #listen to a #deep & #progressive #house #…
The effectiveness of RPD is highly dependent on knowledge and experience in a particular field.
To operate effectively in reflexive System I requires considerable prior investment in analytical System II e.g. broad knowledge acquisition, exposure to multiple situations, reflective practice, mental and practical rehearsal, group discussion and debrief, mentorship etc etc.
Novices tend to expend more effort and time in System II during clinical practice because knowledge and experience limits their ability to recognise and react to varying situations. They are also more likely to make errors if they prematurely move to System I.
Chad’s suggestion above is really a means of accelerating the process for novices to simulate their exposure to a range of context relevant situations in a structured, measured and safe way so that they can more reliably employ System I in real-life.
[…] EMCrit episode of Olympic proportions. Mike Lauria leads an interview with cognitive science giant, Dr Gary Klein, in which expert intuition is put back on the podium. Recognition primed decision making, how […]
Great episode and thank you. I always enjoy hearing from Mike Lauria, He’s a thinker and a natural giving his thoughts and insights. Dr. Klein gave some great examples with firefighters, but I wanted to add a few ideas that our military uses, particularly the Army Rangers. I think some of these ideas are applicable to medicine. The first is the idea of an After Action Review. After every training exercise, mission or action the Army uses an AAR, or After Action Review. From the greenest private, to the mission commanders, every one is expect to answer three question in a group session format. What went right? What went wrong? How can we do it better next time? I think this process would be helpful for codes, emergent procedures and sentinel events. It gives every one perspective. The second is the idea of rehearsal. Every mission (time permitting) is rehearsed ad-nauseum. This goes to the point of creating a life size mock up of the objective and walking through it, to creating sand tables where everyone can get a big picture of the objective, and their position in relation to others. This rehearsal will continue to the point where the… Read more »
beautiful concepts, Frank. I think I have been blacklisted for ACEP 2016 or least did not make the cut.
As somebody who both teaches disaster science & ambulance science, the interview with Dr Klein was amazing! (As was your SMACC Dublin presentations).
One of the best resources of military research on the web is the Air War College website, with thousands of documents relevent to cognition, decisionmaking, etc. It can be found here: http://www.au.af.mil/au/awc/awcgate/awcgate.htm
Within that site is also two specific documents that Dr Klein wrote on Decisonmaking Games (DMG).
-Decision-Centered MOUT Training for Small Unit Leaders
-Evaluating an Approach to MOUT Decision Skills Training
One describes the process of designing DMGs, and the other has the template for the design in the appendix. Both have been incredibly useful in thinking about how to teach learners RPD. “You get good at making decisions when you learn by making lots of decisions….”
Scot Phelps
City University of New York (Paramedic Degree Program),
Adelphi University (Emergency Services Degree Program),
New York Medical College (MPH concentration in Emergency Management, fmrly Emergency Medical Services),
The Emergency Management Academy (AmbulanceScience.org, DisasterScience.org)
fantastic resources, Scot!
Scott – really engaging podcast and a topic that deeply fascinates me. It so fascinated me that I developed a software to enable deliberate practice and train cognitive decision making in medicine! Seems very much like the shadow box concept. With this software, trainees can practice cases on a cloud based computer platform over and over until their approach matches that established by experts. Educators can create a large variety of new cases very easily. The software is called CaseMasterIQ and it was co-developed by Education Management Solutions website http://www.casemasteriq.com You can sign up for free access there since we are still in development.
Check out the VodCast training video that I put together for my EM residents. https://emtoxcast.com/2016/06/
Loved to hear what you think about it if you have time and if not keep calm and blog on — really enjoy all the great stuff you do here at EMCrit!
Cheers,
Rich
Richard J Hamilton MD, FAAEM, FACMT, FACEP
Professor and Chair
Department of Emergency Medicine
Drexel University College of Medicine
Rich,
Looks v. interesting. Just requested access.
That sounds awesome Dr. Hamilton. I would love to check it out. Do you think this software package could be customized or changed for other level providers like nurses and paramedics?
-Mike
For listeners wanting more of Gary Klein
https://m.youtube.com/watch?v=J_N9woWTHGA&autoplay=1
Do people think that emergency medicine is suitably predictable to allow intuition? I wonder how close it is to the stockmarket (that even Gary Klein admits is not a suitable environment for intuition) insofar as being too chaotic and unpredictable?
Hey Luke, You bring up a valid point which and ask an excellent question. There are a number of criteria that actually make recognition primed decision making (intuitive decision making) viable: 1. Observable characteristics in a system 2. Quantifiable changes to a system 3. Perturb the system with confidence (i.e. know your perturbation is causing a the change) 4. Variables reliably predict outcomes 5. Natural systems vs socially produced Dr. Klein talks about these in the article he published with Kahneman (see articles above). Resuscitation in the ED, in the field, or in the ICU is, granted, chaotic sometimes. However, the system is at least governed by certain things that make it very much unlike a stock market. You can measure blood pressure, heart rate, etc. You can give pressor and watch the heart rate/BP increase. You can see the effects of defibrillation. You also know what will happen if a massive pneumothorax or profound hypotension goes untreated. I’ll be the first to admit there might some exceptions to the some of the aforementioned examples (e.g. giving pressor and NOT seeing the changes you want in hemodynamics). But, that’s kind of the point. You know the normal patterns in the… Read more »
Hi Mike,
Thanks very much for that clear answer. I think you have convinced me of recognition primed decision making in critical care type patients. Clearly these patients are the focus of emcrit so this fits well here.
I wonder if the reliability of recognition primed decision making decreases for the those who are less unwell e.g sick but not needing resuscitation / who aren’t significantly compromised physiologically. (I work in an Emergency Department where this is the vast majority of patients: pains in various sites, no clear cause, only minor vital sign abnormalities if any).
I suspect medicine for this population is more chaotic and lacks most of the criteria you listed that make intuition reliable. Many of the inputs are subjective e.g. pain, dyspnoea, diagnostic feedback is slow or non existent etc. In this context I wonder how much physician gestalt is due to time since lunch / the last five patients / cognitive bias etc, much like the stockbrokers. My apologies that this is outside the context of emcrit, am just selfishly relating it to my own practice in a far corner of the world!
Thanks again,
Luke
An interesting commentary on NDM from our Canadian medical educators
https://icenetblog.royalcollege.ca/2017/02/03/ed-theory-made-practical-naturalistic-decision-making/
[…] EmCrit – Interview with Gary Klein: Fantastisk intervju med Gary Klein, psykolog och pionjären bakom “Recognition-Primed Decision (Making)“. Höggradigt relevant för alla som jobbar inom akutsjukvårdom om hur man kan utveckla sin förmåga till beslutsfattande i kritiska situationer. För en utförligare sammanfattning av Kleins tankar rekommenderas boken “Streetlights and Shadows” varmt. […]