Tuesday, March 19, 2013

The Dunning Kruger Effect, or why you shouldn't listen to bloggers

This should make you paranoid - but don’t worry, if it does, you’re fine. If it doesn't make you paranoid, it should. Confused? That’s a good thing!

In 1999 two Cornell University psychologists named David Dunning and Justin Kruger conducted 
a simple study comparing what students thought they knew, and what they actually knew. The results provide some deep insights into human behavior, and a cautionary tale to those willing to engage in some self-criticism.
The Study
Students were tested on humor, logic, and grammar. They were given an exam, and then asked to evaluate their own performance. Actual test scores were compared with the self-evaluation. Logic and grammar are easy to test objectively, but you may wonder about testing a subjective subject like humor. The study came up with a pretty good metric; the student’s answers were compared with a panel of experts - professional comedians.

 The worse a person was at the test; the more they overestimated their abilities. The absolute worst performers rated themselves nearly on par with the best performers. The worst performers also rated themselves above all but the very top performers.

It Gets Worse!
In a follow up study students were asked to grade the work of their peers on the test. Not surprisingly, the worse they scored, the worse were at judging the skills of others. In other words, they didn’t have the expertise to recognize real expertise in others.

Dunning and Kruger provide some insightful analysis of the results. Read the paper 
HERE. You probably don’t have the time to read it yourself, so here’s what I got out of it:
Four Key Points
1. Incompetent people think they are competent
2. Very competent people doubt their own abilities.
3. Incompetent people can’t recognize competence in others.
4. You have been, and will be, one of those people sometimes.
"Does this remind you someone you know?"

Actually, most people immediately recognize the Dunning Kruger effect from abundant incidents in their own life: a terrible boss, a pedantic teacher, an ignorant coworker who thinks he knows everything. We’ve all seen it in small children – they read a book on dinosaurs and proudly announce that they know everything about dinosaurs. Of course it’s OK in kids because we know their experience is so limited. If kids knew how much they had to learn, they would never try anything new. It’s a great coping mechanism for someone just starting out. A friend of mine once said “Kids are a lot like people”. When we see the same thing in adults, it is no longer cute.
"Could it apply to me?" 

Paradoxically, most of us never see ourselves as the incompetent one.  Of course, you and I have never fallen prey to the Dunning Kruger effect.

A few years ago, I was brought into a new project. I immediately looked around and saw that “everyone was doing it wrong!”  Everyone was making a big fuss about what was really very simple. Not so many years ago, I would have confidently explained to everyone within earshot how to fix everything.  Luckily, I kept my mouth shut. After a few months, I started to see the complexity of the project. Four years later I’m getting a sense of all the things I don’t know. Think how much I won’t know after ten years! It makes me appreciate the patient people who took the time to wean me from my ignorance.
"That explains why incompetents rate themselves too high. Why do competent people rate themselves too low?"

The explanation given by Dunning and Kruger is the “
False Consensus Effect”.  People tend to overestimate how much other people agree with them. They know enough to recognize the limits of their own knowledge. Competent people assume that everyone else is competent like them, making them average. The good news here is that when good performers are asked to rate the work of others; they often recognize this incorrect assumption.
How to Use This Information

This study is packed with insights. Here are some of the actionable ones:
  • The first, and obvious thing is don’t believe people’s self-assessment of their abilities. If they are incompetent, they don’t know it. If they are well above average, they may not know that either.
  • Recognize that the effect applies to you. Be humble. Test your assumptions before make a fool of yourself. 
  • Recognize that incompetent people don’t know they are incompetent. Just showing them the facts won’t convince them, because they won’t recognize the facts. You’re going to have to educate them before they can understand. 
Beware of Bloggers

Some bloggers are legitimate experts, but most aren’t. Sometimes that’s OK. Bloggers are usually more like journalists than experts. Journalists fill the role of publicizing and popularizing the work of others. This is important work, but it shouldn’t be mistaken for genuine expertise.

For complex and difficult issues, journalists (and especially bloggers) often get the facts wrong. Most real journalists are held by their editors and publishers to minimum standards.  Unfortunately, editors aren’t experts, and publishers are usually more concerned about issues like profitability. Bloggers don't have any standards at all.

If you’ve read this far, ask yourself about the author of this article. I’m not a PhD in psychology and I’ve never done any original research in the field. I can only hope that my esteemed readers will take me to task if I get the facts wrong.

No comments:

© 2008 Raoul Rubin