Dunning-Kruger Effect Experiment

By Yiler Huang

The Dunning Kruger effect is a term first introduced in 1999 (Kruger et al., 1999). It describes the phenomenon where skilled individuals often act unconfidently, and those who are not skilled often act confidently. This effect can be observed in many areas of our lives. Such as work, education, competitions etc. In order to prove this effect, this research team conducted an experiment. This researcher’s expectation was that people would act like how it was described in the effect. Which means those who are skilled would act more humbly, and those who are less skilled would act more confidently.

This research team designed an experiment that took form in a guessing game about facts about Taiwan. Before the game started, respondents were asked to rank themselves from 1st place to 10th in a hypothetical situation where they were playing with 9 other people. This acts as the first reference of how confident the respondents were. Respondents were told facts about Taiwan and they had to decide if each information was true or false. The questions are as follows.

1. Bubble tea was invented in Taiwan (t)
2. The first direct democratic elections in Taiwan were in 1992 (1996)
3. YangMingShan is an active volcano (t)
4. Cat cafes originated from Taiwan (t)
5. Taipei 101 is currently the 7th tallest building in the world (11)
6. Taiwan has 8 national parks (9)
7. Taiwan is one of the few countries in the world with universal health care insurance (t)
8. Taiwan has the highest proportion of convenience stores to population in the world (Korea)
9. The words of the Taiwan National Anthem were written by Sun-Yat Sen (t)

After answering the questions, respondents were asked how many questions out of ten they think they answered correctly. This acts as the second reference of how confident the respondents were.

In total, this research team collected responses from 20 subjects. All of them were people at working age around the Taipei Yuanshan MRT station. Their genders would not be discussed in this article as the amount of male and female respondents collected were not equal or close. This researcher came up with two ways to interpret the data. One is comparing which place they would be competing with 9 people, with the number of answers they got right. The other one is comparing how many questions they thought they did right, with the number of right answers they got right. Details are shown in the table below.

The results of this experiment were rather surprising, as it didn’t match with this researcher’s hypothesis very much. Looking at how the respondents ranked themselves, 12 of them ranked themselves worse than the 5th place (group A), and 8 of them ranked themselves better or equal to the 5th place (group B). On average, group A answered around 4.9 questions right, and group B answered around 4.75 questions right. Even though the less confident group A does seem to perform better than group B, this researcher believes that a difference of 0.15 is not significant enough to prove this effect. If we were using the second method -comparing how many answers the subjects thought they got right with the actual amount that they got right-, people who thought they answered 4 or less questions right got around 5.1 questions right on average, and those who thought they answered 5 or more questions right got around 4.7 questions right on average. We can see that the difference is more noticeable using the second method, but this researcher still believes a 0.4 difference is rather subtle. Though we could see the Dunning Kruger effect acting on these results, the differences are not very apparent at first glance.

In conclusion, it seems like the Dunning Kruger effect does exist and it affects people’s expectation for their performances. However, this experiment showed that the difference is not very significant in the guessing game this research team designed. This researcher believes that there are a few reasons why. One being that in a true/false guessing game, there is still a fair amount of chance that the respondents could get the questions right just by guessing. Another reason being that there were only 9 questions that were presented to the respondents, and it might not be enough if we were to test people’s knowledge level on a certain topic. In the end, the effect is proven, but only on a very subtle level. If we want to learn more about this effect, further research would have to be done.

Kruger, Justin; Dunning, David(1999). “Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments”. Journal of Personality and Social Psychology.