Normal view MARC view

Human judgement is heavy tailed: empirical evidence and implications for the aggregation of estimates and forecasts

Author: Sousa Lobo, Miguel ; Yao, DaiINSEAD Area: Decision Sciences Series: Working Paper ; 2010/48/DS Publisher: Fontainebleau : INSEAD, 2010.Language: EnglishDescription: 33 p.Type of document: INSEAD Working Paper Online Access: Click here Abstract: How frequent are large disagreements in human judgment? The substantial literature relating to expert assessments of real-valued quantities and their aggregation almost universally assumes that errors follow a jointly normal distribution. We investigate this question empirically using 17 datasets that include over 20,000 estimates and forecasts. We findnd incontrovertible evidence for excess kurtosis, that is, of fat tails. Despite the diversity of the analyzed datasets as regards to the degree of uncertainty about the quantity being assessed and to the level of expertise and sophistication of those making the assessments, we find consistency in the frequency with which an expert is in large disagreement with the consensus. Fitting a generalized normal distribution to the data, we find values for the shape parameter ranging from 1 to 1.6 (where 1 is the double-exponential distribution, and 2 the normal distribution). This has important implications, in particular for the aggregation of expert estimates and forecasts. We describe optimal Bayesian aggregation with heavy tails, and propose a simple average-median average heuristic that performs well for the range of empirically observed distributions.
Tags: No tags from this library for this title. Log in to add tags.
Item type Current location Collection Call number Status Date due Barcode Item holds
INSEAD Working Paper Digital Library
PDF Available BC009226
Total holds: 0

How frequent are large disagreements in human judgment? The substantial literature relating to expert assessments of real-valued quantities and their aggregation almost universally assumes that errors follow a jointly normal distribution. We investigate this question empirically using 17 datasets that include over 20,000 estimates and forecasts. We findnd incontrovertible evidence for excess kurtosis, that is, of fat tails. Despite the diversity of the analyzed datasets as regards to the degree of uncertainty about the quantity being assessed and to the level of expertise and sophistication of those making the assessments, we find consistency in the frequency with which an expert is in large disagreement with the consensus. Fitting a generalized normal distribution to the data, we find values for the shape parameter ranging from 1 to 1.6 (where 1 is the double-exponential distribution, and 2 the normal distribution). This has important implications, in particular for the aggregation of expert estimates and forecasts. We describe optimal Bayesian aggregation with heavy tails, and propose a simple average-median average heuristic that performs well for the range of empirically observed distributions.

Digitized

There are no comments for this item.

Log in to your account to post a comment.
Koha 18.11 - INSEAD Catalogue
Home | Contact Us | What's Koha?