Forecasting and group decision making

The accuracy of predictions of events such as the outcomes of political elections can be improved by a variety of methods for combining the predictions of individual forecasters. We are conducting a study with over a thousand of forecasters, each making hundreds of forecasts, to study how best to elicit, weight, and combine the judgments of many experts. We gave each forecaster a variety of intelligence and personality tests, and then randomly divided them into different "conditions" with different training (e.g. in probability and forecasting) or with different ways of interacting (no interaction, teams, prediction markets, ...). The results show the benefits of information exchange, as in teams and prediction markets. We are also developing new statistical algorithms for aggregating the judgments by many individuals. Issues include how to model the decision processes of individuals and groups, how to optimally transform the probability estimates, and how to best weight individual forecasts based on forecaster confidence, expertise and cognitive style. This work is supported under the IARPA ACE project which aims "to dramatically enhance the accuracy, precision, and timeliness of forecasts for a broad range of event types, through the development of advanced techniques that elicit, weight, and combine the judgments of many intelligence analysts." Joint work with a large team, including Barb Mellers, Phil Tetlock, and a host of others.

For more information

Many of these papers can be found here.