Philip E. Tetlock

Philip Eyrikson Tetlock (born March 2, 1954) is a Canadian-American political psychologist and writer, and is currently the Annenberg University Professor at the University of Pennsylvania, where he is cross-appointed at the Wharton School and the School of Arts and Sciences. He was elected a Member of the American Philosophical Society in 2019.

He has written several non-fiction books at the intersection of psychology, political science and organizational behavior, including Superforecasting: The Art and Science of Prediction; Expert Political Judgment: How Good Is It? How Can We Know?; Unmaking the West: What-if Scenarios that Rewrite World History; and Counterfactual Thought Experiments in World Politics. Tetlock is also co-principal investigator of The Good Judgment Project, a multi-year study of the feasibility of improving the accuracy of probability judgments of high-stakes, real-world events.

Biography

Tetlock was born in Toronto, Canada and grew up in Winnipeg and Vancouver. He studied psychology at the University of British Columbia, where he received a B.A. in 1975 and an M.A. in 1976. He worked with Peter Suedfeld during his master's study on content analysis of diplomatic communications. He then moved to the US and conducted his doctoral research at Yale University, obtaining his PhD in psychology in 1979. His thesis was supervised by Phoebe C. Ellsworth.

Tetlock has served on the faculty of the University of California, Berkeley (1979–1995, assistant professor), the Ohio State University (the Burtt Endowed Chair in Psychology and Political Science, 1996–2001) and again at the University of California Berkeley (the Mitchell Endowed Chair at the Haas School of Business, 2002–2010). Since 2011, he has been the Annenberg University Professor at the University of Pennsylvania.

Tetlock has received awards from scientific societies and foundations, including the American Psychological Association, American Political Science Association, American Association for the Advancement of Science, International Society of Political Psychology, American Academy of Arts and Sciences, the National Academy of Sciences and the MacArthur, Sage, Grawemeyer, and Carnegie Foundations.

Work

Tetlock has published over 200 articles in peer-reviewed journals and has edited or written ten books.

Tetlock's research program over the last four decades has explored five themes:

the concept of good judgment (with special emphasis on the usefulness of forecasting tournaments in assessing one key component of good judgment: accuracy);

the impact of accountability on judgment and choice;

the constraints that sacred values place on the boundaries of the thinkable;

the difficult-to-define distinction between political versus politicized psychology; and the usefulness of hypothetical-society experiments in disentangling fact and value judgments of the impact of competing policy proposals.

The Good Judgment Project

In his early work on good judgment, summarized in Expert Political Judgment: How Good Is It? How Can We Know?, Tetlock conducted a set of small scale forecasting tournaments between 1984 and 2003. The forecasters were 284 experts from a variety of fields, including government officials, professors, journalists, and others, with many opinions, from Marxists to free-marketeers.

The tournaments solicited roughly 28,000 predictions about the future and found the forecasters were often only slightly more accurate than chance, and usually worse than basic extrapolation algorithms, especially on longer–range forecasts three to five years out. Forecasters with the biggest news media profiles were also especially bad. This work suggests that there is an inverse relationship between fame and accuracy.

As a result of this work, he received the 2008 University of Louisville Grawemeyer Award for Ideas Improving World Order, as well as the 2006 Woodrow Wilson Award for best book published on government, politics, or international affairs and the Robert E. Lane Award for best book in political psychology, both from the American Political Science Association in 2005. The expert political judgment project also compared the accuracy track records of "foxes" and "hedgehogs" (two personality types identified in Isaiah Berlin's 1950 essay "The Hedgehog and the Fox"). "Hedgehogs" performed less well, especially on long-term forecasts within the domain of their expertise.

These findings were reported widely in the media and came to the attention of Intelligence Advanced Research Projects Activity (IARPA) inside the United States intelligence community—a fact that was partly responsible for the 2011 launch of a four-year geopolitical forecasting tournament that engaged tens of thousands of forecasters and drew over one million forecasts across roughly 500 questions of relevance to U.S. national security, broadly defined.

Since 2011, Tetlock and his wife/research partner Barbara Mellers have been co-leaders of the Good Judgment Project (GJP), a research collaborative that emerged as the winner of the IARPA tournament. The original aim of the tournament was to improve geo-political and geo-economic forecasting. Illustrative questions include "What is the chance that a member will withdraw from the European Union by a target date?" or "What is the likelihood of naval clashes claiming over 10 lives in the East China Sea?" or "How likely is the head of state of Venezuela to resign by a target date?" The tournament challenged GJP and its competitors at other academic institutions to come up with innovative methods of recruiting gifted forecasters, methods of training forecasters in basic principles of probabilistic reasoning, methods of forming teams that are more than the sum of their individual parts and methods of developing aggregation algorithms that most effectively distill the wisdom of the crowd.

Among the more surprising findings from the tournament were:

the degree to which simple training exercises improved the accuracy of probabilistic judgments as measured by Brier scores;