Measuring Polarization in High-Dimensional Data: Method and Application to Congressional Speech
February 3, 2017
We study trends in the partisanship of Congressional speech from 1873 to 2009. We define
partisanship to be the ease with which an observer could infer a congressperson's party from
a fixed amount of speech, and we estimate it using a structural choice model and methods
from machine learning. The estimates reveal that partisanship is far greater today than at any
point in the past. Partisanship was low and roughly constant from 1873 to the early 1990s,
then increased dramatically in subsequent years. Evidence suggests innovation in political
persuasion beginning with the Contract with America, possibly reinforced by changes in the
media environment, as a likely cause. Naive estimates of partisanship are subject to a severe
finite-sample bias and imply substantially different conclusions.
Matthew Gentzkow is Professor of Economics at Stanford University. He studies empirical industrial organization and political economy, with a focus on media industries. He received the 2014 John Bates Clark Medal, given by the American Economic Association to the American economist under the age of forty who has made the most significant contribution to economic thought and knowledge. He is a fellow of the American Academy of Arts and Sciences and the Econometric Society, and a former co-editor of American Economic Journal: Applied Economics. Other awards include the 2016 Calvó-Armengol International Prize, the Alfred P. Sloan Research Fellowship, several National Science Foundation grants for research on media, and a Faculty Excellence Award for teaching. He was educated at Harvard University, where he earned a bachelor's degree in 1997, a master's degree in 2002, and a PhD in 2004.