Mass media are crucial to representative democracy. This has been acknowledged throughout modern political history, from the founding of democratic republics to the present day. The accuracy of media coverage is central to effective accountability and responsive government. Indeed, it is difficult to imagine how large-scale democracy could work without reasonably accurate coverage of current affairs.
We are currently in the midst of both public and scholarly debate about the nature and quality of media coverage in the United States. First, there is a growing body of research on journalists’ misrepresentation—and the public’s corresponding misunderstanding—of scientific issues such as global warming and vaccinations (Boykoff and Boykoff 2004; Speers and Lewis 2005). Second, there is a growing scholarly literature suggesting that even were media coverage to portray issues accurately, our interpretation of that information is conditioned by our own pre-existing beliefs, and that this enhanced in an increasingly high-choice media environment (e.g., Prior 2007; Garrett 2009; Stroud 2011; Arceneaux and Johnson 2013; van Aelst et al. 2017). Third, scholars, policymakers, and the public have expressed concerns that increasing ideological polarization may amplify the impact of selective exposure and the systematically biased media coverage that may accompany it (e.g., Abramowitz 2010; Ura and Ellis 2012).
Media accuracy is a central theme in our recent book, Information and Democracy, in which we empirically assess the correspondence between the media’s coverage of federal policy changes in the United States and the actual policy changes enacted by legislators. Our findings suggest real variation in the accuracy with which different media outlets report on national policy changes. This has important implications for the functioning of representative democracy. In short: citizens may be differentially informed about government action, and thus base their political decisions on different understandings of what the government is (or is not) doing.
The story of media influence on public policy issues is neither entirely negative nor entirely positive; it is not black and white, but gray. On the one hand, consternation over the accuracy of American mass media news has reached a fever pitch in the wake of Russian interference in the 2016 US presidential election (e.g., Shane and Mazzetti 2018), loose interpretations of facts by the Trump administration (e.g., Salam 2018), ongoing claims of and concerns about “fake news” (e.g., Allcott and Gentzkow 2017; Kucharski 2016), and the persistent “Big Lie” about the 2020 election results. There have been few moments since the rise of modern mass media during which information about current affairs was so suspect. But concerns about inaccuracies in media coverage are not unique to the current moment. There have been previous periods of heightened concern about media accuracy—during the Vietnam War, for instance (Delli Carpini 1990). And the quality and accuracy of media coverage have always varied over time and across issues and media outlets. Consider, for example, the large body of evidence highlighting systematic biases in past crime reporting (e.g., Altheide 1997; Soroka 2014).
On the other hand, media can and sometimes do provide the information required for the public to effectively guide politicians and hold them accountable. Despite flaws in media coverage, previous work makes clear that individuals can and do learn about what government does from news content (e.g., Barabas and Jerit 2009; Barabas et al. 2014; Druckman 2005; Eveland, Seo, and Marton 2002; Jerit, Barabas, and Bolsen 2006; Neuner, Soroka, and Wlezien 2019).
Measuring media accuracy
The degree to which media impede or inform the public’s understanding of policy change likely varies across policy issues and media outlets. The figure below illustrates media accuracy across three major policy domains: federal spending on defense, welfare, and health. We focus on these issues because in each case, past work has found evidence that the public responds “thermostatically” to policy, adjusting its preferences for more policy downward in response to policy increases. In order for this to happen, media must be providing some accurate information. Defense, welfare, and health spending thus offer a kind of best-case scenario for identifying media accuracy.
How exactly do we measure media accuracy? We provide all the necessary information here. The details involve statistical analysis of the relationships between news coverage of changes in federal spending and the actual changes in the budget. But the most important aspect, our estimate of media coverage of public policy change—what we refer to as the “media policy signal”—is relatively straightforward. We rely on computer-automated dictionaries to identify sentences in media content about policy, spending, and direction. “Layering” these dictionaries reliably identifies and codes sentences about spending change in the different areas—for example, “Reagan increased spending on defense.” We also use machine learning in place of dictionaries, although it produces very similar results (for more, see Information and Democracy). We then aggregate sentences about spending change to produce the media policy signal and assess the over-time relationship between that signal and actual spending change in the previous, current, and upcoming years.
The figure above shows the results for each of 17 newspapers and six television networks in a pooled estimation combining federal spending—and media coverage of spending—on defense, welfare, and health. It focuses on data beginning in 1995, the year by which data for all but one of our newspapers are available (cable television data begin several years later). Our measure is “standardized” so that a score of 1 suggests a very strong connection between spending and media coverage—that is, very high accuracy. Scores closer to 0 suggest progressively weaker connections between spending and media coverage—that is, increasing degrees of inaccuracy.
The results are telling. Even in these three “best case” domains, accuracy varies considerably across media outlets. Newspapers like the Houston Chronicle and the St. Louis Post-Dispatch have provided, on average, highly accurate coverage of budgetary policy in defense, welfare, and health. The Arkansas Democrat-Gazette and Minneapolis Star-Tribune, by contrast, have provided only minimally accurate coverage in these domains. More to the point: an American getting most of their news from the Washington Post, for instance, is likely to get more-accurate information about federal government action in these domains than is an American getting most of their news from USA Today.
More Americans get their news from television than from newspapers, so it is notable that accuracy scores for TV news are in most cases better than accuracy scores for newspapers. The three cable networks—MSNBC, CNN, and Fox News—and NBC have comparatively high accuracy scores in these three policy domains. There are not significant differences across these networks, though others, CBS and especially ABC, provide coverage more in line with some of the moderate-to-low accuracy newspapers.
These aggregated results mask differences across different domains, of course. More detailed analyses highlight larger variation in the accuracy of the cable networks in welfare and health, for instance, and much lower accuracy scores for most media outlets in other policy domains—education and the environment. Note that these scores speak to the accuracy of years of media coverage on budgetary policy. Media outlets may produce highly accurate or wholly inaccurate coverage of a single event; what we capture here is the accuracy of their reporting of spending change, on average, over time.
Ours is a somewhat different perspective than is common in the literature on misinformation. Particularly in recent years, misinformation in the media about issues like climate change, health-care reform, and critical race theory has motivated important scholarship and commentary. Our approach adds to this conversation by showing that media accuracy fluctuates considerably even outside of these “hot button” issues. Where we get our news from matters. Misinformation occurs not just through flawed reporting of the Big Lie, but through the more routine reporting of policy changes.
It can be difficult for news consumers to know when information is accurate, of course. One approach to becoming better informed may be for consumers to combine information from multiple sources, a task made easier by online news aggregators and social media. Analyses detailed in Information and Democracy suggest some risks with doing so using social media. Taken as a whole, the “media signal” on Facebook—based on policy stories posted by news outlets on that platform—can be relatively accurate, but weighting stories based on likes and reposts produces a very low accuracy signal. The priorities and interests of social media users differ from the priorities of major news organizations. As a consequence, a social media-filtered version of news is not likely to produce accurate information about policy change.
At mediaaccuracy.net, we provide more detailed measures of volume and accuracy of news coverage across given policy domains for each of the 17 major US newspapers and six television networks examined here. Our work provides news consumers with a general sense of which news outlets have tended to be more-or-less accurate about federal spending policy in the past. Consumers can make decisions about their news courses based on these data. News producers can also use these data to adjust their policy coverage. Inaccurate policy coverage need not (and probably most often does not) come from editors and journalists consciously avoiding the truth. It more likely is a function of factors such as time pressures in reporting, financial incentives to seek audiences drawn to flashier and/or local content (keeping in mind that we are assessing coverage of federal policy), or possibly false information from political candidates or elected officials. Even a small rebalancing of newsroom priorities and assignments may produce coverage that augments media accuracy, public responsiveness, and government accountability.
Communication and political science scholars can apply our approach to media accuracy to other policy domains, and indeed to any specific policy change in which there is clear, measurable government action. If a policy action taken by legislators receives accurate media coverage, then there is at least the possibility of an informed public. If policy actions do not receive accurate media coverage, however, there is little reason to expect that the public can effectively guide policymakers or hold them accountable.
The authors are grateful to Molly Laas and Shannon McGregor for comments on earlier drafts.
[workscited]
Abramowitz, A.I. 2010. The Disappearing Center: Engaged Citizens, Polarization, and American Democracy. New Haven: Yale University Press.
Allcott, Hunt, and Matthew Gentzkow. 2017. “Social Media and Fake News in the 2016 Election.” Journal of Economic Perspectives 31 (2): 211–36.
Altheide, David L. 1997. “The News Media, the Problem Frame, and the Production of Fear.” Sociological Quarterly 38: 647–68.
Arceneaux, Kevin, and Martin Johnson. 2013. Changing Minds or Changing Channels? Partisan News in an Age of Choice. Chicago: University of Chicago Press.
Barabas, Jason, and Jennifer Jerit. 2009. “Estimating the Causal Effects of Media Coverage on Policy-Specific Knowledge.” American Journal of Political Science 53 (1): 73–89.
Barabas, Jason, Jennifer Jerit, William Pollock, and Carlisle Rainey. 2014. “The Question(s) of Political Knowledge.” American Political Science Review 108 (4): 840–55.
Boykoff, Maxwell T, and Jules M Boykoff. 2004. “Balance as Bias: Global Warming and the US Prestige Press.” Global Environmental Change 14 (2): 125–36.
Delli Carpini, Michael. 1990. “Vietnam and the Press.” In Legacy: The Vietnam War in the American Imagination, edited by D.M. Shafer, 125–56. Boston: Beacon Press.
Druckman, James N. 2005. “Media Matter: How Newspapers and Television News Cover Campaigns and Influence Voters.” Political Communication 22 (4): 463–81.
Eveland, William P., Mihye Seo, and Krisztina Marton. 2002. “Learning from the News in Campaign 2000: An Experimental Comparison of TV News, Newspapers, and Online News.” Media Psychology 4 (4): 353–78.
Garrett, R. Kelly. 2009. “Politically Motivated Reinforcement Seeking: Reframing the Selective Exposure Debate.” Journal of Communication 59 (4): 676–99.
Jerit, Jennifer, Jason Barabas, and Toby Bolsen. 2006. “Citizens, Knowledge, and the Information Environment.” American Journal of Political Science 50 (2): 266–82.
Kucharski, Adam. 2016. “Post-Truth: Study Epidemiology of Fake News.” Nature 540 (December): 525.
Neuner, Fabian G., Stuart N. Soroka, and Christopher Wlezien. 2019. “Mass Media as a Source of Public Responsiveness.” International Journal of Press/Politics 24 (3): 269–92.
Prior, Markus. 2007. Post-Broadcast Democracy: How Media Choice Increases Inequality in Political Involvement and Polarizes Elections. New York: Cambridge University Press.
Salam, Maya. 2018. “Trump’s Mixed Claims on His First 500 Days.” New York Times, June 7, 2018, sec. U.S.
Shane, Scott, and Mark Mazzetti. 2018. “The Plot to Subvert an Election: Unraveling the Russia Story So Far.” New York Times, September 20, 2018, sec. U.S.
Soroka, Stuart N. 2014. Negativity in Democratic Politics: Causes and Consequences. New York: Cambridge University Press.
Soroka, Stuart N., and Christopher Wlezien. 2010. Degrees of Democracy: Politics, Public Opinion and Policy. Cambridge: Cambridge University Press.
———. 2022. Information and Democracy: Public Policy in the News. Communication, Society and Politics. Cambridge: Cambridge University Press.
Speers, Tammy, and Justin Lewis. 2005. “Journalists and Jabs: Media Coverage of the MMR Vaccine.” Communication & Medicine 1 (2): 171–81.
Stroud, Natalie Jomini. 2011. Niche News: The Politics of News Choice. Oxford: Oxford University Press.
Ura, Joseph Daniel, and Christopher R Ellis. 2012. “Partisan Moods: Polarization and the Dynamics of Mass Party Preferences.” Journal of Politics 74 (01): 277–91.
Van Aelst, Peter, Jesper Strömbäck, Toril Aalberg, Frank Esser, Claes de Vreese, Jörg Matthes, David Hopmann, et al. 2017. “Political Communication in a High-Choice Media Environment: A Challenge for Democracy?” Annals of the International Communication Association 41 (1): 3–27.
Wlezien, Christopher. 2004. “Patterns of Representation: Dynamics of Public Preferences and Policy.” Journal of Politics 66: 1–24.
[/workscited]