Skip to ContentGo to accessibility pageKeyboard shortcuts menu
OpenStax Logo
Introduction to Political Science

5.5 How Do We Measure Public Opinion?

Introduction to Political Science5.5 How Do We Measure Public Opinion?
Menu
Table of contents
  1. Preface
  2. Introduction to Political Science
    1. 1 What Is Politics and What Is Political Science?
      1. Introduction
      2. 1.1 Defining Politics: Who Gets What, When, Where, How, and Why?
      3. 1.2 Public Policy, Public Interest, and Power
      4. 1.3 Political Science: The Systematic Study of Politics
      5. 1.4 Normative Political Science
      6. 1.5 Empirical Political Science
      7. 1.6 Individuals, Groups, Institutions, and International Relations
      8. Summary
      9. Key Terms
      10. Review Questions
      11. Suggested Readings
  3. Individuals
    1. 2 Political Behavior Is Human Behavior
      1. Introduction
      2. 2.1 What Goals Should We Seek in Politics?
      3. 2.2 Why Do Humans Make the Political Choices That They Do?
      4. 2.3 Human Behavior Is Partially Predictable
      5. 2.4 The Importance of Context for Political Decisions
      6. Summary
      7. Key Terms
      8. Review Questions
      9. Suggested Readings
    2. 3 Political Ideology
      1. Introduction
      2. 3.1 The Classical Origins of Western Political Ideologies
      3. 3.2 The Laws of Nature and the Social Contract
      4. 3.3 The Development of Varieties of Liberalism
      5. 3.4 Nationalism, Communism, Fascism, and Authoritarianism
      6. 3.5 Contemporary Democratic Liberalism
      7. 3.6 Contemporary Ideologies Further to the Political Left
      8. 3.7 Contemporary Ideologies Further to the Political Right
      9. 3.8 Political Ideologies That Reject Political Ideology: Scientific Socialism, Burkeanism, and Religious Extremism
      10. Summary
      11. Key Terms
      12. Review Questions
      13. Suggested Readings
    3. 4 Civil Liberties
      1. Introduction
      2. 4.1 The Freedom of the Individual
      3. 4.2 Constitutions and Individual Liberties
      4. 4.3 The Right to Privacy, Self-Determination, and the Freedom of Ideas
      5. 4.4 Freedom of Movement
      6. 4.5 The Rights of the Accused
      7. 4.6 The Right to a Healthy Environment
      8. Summary
      9. Key Terms
      10. Review Questions
      11. Suggested Readings
    4. 5 Political Participation and Public Opinion
      1. Introduction
      2. 5.1 What Is Political Participation?
      3. 5.2 What Limits Voter Participation in the United States?
      4. 5.3 How Do Individuals Participate Other Than Voting?
      5. 5.4 What Is Public Opinion and Where Does It Come From?
      6. 5.5 How Do We Measure Public Opinion?
      7. 5.6 Why Is Public Opinion Important?
      8. Summary
      9. Key Terms
      10. Review Questions
      11. Suggested Readings
  4. Groups
    1. 6 The Fundamentals of Group Political Activity
      1. Introduction
      2. 6.1 Political Socialization: The Ways People Become Political
      3. 6.2 Political Culture: How People Express Their Political Identity
      4. 6.3 Collective Dilemmas: Making Group Decisions
      5. 6.4 Collective Action Problems: The Problem of Incentives
      6. 6.5 Resolving Collective Action Problems
      7. Summary
      8. Key Terms
      9. Review Questions
      10. Suggested Readings
    2. 7 Civil Rights
      1. Introduction
      2. 7.1 Civil Rights and Constitutionalism
      3. 7.2 Political Culture and Majority-Minority Relations
      4. 7.3 Civil Rights Abuses
      5. 7.4 Civil Rights Movements
      6. 7.5 How Do Governments Bring About Civil Rights Change?
      7. Summary
      8. Key Terms
      9. Review Questions
      10. Suggested Readings
    3. 8 Interest Groups, Political Parties, and Elections
      1. Introduction
      2. 8.1 What Is an Interest Group?
      3. 8.2 What Are the Pros and Cons of Interest Groups?
      4. 8.3 Political Parties
      5. 8.4 What Are the Limits of Parties?
      6. 8.5 What Are Elections and Who Participates?
      7. 8.6 How Do People Participate in Elections?
      8. Summary
      9. Key Terms
      10. Review Questions
      11. Suggested Readings
  5. Institutions
    1. 9 Legislatures
      1. Introduction
      2. 9.1 What Do Legislatures Do?
      3. 9.2 What Is the Difference between Parliamentary and Presidential Systems?
      4. 9.3 What Is the Difference between Unicameral and Bicameral Systems?
      5. 9.4 The Decline of Legislative Influence
      6. Summary
      7. Key Terms
      8. Review Questions
      9. Suggested Readings
    2. 10 Executives, Cabinets, and Bureaucracies
      1. Introduction
      2. 10.1 Democracies: Parliamentary, Presidential, and Semi-Presidential Regimes
      3. 10.2 The Executive in Presidential Regimes
      4. 10.3 The Executive in Parliamentary Regimes
      5. 10.4 Advantages, Disadvantages, and Challenges of Presidential and Parliamentary Regimes
      6. 10.5 Semi-Presidential Regimes
      7. 10.6 How Do Cabinets Function in Presidential and Parliamentary Regimes?
      8. 10.7 What Are the Purpose and Function of Bureaucracies?
      9. Summary
      10. Key Terms
      11. Review Questions
      12. Suggested Readings
    3. 11 Courts and Law
      1. Introduction
      2. 11.1 What Is the Judiciary?
      3. 11.2 How Does the Judiciary Take Action?
      4. 11.3 Types of Legal Systems around the World
      5. 11.4 Criminal versus Civil Laws
      6. 11.5 Due Process and Judicial Fairness
      7. 11.6 Judicial Review versus Executive Sovereignty
      8. Summary
      9. Key Terms
      10. Review Questions
      11. Suggested Readings
    4. 12 The Media
      1. Introduction
      2. 12.1 The Media as a Political Institution: Why Does It Matter?
      3. 12.2 Types of Media and the Changing Media Landscape
      4. 12.3 How Do Media and Elections Interact?
      5. 12.4 The Internet and Social Media
      6. 12.5 Declining Global Trust in the Media
      7. Summary
      8. Key Terms
      9. Review Questions
      10. Suggested Readings
  6. States and International Relations
    1. 13 Governing Regimes
      1. Introduction
      2. 13.1 Contemporary Government Regimes: Power, Legitimacy, and Authority
      3. 13.2 Categorizing Contemporary Regimes
      4. 13.3 Recent Trends: Illiberal Representative Regimes
      5. Summary
      6. Key Terms
      7. Review Questions
      8. Suggested Readings
    2. 14 International Relations
      1. Introduction
      2. 14.1 What Is Power, and How Do We Measure It?
      3. 14.2 Understanding the Different Types of Actors in the International System
      4. 14.3 Sovereignty and Anarchy
      5. 14.4 Using Levels of Analysis to Understand Conflict
      6. 14.5 The Realist Worldview
      7. 14.6 The Liberal and Social Worldview
      8. 14.7 Critical Worldviews
      9. Summary
      10. Key Terms
      11. Review Questions
      12. Suggested Readings
    3. 15 International Law and International Organizations
      1. Introduction
      2. 15.1 The Problem of Global Governance
      3. 15.2 International Law
      4. 15.3 The United Nations and Global Intergovernmental Organizations (IGOs)
      5. 15.4 How Do Regional IGOs Contribute to Global Governance?
      6. 15.5 Non-state Actors: Nongovernmental Organizations (NGOs)
      7. 15.6 Non-state Actors beyond NGOs
      8. Summary
      9. Key Terms
      10. Review Questions
      11. Suggested Readings
    4. 16 International Political Economy
      1. Introduction
      2. 16.1 The Origins of International Political Economy
      3. 16.2 The Advent of the Liberal Economy
      4. 16.3 The Bretton Woods Institutions
      5. 16.4 The Post–Cold War Period and Modernization Theory
      6. 16.5 From the 1990s to the 2020s: Current Issues in IPE
      7. 16.6 Considering Poverty, Inequality, and the Environmental Crisis
      8. Summary
      9. Key Terms
      10. Review Questions
      11. Suggested Readings
  7. References
  8. Index

Learning Outcomes

By the end of this section, you will be able to:

  • Describe different methods of measuring public opinion.
  • Explain the shortcomings of these methods.

Earlier in this chapter, we discussed how writing to public officials is an important facet of political participation. Before we had a good way to measure public opinion, constituent letters were one of the few ways officials could gauge how the public felt. The advent of public opinion polls provided a scientific way of identifying and measuring opinions. Social scientist Jean Converse, in her history of the field, writes that surveys can be traced back 2,000 years but were forged in the 20th century as a way to understand mass populations and societies and to gain insight into elites.127 Over time, polls and surveys have become more precise through careful sampling and improved techniques. A sample is a group selected by a researcher to represent the characteristics of the entire population, and because we can never poll the entire population, getting the right sample is important to the accuracy of any poll. But how can we accurately gauge the opinions of the whole country on a sample of 1,400 or 2,000 people? The way the sample is drawn affects its accuracy. In the most common method, probability sampling, researchers randomly choose samples from the larger population. This method requires that everyone has an equal chance of being part of the sample and that they are randomly selected, which allows researchers to make generalizations about the larger population. If a researcher chooses people at random from a population, it is likely that their views will match the opinions of the larger population as a whole. These types of samples are often generated through random digit dialing, in which respondents are chosen at random by a computerized phone number generator. Researchers then use these randomly generated phone numbers to reach people at home and ask them about their opinions. While random digit dialing has been the go-to for decades, the decrease in landlines, increased adoption of cell phones, and increased time that people are at work have all contributed to the decreased reliance on home-based phone numbers. A Los Angeles Times article found that Internet-based surveys and automated interviewing systems (as opposed to live pollsters) were particularly accurate and may reflect a shift in how researchers measure public opinion moving forward.128

The difficulty of reaching people for polls is not just an American phenomenon. Researchers in Japan have found steep decreases in responses to nationally conducted surveys, with the steepest declines in metropolitan areas and among younger demographics. Scholars point to increased commute times, longer work hours, and higher mobility among younger Japanese as contributing to this problem.129 Sampling in countries facing violence or instability can be a serious—and dangerous—problem for pollsters. James Bell, director of international survey research for the Pew Research Center, notes that when Pew conducted polls during civil unrest in Ukraine and Venezuela, polls needed to be conducted face-to-face rather than by phone. In addition, sometimes the data acquired in polls must be processed locally if pollsters cannot immediately evacuate the area.130

What Can I Do?

The Importance of Empirical and Quantitative Skills

A line graph shows consistent levels of support for political parties in New Zealand between January 1, 2009 and January 1, 2012. The National Party received the most support, with around 50%, the Labour Party received the second most support, with around 30%, and the Green Party received the least support, with around 10%.
Figure 5.12 This graph shows support for political parties in New Zealand between 2009 and 2012, according to various political polls. (credit: “File:NZ opinion polls 2009-2011 -parties.png” Mark Payne, Denmark/Wikimedia Commons, CC BY 3.0)

Everyone loves a good public opinion poll. As you’ll see in other chapters, data from polls are utilized throughout society, from the media to candidates running for office, and even to decide what gets included in legislation. However, being able to properly understand what the data are telling us is a skill that is developed and can be utilized in a wide range of areas. If you can look at a set of numerical data or observable facts and reach an informed conclusion about what is happening—for example, about whether a group of voters prefers a certain candidate or if a group of residents wants a park built in their town—this is really no different from determining if a group of consumers prefers brand X bread or brand Y bread. In the modern digital era, we have a wealth of information at our fingertips. Being able to properly understand and interpret that information is a skill that is becoming fundamental in today’s workforce.

There are also “nontraditional” sampling methods, which may be less scientific but offer certain benefits. One nontraditional method is a convenience sample, which, as the name suggests, is a sample based on convenience rather than probability. If you do not have the funds to create a poll based on a probability sample and random digit dialing, you might instead ask your classmates or your coworkers to respond to your survey with their opinions on the last election. While this method is both convenient and easy, we cannot extrapolate much from the information beyond the sample from which it is drawn. Another type of polling method is called cluster sampling, in which researchers divide the overall population into clusters, based on characteristics such as shared cities or schools, then randomly select people from within those clusters to poll. This type of sampling is cheaper than probability sampling, but the results are also not quite as representative because they are not randomly drawn.

How reliable are polls? One of the most basic issues with a poll is a sample that is too small, which leads to sampling errors. Generally speaking, the larger the sample, the less chance of error. A typical sample of 1,500 people will have a sampling error of approximately 2.6 percent, which is generally considered an “acceptable” margin of error in public opinion polling. This means that out of 1,500 respondents, if 60 percent say that the LA Lakers is their favorite NBA team, due to sampling error, the true figure could be anywhere between 57.4 percent and 62.6 percent who prefer the Lakers. The smaller the sample, the larger the error.

The methods by which respondents are contacted can also affect a poll’s accuracy. According to the Centers for Disease Control and Prevention (CDC), in 2020, 83 percent of Americans aged 30–34, 74.5 percent of those aged 35–44, and almost 60 percent of those aged 45–64 used cell phones exclusively.131 This trend away from landlines can contribute to selection bias, whereby the sample drawn is not representative of the population being studied. In this case, any sample drawn from people using landlines would probably skew heavily toward individuals who are much older and those who are likely to be at home more often.

The design of the survey itself can limit a poll’s accuracy. Question wording, interviewer bias, and response bias can all lead to measurement error, or limitations in response validity due to survey design problems. Questions should be worded in a straightforward manner in order to solicit a truthful response. Studies have shown that alterations in question wording, also known as question wording effects, change how people respond to polls and surveys. For example, University of Chicago Professor Kenneth Rasinki found that even the slightest changes in wording altered people’s support for government spending,132 while Cornell University Professor Jonathon Schuldt, Indiana University Professor Sara Konrath, and University of Southern California Professor Norbert Schwarz found that responses changed depending on whether they used the term “global warming” or the term “climate change.”133 Bias that stems from the identity of the individual conducting the interview known as interviewer bias, can also change people’s opinions. For example, Princeton University Professor Daniel Katz found that the social class of the interviewer had an effect on survey response,134 while a study of breast cancer patients found that response rates to surveys were higher when the race of the person administering the survey was the same as that of the respondent.135 Similar effects have been found when interviewers are of different genders.136 In other words, people sometimes respond differently based on the gender of the person conducting the survey.

Inaccuracies can also arise from response bias, when respondents inaccurately report their true opinions for one reason or another. One famous example of response bias is called the “Bradley effect.” This theory refers to a phenomenon observed in the 1982 California gubernatorial race between Tom Bradley, a Black man, and George Deukmejian, a White man of Armenian descent. In polls leading up to this race, Bradley was shown to be in the lead, but he ultimately lost by a narrow margin. The theory behind the Bradley effect is that White voters are unlikely to admit to bias against minority candidates, and as such, polls may overestimate support for a minority candidate. Also known as social desirability bias, this type of response bias occurs when respondents give the answer they think they should give, and not what they really feel.

PreviousNext
Order a print copy

As an Amazon Associate we earn from qualifying purchases.

Citation/Attribution

Want to cite, share, or modify this book? This book uses the Creative Commons Attribution License and you must attribute OpenStax.

Attribution information
  • If you are redistributing all or part of this book in a print format, then you must include on every physical page the following attribution:
    Access for free at https://openstax.org/books/introduction-political-science/pages/1-introduction
  • If you are redistributing all or part of this book in a digital format, then you must include on every digital page view the following attribution:
    Access for free at https://openstax.org/books/introduction-political-science/pages/1-introduction
Citation information

© Oct 27, 2022 OpenStax. Textbook content produced by OpenStax is licensed under a Creative Commons Attribution License . The OpenStax name, OpenStax logo, OpenStax book covers, OpenStax CNX name, and OpenStax CNX logo are not subject to the Creative Commons license and may not be reproduced without the prior and express written consent of Rice University.