Skip to content

Lesson 12: Bias in Survey Sampling

Lesson 12: Bias in Survey Sampling

Objective:

Students will learn about bias in relation to survey sampling. More specifically, they will learn what types of sampling methods could result in a biased sample, who might be over/under-represented in the sample, and how to select a better sample.

Materials:

  1. Identifying Biased Samples handout (LMR_U3_L12_A)

  2. Poster paper

  3. Survey Sampling handout (LMR_U3_L12_B)

Vocabulary:

survey sample over-represented under-represented random sampling

Essential Concepts:

Essential Concepts:

Bias concerning survey sampling includes identifying sampling methods that may lead to biased samples, recognizing potential over- or under-representation in samples, and acquiring skills to choose more reliable sampling techniques.

Lesson:

  1. Remind students that they learned about biased samples during the last few lessons. Today, they will continue with this topic and discuss how people are selected to be in a sample.

  2. The people who are asked to participate in a survey are known as the survey sample. Ideally, the people who are included in the survey sample are a representative group of the target population, or the population we would like to make inferences about.

  3. Propose the following scenario to the class: “An elementary school is going to start serving ice cream in the cafeteria every Friday during lunch, and needs to know the favorite flavor of its students."

  4. In pairs, ask students to come up with two examples of samples that might be biased. For instance, one biased sample might include only the four 3rd grade classes at the school. For each biased sample, the students should answer the following questions in their DS journals:

    1. Who is the target population? Answer: All students at the elementary school.

    2. Who is included in your biased sample? Answer: Only 3rd grade students. These students are over-represented in the sample.

    3. Who is not included in your biased sample? Answer: All other students in the school (Kindergartners, 1st, 2nd, 4th, and 5th graders). These students are under-represented in the sample.

    4. Is your sample representative of the target population? Answer: No! We’re only including 3rd graders, and they may not have the same preferences as other students.

  5. Once pairs have come up with their biased samples and answered the questions in step 4 above, they should share out with their student teams and answer the following questions:

    1. How is your biased sample different from the samples created by other pairs in your team? Answers will vary by class. An example might be that one pair sampled only 3rd graders and the other pair sampled only girls.

    2. Which do you think is more representative of the target population? Why? Answers will vary by class. Using the example above, we could argue that either one is more representative. We could maybe say that, since 3rd graders include both boys and girls, we have a more representative sample than if we just sampled girls. Or, we could say that since girls come from all grade levels, they’re more representative of the entire school than just 3rd graders.

  6. After the teams have discussed their samples, they should select one pair’s biased sample to share with the rest of the class. Record each team’s biased sample on a sheet of poster paper with the following layout:

    Biased Sample Who is overrepresented? Who is underrepresented?
    All 3rd grade students 3rd grade students All other students (kindergartners, 1st, 2nd, 4th, and 5th graders)

  7. Distribute the Identifying Biased Samples handout (LMR_U3_L12_A). In this activity, students will explain why a particular sampling method might result in a biased sample – a sample that is not representative of the population of interest.

    Note: It is NOT enough for students to say that the “sample is not random.” They need to explain how the sample is biased.

    Note: Page 2 of the handout provides sample answers for teacher reference. Do NOT distribute page 2 to students.

  8. Each student should complete the handout independently. Afterwards, conduct a whole-class discussion to compare and contrast different students’ explanations of how the samples might be biased. For each example given in the handout, discuss who is most likely over-represented and who is most likely under-represented in the sample.

  9. Ask the students:

    1. Now that we have learned about sampling biases, how can we eliminate this type of bias? Answers will vary by class.

    Note: Allow students to collaborate and come up with a few ideas on their own of how to eliminate sampling bias. If desired, ideas can be written on the board for discussion and comparison.

  10. Conclude with the actual answer: random sampling.

    1. If we randomly sample people from our population of interest, we can reduce the bias of any sample statistics obtained from the survey responses.

    2. If we have a biased sample, we can only give descriptions about that particular sample; we CANNOT generalize to the population of interest.

Class Scribes:

One team of students will give a brief talk to discuss what they think the 3 most important topics of the day were.

Homework

Students will complete the Survey Sampling handout (LMR_U3_L12_B) for homework.