Why Trust Science?

Empowering Humanity with Reliable Knowledge

TOSLS: Test of Scientific Literacy Skills

Click here for access to answers, including feedback for both correct and incorrect student choices.

1. Which of the following is a valid scientific argument?

  1. Measurements of sea level on the Gulf Coast taken this year are lower than normal; the average monthly measurements were 1mm lower than normal in some areas. These facts prove that sea level rise is not a problem.
  2. A strain of mice was genetically engineered to lack a certain gene, and the mice were unable to reproduce. Introduction of the gene back into the mutant mice restored their ability to reproduce. These facts indicate that the gene is essential for mouse reproduction.
  3. A poll revealed that 34% of Americans believe that dinosaurs and early humans co-existed because fossil footprints of each species were found in the same location. This widespread belief is appropriate evidence to support the claim that humans did not evolve from ape ancestors.
  4. This winter, the northeastern US received record amounts of snowfall, and the average monthly temperatures were more than 2°F lower than normal in some areas. These facts indicate that climate change is occurring.

4. Which of the following research studies is least likely to contain a confounding factor – a variable that provides an alternative explanation for the results?

  1. Researchers randomly assign participants to experimental and control groups. Females make up 35% of the experimental group and 75% of the control group.
  2. To explore trends in the spiritual/religious beliefs of students attending U.S. universities, researchers survey a random selection of 500 freshmen at a small private university in the South.
  3. To evaluate the effect of a new diet program, researchers compare weight loss between participants randomly assigned to treatment (diet) and control (no diet) groups, while controlling for average daily exercise and pre-diet weight.
  4. Researchers tested the effectiveness of a new tree fertilizer on 10,000 saplings. Saplings in the control group (no fertilizer) were tested in the fall, whereas the treatment group (fertilizer) were tested the following spring.

5. Which of the following actions is a valid scientific course of action?

  1. A government agency relies heavily on two industry-funded studies in declaring a chemical found in plastics safe for humans, while ignoring studies linking the chemical with adverse health effects.
  2. Journalists give equal credibility to both sides of a scientific story, even though one side has been disproven by many experiments.
  3. A government agency decides to alter public health messages about breast-feeding in response to pressure from a council of businesses involved in manufacturing infant formula.
  4. Several research studies have found a new drug to be effective for treating the symptoms of autism; however, a government agency refuses to approve the drug until long-term effects are known.

9. Which of the following is not an example of an appropriate use of science?

  1. A group of scientists is asked to review research grant proposals and base its funding recommendations on the researcher’s experience, project plans, and preliminary data.
  2. Scientists are selected to help conduct a government-sponsored research study on global climate change based on their political beliefs.
  3. The Fish & Wildlife Service revises its list of protected and endangered species in response to new research findings.
  4. The Senate stops funding a widely used sex-education program after studies show limited effectiveness of the program.

Background for Questions 11–14 Use the excerpt below from a TV news report for the next few questions:

“A recent study, following more than 2,500 New Yorkers for 9 years, found that people who drank diet soda every day had a 61% higher risk of vascular events, including stroke and heart attack, compared to those who avoided diet drinks. For this study, Hannah Gardner’s research team randomly surveyed 2,564 New Yorkers about their eating behaviors, exercise habits, as well as cigarette and alcohol consumption. Participants were also given physical check-ups, including blood pressure measurements and blood tests for cholesterol and other factors that might affect the risk for heart attack and stroke. The increased likelihood of vascular events remained even after Gardener and her colleagues accounted for risk factors, such as smoking, high blood pressure and high cholesterol levels. The researchers found no increased risk among people who drank regular soda.”

11. The findings of this study suggest that consuming diet soda might lead to increased risk for heart attacks and strokes. From the statements below, identify additional evidence that supports this claim:

  1. Findings from an epidemiological study suggest that NYC residents are 6.8 times more likely to die of vascular-related diseases compared to people living in other U.S. cities.
  2. Results from an experimental study demonstrated that individuals randomly assigned to consume one diet soda each day were twice as likely to have a stroke compared to those assigned to drink one regular soda each day.
  3. Animal studies suggest a link between vascular disease and consumption of caramel-containing products (an ingredient that gives sodas their dark color).
  4. Survey results indicate that people who drink one or more diet sodas each day smoke more frequently than people who drink no diet soda, leading to increases in vascular events.

12. The excerpt above comes from what type of source of information?

  1. Primary (Research studies performed, written and then submitted for peer-review to a scientific journal.)
  2. Secondary (Reviews of several research studies written up as a summary article with references that are submitted to a scientific journal.)
  3. Tertiary (Media reports, encyclopedia entries, or documents published by government agencies.)
  4. None of the above

13. The lead researcher was quoted as saying, “I think diet soda drinkers need to stay tuned, but I don’t think that anyone should change their behaviors quite yet.” Why didn’t she warn people to stop drinking diet soda right away?

  1. The results should be replicated with a sample more representative of the U.S. population.
  2. There may be significant confounds present (alternative explanations for the relationship between diet sodas and vascular disease).
  3. Subjects were not randomly assigned to treatment and control groups.
  4. All of the above

14. Which of the following attributes is not a strength of the study’s research design?

  1. Collecting data from a large sample size.
  2. Randomly sampling NYC residents.
  3. Randomly assigning participants to control and experimental groups.
  4. All of the above

17. The most important factor influencing you to categorize a research article as trustworthy science should be:

  1. The presence of data or graphs
  2. The article was evaluated by unbiased third-party experts
  3. How clearly the article is written
  4. The publisher of the article

27. Which of the following represents a scientifically valid course of action?

  1. A scientific journal rejects a study because the results provide evidence against a widely accepted model.
  2. A leading scientific journal, Science, retracts a published article after discovering that the researcher misrepresented the data.
  3. A researcher distributes free samples of a new drug that she is developing to patients in need.
  4. A senior scientist encourages his graduate student to publish a study containing ground-breaking findings, even though some of the student’s crucial experiments could not be repeated.

Selected questions, some lightly edited, are from Gormally, Brickman, and Lutz.
CBE—Life Sciences Education 11, 364–377 (2012). They are listed here by their published numbers.