Event details

Quizzes are central to contemporary education around the world and of great interest to researchers across the social sciences (in psychology, sociology, political science, health, business, and other fields) who want to assess knowledge levels, to be used as predictors of behavior or opinions (do people who understand health risks of smoking better smoke less?), as moderators of effects of treatments (are people who know a lot about politics less influenced by political ads?), as foci of interventions (does educating people about the Gaza conflict lead those people to change their sympathies?), and in other analytic ways. For decades, the National Science Foundation’s National Center for Science and Engineering Statistics has conducted national surveys measuring Americans’ knowledge about science. In NSF’s annual reports to Congress, the quiz results have been presented as simply documenting levels of information in the public and trends in such understanding. This talk will put a spotlight on the questions asked in these quizzes, highlight some ways in which the design of those questions violate what the questionnaire design literature suggests are best practices, and report a series of studies done to test for artifacts in the resulting measurements. The results are of value to any researchers who wish to make accurate measurements with questionnaires and give reason to pause when interpreting the results of existing surveys intended to document levels of knowledge about science.