An electronic newsletter for the science education community
National Sample Puts Assessment Items to the Test
To be included in a national field test, a Project 2061 assessment item must travel a long and difficult road. Each middle school science assessment item is the result of a two-year development cycle. Project 2061’s assessment experts clarify the target learning goals, review the research literature on student learning, and design items that are content-aligned and that use student misconceptions as distractors. The researchers then pilot test each item, organize formal evaluations by teams of external reviewers, and finally field test each item using a national sample, revising items at each stage along the way. Each field-tested item also goes through a number of statistical analyses.
The process may be long, but the purpose is clear: to develop high-quality assessment items that accurately measure students’ achievement of science standards. With funding from the National Science Foundation, Project 2061 is developing a collection of multiple-choice items for 16 middle school science topics. The items are closely aligned to the learning goals in Benchmarks for Science Literacy (American Association for the Advancement of Science, 1993) and the National Science Education Standards (National Research Council, 1996).
At the National Association for Research in Science Teaching (NARST) Annual Conference in April, Project 2061 presented the latest findings from its ongoing item development work. The two papers below discuss items in middle school science topics at different stages in the development process. The first paper reports the results of national field tests of items in four topics: Matter and Energy in Living Systems; Chemical Reactions and Conservation of Matter; Plate Tectonics; and Control of Variables. The second paper reports the results of a pilot test of items aligned to the topic of Energy.
“Assessment Linked to Middle School Science Learning Goals: A Report on Field Test Results for Four Middle School Science Topics”
George E. DeBoer, Cari F. Herrmann-Abell, Jill Wertheim, and Jo Ellen Roseman, AAAS Project 2061
Read the full paper [PDF 254KB].
Abstract: This paper is a summary of certain aspects of an assessment development project being conducted at AAAS Project 2061. The project involves developing multiple-choice assessment items for 16 middle school science topics closely aligned to the ideas in Benchmarks for Science Literacy and the National Science Education Standards. The items can be used to gather data on what students currently know, the growth in their knowledge over time, or gains in their knowledge made following instruction. In this paper, we discuss the results of field testing of four topics: Matter and Energy in Living Systems; Chemical Reactions and Conservation of Matter; Plate Tectonics; and Control of Variables.
The item development procedure involves rigorous qualitative alignments and the use of quantitative psychometric methods. It is this balance of the two approaches that distinguishes Project 2061’s work from many other approaches to item development. The items discussed in this paper have gone through this rigorous development process, including pilot testing, expert review, and various statistical analyses.
“Using Content-Aligned Assessment to Probe Middle School Students' Understanding of Ideas About Energy”
Cari F. Herrmann-Abell and George E. DeBoer, AAAS Project 2061
Read the full paper [PDF 67KB].
Abstract: This paper reports the results of a pilot test of assessment items aligned to the middle school topic focused on forms of energy. The pilot test was administered to 1728 sixth, seventh, and eighth grade students from 11 widely varying school districts across the country in the spring of 2008. This paper presents data from assessment items aligned to key ideas for motion energy, thermal energy, gravitational energy, and elastic energy, and describes how we use information gathered from the students to gain insight into students’ thinking as well as insights into the quality of the items themselves.
# # #
For more information about Project 2061’s assessment research, please contact:
Principal Investigator: Dr. George DeBoer, (202) 326-6624
References
American Association for the Advancement of Science. (1993). Benchmarks for science literacy. New York: Oxford University Press.
National Research Council. (1996). National science education standards. Washington, DC: National Academy Press.
Also in the May/June 2009 issue: