Student Feedback Takes Center Stage
Posters highlight assessment of force and motion concepts
What do researchers learn when they compare students’ test answers to the explanations they give for their answers? Often, as Project 2061 is finding out, they learn that test items have a long way to go before they can be considered accurate measures of what the students know. In some cases, students choose the correct answer but don’t really know the targeted science idea. In others, students are too confused by ambiguous language or unclear diagrams to demonstrate their science knowledge.
As part of its development of an online collection of model assessment items and resources, AAAS Project 2061 is making the most of data collected from student interviews and pilot tests. The student feedback is proving essential as researchers seek to design test items that gauge as precisely as possible students' understanding of the key ideas in AAAS's Benchmarks for Science Literacy and the National Research Council's National Science Education Standards (read an overview of the assessment project).
In January, research associate Dr. Thomas Regan traveled to the American Association of Physics Teachers (AAPT) Winter Meeting in Seattle to share what the research team has learned so far about middle grades items targeting key force and motion concepts.
Learning from the Students
Regan presented two posters that highlight how the student data is informing Project 2061's design of multiple-choice assessment items. Regan spoke with teachers, professors of education, graduate students, state administrators, and education outreach officers from national laboratories, all of whom could use the assessment resources in their work.
"There's a real need for assessment items that are aligned to the specific ideas that standards expect students to know," says Regan. "The educators I met with appreciated the detailed information we are gathering about each item when we ask students in pilot tests to comment on every answer choice, not just the one they chose. By finding out if an incorrect answer choice is a plausible distractor, we learn a lot more about student thinking and the features of items that need improvement."
The posters present examples that show how student feedback from pilot testing and interviews is being used to improve test items in terms of vocabulary, context, incorrect answer choices (distractors), comprehensibility, grade-level suitability, and effective representations:
View “Modification of Multiple-Choice Assessment Items Based on Student Feedback” [PDF 444KB]
Thomas Regan, George DeBoer
View “Representations of Force and Motion Concepts at the Middle Level” [PDF 236KB]
Thomas Regan, Brian Sweeney, Ted Willard, George DeBoer
The test items featured on the posters are still in development and researchers continue to modify them in a number of areas (not all of which are indicated on the posters).
As the research team develops assessment items in a number of key topics for the online collection, they are supplementing the items with a wealth of related resources: (1) clarifications of each key science idea that pinpoint what student are expected to know, (2) common student misconceptions identified by research and useful for designing distractors, and (3) assessment maps that show how key ideas build toward student understanding.
For Regan, the usefulness of the forthcoming collection for science educators lies in how all of these resources work together. “Project 2061's goal in this project is to put together a model system of the components needed for effective standards-based assessment,” says Regan. “Teachers and researchers alike will be able to use the items and other resources to study and improve assessment in the context of classrooms or statewide tests, or as part of curriculum development projects.”
# # #
For more information about Project 2061's assessment research, please contact:
Principal Investigator: Dr. George DeBoer, (202) 326-6624