An electronic newsletter for
the science education community
High-Quality Assessment Items on the Horizon
Online collection of science and mathematics test items aligned to content standards
Through its Benchmarks for Science Literacy and other resources, AAAS Project 2061 has influenced the way states across the country develop and use K–12 science content standards. Now Project 2061 is working to ensure that assessment is meaningfully tied to those standards.
With funding from the National Science Foundation’s Instructional Materials Development (IMD) Assessment Program, Project 2061 has begun a five-year, $4.1 million project to develop a collection of high-quality middle and early high school science and mathematics assessment items—including multiple choice and open-response questions. The resulting bank of more than 300 items will be electronically linked to state and national science content standards and accessible online.
Assessment with Precision
The new effort builds on Project
2061’s ongoing studies of assessment,
which have found that too many science and
math items are poorly written and fail to
measure the knowledge for which students
are being held responsible. While many existing
items cover topics—such as cells or
fractions—that are identified in the
content standards, few items are aligned
to the precise ideas and skills targeted
by content standards.
In contrast, the items to be included in Project 2061’s new collection will be specially designed to provide explicit evidence that a student has—or has not—learned a specific idea or skill. As a result, teachers will be able to determine what their students know and can do and to pinpoint areas where they need more help. This precision and the diagnostic assessment it makes possible have new urgency given the standards-based testing requirements of the No Child Left Behind Act.
Project 2061’s analysis of assessment items for the collection will involve the following considerations:
- Are the ideas and skills specified in the targeted content standard needed to successfully complete the assessment item or can the item be answered without that knowledge and skill?
- Are the ideas and skills specified in the content standard enough by themselves to successfully complete the assessment item or is other knowledge and skill needed?
- Are students likely to understand the task statement, diagrams, symbols, and other parts of the task?
- Are students likely to understand what they are expected to do and what sort of response is considered satisfactory?
- Is the task context appropriately familiar, engaging, and realistic to students?
- Could students respond satisfactorily to the task by guessing or employing other general test-taking strategies?
- Are scoring rubrics for open-ended items accurate, clear, complete, and specific?
In addition, to fully address equity concerns and to increase access to assessment items, especially among English language learners, items will be reviewed for linguistic features. Looking at such features increases the validity of inferences that can be drawn about student understanding for a wider range of students.
Contributions to Research
While Project 2061’s work
will provide high-quality assessment items
for potential use in the classroom and in
statewide tests, it will also benefit curriculum
materials research and development. Assessment
items that are aligned with content standards
but not specific to any single materials
development project will enable researchers
to compare the effectiveness of various
instructional materials objectively. Curriculum
researchers need assessment items that policy
makers and the public regard as fair measures
of student knowledge. Without credible evidence
that new and innovative materials can help
students learn, stakeholders may decide
that the benefits of implementing such materials
do not justify the costs.
Researchers also need items linked to content standards to test such things as the comparative effectiveness of instructional sequences, the viability of particular visual representations of abstract concepts, and the value of using certain phenomena and real-world examples to make ideas concrete and understandable to students. Broad stroke evaluation of the effectiveness of curriculum materials is not enough. Through its work in the Center for Curriculum Materials in Science, Project 2061 has found that items aligned to content standards are essential for conducting fine-grained research on materials as they are being developed. Assessment tasks that provide precise measures of student understanding of the specific ideas and skills addressed in a curriculum material make it possible to conduct rigorous research studies with replicable results.
To make its assessment resources widely available to researchers, teachers, curriculum and test developers, and the general public, Project 2061 will provide online access to items through an interface adapted from the conceptual strand maps in its popular Atlas of Science Literacy. Interactive maps will allow users to search by state standard, national standard, topic, or type of assessment item.
# # #
For more information, please contact:
Principal Investigator: Dr. George DeBoer, (202) 326-6624