2061 Connections
An electronic newsletter for the science education community

May 2004

High-Quality Assessment Items on the Horizon

Online collection of science and mathematics test items aligned to content standards

Through its Benchmarks for Science Literacy and other resources, AAAS Project 2061 has influenced the way states across the country develop and use K–12 science content standards. Now Project 2061 is working to ensure that assessment is meaningfully tied to those standards.

With funding from the National Science Foundation’s Instructional Materials Development (IMD) Assessment Program, Project 2061 has begun a five-year, $4.1 million project to develop a collection of high-quality middle and early high school science and mathematics assessment items—including multiple choice and open-response questions. The resulting bank of more than 300 items will be electronically linked to state and national science content standards and accessible online.

Assessment with Precision
The new effort builds on Project 2061’s ongoing studies of assessment, which have found that too many science and math items are poorly written and fail to measure the knowledge for which students are being held responsible. While many existing items cover topics—such as cells or fractions—that are identified in the content standards, few items are aligned to the precise ideas and skills targeted by content standards.

In contrast, the items to be included in Project 2061’s new collection will be specially designed to provide explicit evidence that a student has—or has not—learned a specific idea or skill. As a result, teachers will be able to determine what their students know and can do and to pinpoint areas where they need more help. This precision and the diagnostic assessment it makes possible have new urgency given the standards-based testing requirements of the No Child Left Behind Act.

Project 2061’s analysis of assessment items for the collection will involve the following considerations:

  • Are the ideas and skills specified in the targeted content standard needed to successfully complete the assessment item or can the item be answered without that knowledge and skill?
  • Are the ideas and skills specified in the content standard enough by themselves to successfully complete the assessment item or is other knowledge and skill needed?
  • Are students likely to understand the task statement, diagrams, symbols, and other parts of the task?
  • Are students likely to understand what they are expected to do and what sort of response is considered satisfactory?
  • Is the task context appropriately familiar, engaging, and realistic to students?
  • Could students respond satisfactorily to the task by guessing or employing other general test-taking strategies?
  • Are scoring rubrics for open-ended items accurate, clear, complete, and specific?

In addition, to fully address equity concerns and to increase access to assessment items, especially among English language learners, items will be reviewed for linguistic features. Looking at such features increases the validity of inferences that can be drawn about student understanding for a wider range of students.

Contributions to Research
While Project 2061’s work will provide high-quality assessment items for potential use in the classroom and in statewide tests, it will also benefit curriculum materials research and development. Assessment items that are aligned with content standards but not specific to any single materials development project will enable researchers to compare the effectiveness of various instructional materials objectively. Curriculum researchers need assessment items that policy makers and the public regard as fair measures of student knowledge. Without credible evidence that new and innovative materials can help students learn, stakeholders may decide that the benefits of implementing such materials do not justify the costs.

Researchers also need items linked to content standards to test such things as the comparative effectiveness of instructional sequences, the viability of particular visual representations of abstract concepts, and the value of using certain phenomena and real-world examples to make ideas concrete and understandable to students. Broad stroke evaluation of the effectiveness of curriculum materials is not enough. Through its work in the Center for Curriculum Materials in Science, Project 2061 has found that items aligned to content standards are essential for conducting fine-grained research on materials as they are being developed. Assessment tasks that provide precise measures of student understanding of the specific ideas and skills addressed in a curriculum material make it possible to conduct rigorous research studies with replicable results.

To make its assessment resources widely available to researchers, teachers, curriculum and test developers, and the general public, Project 2061 will provide online access to items through an interface adapted from the conceptual strand maps in its popular Atlas of Science Literacy. Interactive maps will allow users to search by state standard, national standard, topic, or type of assessment item.

# # #

For more information, please contact:

Principal Investigator: Dr. George DeBoer, (202) 326-6624

[Table of Contents]