An electronic newsletter for
the science education community
May 2004
High-Quality Assessment Items
on the Horizon
Online
collection of science and mathematics test items aligned
to content standards
Through its Benchmarks for
Science Literacy and other resources, AAAS Project 2061
has influenced the way states across the country develop and use
K–12 science content standards. Now Project 2061 is working
to ensure that assessment is meaningfully tied to those standards.
With funding
from the National Science Foundation’s Instructional
Materials Development (IMD) Assessment Program, Project
2061 has begun a five-year, $4.1 million project to
develop a collection of high-quality middle and early
high school science and mathematics assessment items—including
multiple choice and open-response questions. The resulting
bank of more than 300 items will be electronically linked
to state and national science content standards and
accessible online.
Assessment with Precision
The new effort builds on Project
2061’s ongoing studies of assessment,
which have found that too many science and
math items are poorly written and fail to
measure the knowledge for which students
are being held responsible. While many existing
items cover topics—such as cells or
fractions—that are identified in the
content standards, few items are aligned
to the precise ideas and skills targeted
by content standards.
In contrast, the items to be included in Project 2061’s
new collection will be specially designed to provide explicit evidence
that a student has—or has not—learned a specific idea
or skill. As a result, teachers will be able to determine what their
students know and can do and to pinpoint areas where they need more
help. This precision and the diagnostic assessment it makes possible
have new urgency given the standards-based testing requirements
of the No Child Left Behind Act.
Project 2061’s analysis of assessment items for the collection
will involve the following considerations:
- Are the ideas and skills specified
in the targeted content standard needed
to successfully complete the assessment
item or can the item be answered without
that knowledge and skill?
- Are the ideas and skills specified
in the content standard enough by themselves
to successfully complete the assessment
item or is other knowledge and skill needed?
- Are students likely to understand the
task statement, diagrams, symbols, and
other parts of the task?
- Are students likely to understand what
they are expected to do and what sort of
response is considered satisfactory?
- Is the task context appropriately familiar,
engaging, and realistic to students?
- Could students respond satisfactorily
to the task by guessing or employing other
general test-taking strategies?
- Are scoring rubrics for open-ended
items accurate, clear, complete, and specific?
In addition, to fully address equity concerns and to increase
access to assessment items, especially among English language learners,
items will be reviewed for linguistic features. Looking at such
features increases the validity of inferences that can be drawn
about student understanding for a wider range of students.
Contributions to Research
While Project 2061’s work
will provide high-quality assessment items
for potential use in the classroom and in
statewide tests, it will also benefit curriculum
materials research and development. Assessment
items that are aligned with content standards
but not specific to any single materials
development project will enable researchers
to compare the effectiveness of various
instructional materials objectively. Curriculum
researchers need assessment items that policy
makers and the public regard as fair measures
of student knowledge. Without credible evidence
that new and innovative materials can help
students learn, stakeholders may decide
that the benefits of implementing such materials
do not justify the costs.
Researchers
also need items linked to content standards to test
such things as the comparative effectiveness of instructional
sequences, the viability of particular visual representations
of abstract concepts, and the value of using certain
phenomena and real-world examples to make ideas concrete
and understandable to students. Broad stroke evaluation
of the effectiveness of curriculum materials is not
enough. Through its work in the Center for Curriculum Materials
in Science, Project 2061 has found that items aligned
to content standards are essential for conducting fine-grained
research on materials as they are being developed. Assessment
tasks that provide precise measures of student understanding
of the specific ideas and skills addressed in a curriculum
material make it possible to conduct rigorous research
studies with replicable results.
To make its assessment resources widely available to researchers,
teachers, curriculum and test developers, and the general public,
Project 2061 will provide online access to items through an interface
adapted from the conceptual strand maps in its popular Atlas of Science Literacy. Interactive
maps will allow users to search by state standard, national standard,
topic, or type of assessment item.
# # #
For more information, please contact:
Principal Investigator: Dr.
George DeBoer, (202) 326-6624
[Table
of Contents]
