Copyright © 2002 Research for Better Schools. Reproduced with permission of Research for Better Schools. RBS Currents is in the public domain and may be reproduced without permission.
Accountability and Assessments
by Leah Bricker
Early this year the federal No Child Left Behind (NCLB) Act was signed into law. Part of the NCLB agenda focuses on improving the performance of "low achieving children in our Nation's highest poverty schools" and assuring that all students make progress towards meeting rigorous standards established by the states. The Act proposes powerful consequences for students, teachers, and other education stakeholders if assessment results do not show achievement of these standards.
There is great concern among many educators, however, that the assessment instruments used to measure the achievement of these standards fail to accurately measure the important ideas specified in the standards. Without accurate measurements, we cannot have confidence in the results of these assessments, and consequences may be unfairly and undeservedly imposed.
While NCLB calls for alignment of assessments with standards, "alignment" is a word with many definitions. How do we know the assessments being used to measure achievement are truly aligned with the standards? Researchers at the American Association for the Advancement of Science Project 2061, with support from the National Science Foundation, are studying this question to produce a clearer definition of alignment. The researchers are also developing an assessment analysis procedure that can be used to determine the relationship between K-12 assessments in science and mathematics and national, state, and district standards and benchmarks.
Probing Alignment More Closely
Project 2061's approach to alignment differs from traditional approaches, and is important because of its innovation. The Project 2061 approach does not focus on topic distribution or whether the assessment tasks fit under goal topics (e.g., "cells" or "fractions"). Instead, Project 2061 seeks to probe more deeply in order to judge how well each individual task gets at the exact knowledge specified in the goal or goals that it was written to assess. Project 2061 considers an assessment task to be well aligned to a learning goal only if it both aims at the exact knowledge (fact, idea, or skill) specified in the goal and will likely be an effective probe of that specific knowledge.
Project 2061's analysis procedure first helps analysts to identify the specific learning goals that an assessment task targets, and then helps analysts to judge the likely effectiveness of the task in probing student achievement of those goals. Some of the questions asked in the procedure are:
- Is the exact knowledge specified in the standard(s) needed to make a satisfactory response?
- Is the exact knowledge specified in the standard(s) enough by itself to make a satisfactory response or is additional knowledge needed?
- Are students likely to comprehend the task?
- Are students likely to understand what they are expected to do and what sort of response is considered satisfactory?
- Is the task's context appropriate?
- Could students respond satisfactorily to the task by guessing or employing other general test-taking strategies?
- Is the task's scoring guide adequate and accurate?
This original research will benefit groups such as commercial developers and publishers of instructional and assessment material; districts and states that create, select, and administer large-scale testing programs; and classroom teachers who create or assemble their own quizzes or tests. The research will yield longer-term benefits as it helps to change educators' views on what to expect of assessment tasks and to make better choices among what publishers offer them.
New Approaches Needed
This research will result in:
- An analysis procedure to judge alignment of assessment tasks to specific learning goals;
- Case studies illustrating the use of the procedure to evaluate and revise existing assessment tasks and create new ones;
- A collection of analysis profiles for the assessment tasks that are reviewed; and
- Lessons learned--information to further the national discussion about alignment, including recommendations.
The testing mandated by the No Child Left Behind legislation will have important consequences for students and teachers. What is more, testing will continue to drive the curriculum-what gets tested is typically what gets taught. New and innovative approaches are essential to ensure that testing student knowledge of learning goals is as accurate and fair as possible.
Bricker is a senior program associate, Project 2061, American Association for the Advancement of Science.