For students to be able to achieve mathematics benchmarks, educators need textbooks that not only address these important concepts and skills but also provide instructional strategies that are likely to help students learn them. With that goal in mind, Project 2061’s textbook evaluation has several distinct features that set it apart from other evaluation efforts.

Six independent, two-person teams made up of classroom teachers and college-level mathematics educators evaluated each textbook. Using custom-designed software to document what they found, analysts examined the student and teacher editions of the textbooks to identify specific lessons, activities, teacher notes, assessments, and other materials that addressed ideas in each of six selected benchmarks drawn from Project 2061’s Benchmarks for Science Literacy. Three questions were considered to determine whether the content addressed the chosen benchmarks: Did the content address the substance of the benchmark or was there only a topic match? Did the content match the sophistication (appropriate age level) of the benchmark? Did the content match part(s) or all of the benchmark? Analysts then rated that content using a set of research-based instructional criteria.

In creating its analysis procedure, Project 2061 found that studying a material’s treatment of a relatively small but carefully chosen set of benchmarks can reveal the strengths and weaknesses of the material as a whole, particularly its instructional design and guidance. Six benchmarks representing three important mathematical strands—number, geometry, and algebra—were used to conduct the analysis. These benchmarks provide a sufficiently comprehensive example of the core mathematics content likely to appear in any middle grades material. Specifically, they include a concept benchmark dealing with fractions and operations on them; a skill benchmark dealing with equivalent forms of numbers; a concept and a skill benchmark dealing with properties of shapes and computations of circumference, area, and volume; and two concept benchmarks dealing with graphing and equations.

To judge how well a textbook was likely to help students learn these key concepts and skills, the analysts applied a set of carefully developed instructional criteria to the text’s treatment of each selected benchmark. The criteria were derived from research on learning and teaching and on the craft knowledge of experienced educators. For example, analysts rated the extent to which textbooks informed teachers about prerequisite ideas or skills that students would need to understand a benchmark. They also considered whether the textbook provided appropriate assessment items that focus on understanding benchmark ideas and whether it offered teachers advice on how to use the assessment results to guide their classroom activities. The analysts rated each textbook on 24 criteria grouped under seven instructional categories. They looked for specifically defined characteristics, or indicators, of each criterion in the instructional guidance provided by a textbook.

Part 2