Textbook Evaluation Reports
For students to be able to achieve mathematics benchmarks, educators need textbooks that not only address these important concepts and skills but also provide instructional strategies that are likely to help students learn them. With that goal in mind, Project 2061s textbook evaluation has several distinct features that set it apart from other evaluation efforts.
An evidence-based analysis procedure
Six independent, two-person teams made up of classroom teachers and college-level mathematics educators evaluated each textbook. Using custom-designed software to document what they found, analysts examined the student and teacher editions of the textbooks to identify specific lessons, activities, teacher notes, assessments, and other materials that addressed ideas in each of six selected benchmarks drawn from Project 2061s Benchmarks for Science Literacy. Three questions were considered to determine whether the content addressed the chosen benchmarks: Did the content address the substance of the benchmark or was there only a topic match? Did the content match the sophistication (appropriate age level) of the benchmark? Did the content match part(s) or all of the benchmark? Analysts then rated that content using a set of research-based instructional criteria.
Key mathematics benchmarks
In creating its analysis procedure, Project 2061 found that studying a materials treatment of a relatively small but carefully chosen set of benchmarks can reveal the strengths and weaknesses of the material as a whole, particularly its instructional design and guidance. Six benchmarks representing three important mathematical strandsnumber, geometry, and algebrawere used to conduct the analysis. These benchmarks provide a sufficiently comprehensive example of the core mathematics content likely to appear in any middle grades material. Specifically, they include a concept benchmark dealing with fractions and operations on them; a skill benchmark dealing with equivalent forms of numbers; a concept and a skill benchmark dealing with properties of shapes and computations of circumference, area, and volume; and two concept benchmarks dealing with graphing and equations.
Research-based instructional criteria
To judge how well a textbook was likely to help students learn these key concepts and skills, the analysts applied a set of carefully developed instructional criteria to the texts treatment of each selected benchmark. The criteria were derived from research on learning and teaching and on the craft knowledge of experienced educators. For example, analysts rated the extent to which textbooks informed teachers about prerequisite ideas or skills that students would need to understand a benchmark. They also considered whether the textbook provided appropriate assessment items that focus on understanding benchmark ideas and whether it offered teachers advice on how to use the assessment results to guide their classroom activities. The analysts rated each textbook on 24 criteria grouped under seven instructional categories. They looked for specifically defined characteristics, or indicators, of each criterion in the instructional guidance provided by a textbook.