Project 2061
Algebra Textbooks
A Standards-Based Evaluation


Browse evaluations of algebra textbooks
Make comparisons of data
Examine Project 2061's analysis procedure
Find out how the evaluation was conducted
Learn how to use the features of this report
Explore how to use the results of the evaluation
 
Example Scenarios for Applying the Results

The following scenarios reflect how various groups or individuals might use Algebra Textbooks: A Standards-Based Evaluation, including the background material, individual textbook summary reports, and the data sets.

 

Scenario 1:

Thinking about Improving Mathematics Curriculum Materials
Scenario 2: Narrowing the List of Textbook Adoption Choices
Scenario 3: Choosing Mathematics Textbooks
Scenario 4: Planning Staff Development for Using a Textbook Series
Scenario 5: A Unit for an Undergraduate or Graduate Mathematics Methods Course
Scenario 1: Thinking about Improving Mathematics Curriculum Materials
 

Motivated by the Third International Mathematics and Science Study (TIMSS), the TIMSS-Repeat (TIMSS-R), and state assessment results, and by a commitment to improve mathematics achievement for all students, a district committee composed of teachers and parent leaders has become interested in changing the mathematics curriculum materials being used in teaching algebra. They have reviewed and studied the National Council of Teachers of Mathematics' (NCTM) Principles and Standards for School Mathematics and AAAS's Benchmarks for Science Literacy, deciding that these documents would be useful in guiding their work. They wonder whether they should adopt the newest edition of their current commercial textbook, which claims to address NCTM Standards; consider some of the new materials they have heard about; or consider writing their own curriculum materials so they could adapt to their own students' needs and interests. After visiting the Project 2061 Web site and finding the Algebra Textbook Evaluation report, they do the following:

(a) The group uses the information in "Project 2061 Analysis Procedure," which provides details about the textbook analysis procedure, rating criteria, and background research. They use this material to study, reflect on, and discuss the key content learning goals in mathematics, the instructional criteria, and the research on teaching and learning relevant to these criteria.

(b) The group agrees that their students should understand concepts well by the end of an Algebra I course and that Category II (Building on Student Ideas about Mathematics) and Category V (Promoting Student Thinking about Mathematics) are especially important to them.

(c) They see the Typical Sightings charts in the report and decide to become more familiar with the Project 2061 ratings. They use the link to publishers' Web sites to order examination copies of textbook series. They focus on textbooks that vary on coverage of the idea sets and on instructional Categories II and V.

(d) When the examination copies of the textbooks arrive, the group browses all of the data sets for each of the books they have decided to review. They use the Typical Sightings charts to find lessons in each of the books that address the instructional criteria, learning what characteristics contribute to high and low ratings, especially on the content and instructional criteria of most interest to them.

(e) After several weeks of study, the group is ready to make some recommendations to the district on the key criteria and strategies that should be followed in the coming textbook adoption.

Scenario 2: Narrowing the List of Textbook Adoption Choices
 

A district mathematics textbook adoption committee has been charged with selecting three top textbooks for secondary mathematics. The early part of the adoption process has produced a set of basic requirements for the books: strong development of algebra content, support for teachers, adaptability for diverse students, alignment with national standards, strong assessment, and a high level of student engagement.

The committee has reviewed the Project 2061 Web site. Their work to narrow the list of candidate textbook series includes the following:

(a) The committee studies the individual content reports and the Instruction Highlights graphs for each of the 12 textbooks. In addition to all of the algebra ideas examined in the content reports, they focus on instructional Category III (Engaging Students in Mathematics), Category VI (Assessing Student Progress in Mathematics), and Category VII (Enhancing the Mathematics Learning Environment), in which they have the most interest. They note which textbooks have acceptable content for the Functions, Operations, and Variables Idea Sets, and relatively high ratings on instructional Categories III and VI, and strong supportive commentary in the Instructional Analysis reports for Category VII.

(b) The preliminary review has identified four or five textbooks that have some strengths related to the committee's requirements. They decide to use the data sets to do a more in-depth review of the criteria. Using the Compare feature, the committee members do a comparison of the Content Analysis of algebra idea sets for each of five textbooks. This results in the elimination of one of the textbooks, which does not develop the ideas in the Variables Idea Set sufficiently.

(c) The committee members continue with the remaining four series, comparing instructional Categories III, VI, and VII using both the Instructional Analysis and the Graphs. These data help them to select their three finalist textbooks.

(d) In addition to having a great deal of evidence to support their selections, the committee has developed a set of recommendations for selecting the best textbooks for their students, and has a tentative ranking of the three finalists just in case anyone is interested.

Scenario 3: Choosing Mathematics Textbooks
 

A school mathematics textbook adoption committee has been charged with guiding the selection of secondary mathematics textbooks. They have narrowed the list to three top candidates and plan to continue having wide engagement by the teachers and community. The committee has already reviewed the Project 2061 report. They continue to focus on key ideas and instructional criteria but also decide to consider other strengths and weaknesses in key content and in the 24 instructional criteria. Their work to make a final selection includes the following:

(a) In order to align the content with state and district standards, the committee adds standards statements from their state and district frameworks to the comparison chart in "Project 2061 Analysis Procedure." They note that some of the ideas from the three idea sets are more closely aligned than others with their own standards. They decide to pay close attention to these particular parts of the idea sets when looking at the content reports of the textbooks.

(b) The committee decides to use the Sightings by Idea Set charts to guide an in-depth examination of the books. They print the sightings, then three two-person teams—one for each idea set—volunteer to compare the three books on each idea set. They agree to focus on the ideas in the idea set that are best aligned with the district and state standards. They pay special attention to how the concepts are developed and make notes. Afterward, the teams meet and compare their notes and compare their own findings with Project 2061's Content Analysis reports. They use the Compare feature, viewing the Content Analyses side by side for each of the three books. Finally, the committee prepares a ranking of the three books on the content of the ideas contained in their district and state standards. One of the books appears stronger than originally thought, because even though it does not address all of the ideas in each idea set, many of those ideas are not included in the district framework. The committee does decide to recommend a study by the framework committee on whether these ideas should be included in the district mathematics framework.

(c) Using the Compare feature, the committee compares the Instructional Analyses for the three textbooks, taking into account their content findings. They keep track of the ranking of the three textbooks for each of the 21 criteria that was rated, and make special note of instructional strength on key mathematics content ideas. At the end of this process, two of the textbooks appear to be stronger.

(d) Using the Compare feature, the committee examines each of the four Graphs side by side for the top two textbooks. They pay special attention to criteria that are important for their students and teachers, and to the graphs of ratings on individual idea sets that are most important. They keep track of the ranking of the two textbooks, accumulated across all of the comparisons.

(e) At a summary meeting, the committee discusses their data and evidence. A clear consensus is reached on the top-rated textbook. The committee not only has made an evidence-based decision but also has compiled a great deal of information about the textbook that they are eager to share with their colleagues and the community. Further, they have a list of areas that they believe will require supplementing, even for the top-rated book. They also know there are some key areas for which teachers will need professional development in order to implement the textbook successfully.

Scenario 4: Planning Staff Development for Using a Textbook Series
 

A district has just adopted a secondary mathematics series. The mathematics curriculum specialist is responsible for providing staff development for the teachers who will be using the books, and for considering additional materials that might be needed both for staff development and for student use. She knows that some of the teachers may need assistance in order to implement the conceptual development approach and emphasis on applications in the new series. Other teachers need work on instructional strategies, especially taking account of student knowledge, using embedded assessments, and helping students to learn problem solving and to reflect on what they have learned. Working closely with the teachers, she does the following:

(a) To identify the main areas of content that require attention, she examines the report of the textbook they have adopted, focusing on the content analysis. From the Content Analysis reports, she notes that some ideas in the Operations Idea Set have minimal content coverage. She decides to examine this by printing copies of the Sightings by Idea Set charts for this idea set. With the aid of the information about the content analysis procedure, she checks some other ideas, trying to find lessons that address them. She concludes that, indeed, there is a need for additional conceptual development of some of the ideas in that idea set. She sets up a meeting with the mathematics teachers to explore possible supplementary units that might be used, including units from some of the textbooks that were rated highly by AAAS.

(b) Turning to the instructional criteria, she examines the Instruction Highlights graph. She notes that many of the criteria are less than satisfactory, but is most concerned about the very low ratings for the criteria in Category II (Building on Student Ideas about Mathematics) and Category V (Promoting Student Thinking about Mathematics), as well as Criterion VI.3 (Using Embedded Assessment).

(c) In order to understand just what is involved in the instructional criteria she identified, she studies the instructional analysis procedure, then uses the Typical Sightings chart to find examples of the criteria in the textbook. She also studies the Instructional Ratings files to see the indicators for the criteria and the explanations that the analysts gave for assigning the low scores.

(d) With all of this information, and a better understanding of the instructional weaknesses of the textbook her district is using, she plans a series of workshops to prepare teachers to use the supplementary units that will be needed. She sets up a meeting with the assistant superintendent to present her findings and discuss support for her plan.

Scenario 5: A Unit for an Undergraduate or Graduate Mathematics Methods Course
 

A professor of secondary mathematics methods has read the report and decides that the analysis procedure, especially the instructional analysis, would make an important unit in her course. It would replace the previous unit in which she had students look through and use checklists to review textbooks as a way to familiarize themselves with adoption procedures. The objectives would include learning about the content and instructional analysis, reviewing some of the relevant research, and learning to use the report to review mathematics textbooks. She decides that a good strategy for having students understand the instructional criteria and the report would be to have them do at least some part of the analysis themselves. The following are some of the activities she does with the students:

(a) As background reading, she assigns selected pages from NCTM's Principles and Standards for School Mathematics, and AAAS's Science for All Americans and Benchmarks for Science Literacy that deal with the ideas in one of the idea sets (representing and modeling with functions) used in the Project 2061 analysis. In class, she employs the Project 2061 clarification process, guiding the students in an examination and discussion of the idea set. The comparison chart helps to identify related benchmarks and standards. She provides a chapter from a textbook and a printout of the Sightings by Idea Set for the idea set they have clarified. As an assignment for the next class, she asks the students to find five sightings of the idea set and write a brief justification of why the sightings align with the ideas. She also asks them to review the analysis procedure, with special attention to instructional Categories II and V.

(b) At the next class meeting, the students discuss their justifications of content sightings. The professor focuses their attention on the substance of the content alignment and on the specific ideas of the idea set that are addressed. She then assigns instructional Categories II through V to groups and asks the students in each group to identify the instructional criteria from their category that are addressed by the content sightings. When they are finished, the groups make brief presentations to the class, then answer questions to justify their decisions. They reconvene to consider this input and make changes. The professor assigns them the task of calculating the rating for each instructional criterion, based on the content sightings they have decided on, using the indicators and scoring guide in "The Procedure in Detail" section of "Project 2061 Analysis Procedure."

(c) At the third class meeting, the professor asks the students to reconvene in their groups, then hands each group a printout for their instructional criterion from the Instructional Ratings files. She asks each group to compare their own ratings with one another and with the ratings on the printout, then reconcile the differences. At the conclusion, she leads a discussion focused on probing their understanding of the instructional criteria.

(d) As a final unit project, the professor assigns pairs of students to use another idea set to review a textbook on content and on instructional Categories II, III, IV and V.

Copyright © 2002 American Association for the Advancement of Science