

Example
Scenarios for Applying the Results 
The following scenarios
reflect how various groups or individuals might use Algebra
Textbooks: A StandardsBased Evaluation, including the background
material, individual textbook summary reports, and the data sets.


Scenario 1:

Thinking
about Improving Mathematics Curriculum Materials 
Scenario
2: 
Narrowing
the List of Textbook Adoption Choices 
Scenario
3: 
Choosing
Mathematics Textbooks 
Scenario
4: 
Planning
Staff Development for Using a Textbook Series 
Scenario
5: 
A
Unit for an Undergraduate or Graduate Mathematics Methods Course 
Scenario
1: Thinking about Improving Mathematics Curriculum Materials 

Motivated by the Third
International Mathematics and Science Study (TIMSS), the TIMSSRepeat
(TIMSSR), and state assessment results, and by a commitment to
improve mathematics achievement for all students, a district committee
composed of teachers and parent leaders has become interested
in changing the mathematics curriculum materials being used in
teaching algebra. They have reviewed and studied the National
Council of Teachers of Mathematics' (NCTM) Principles and Standards
for School Mathematics and AAAS's Benchmarks for Science
Literacy, deciding that these documents would be useful in
guiding their work. They wonder whether they should adopt the
newest edition of their current commercial textbook, which claims
to address NCTM Standards; consider some of the new materials
they have heard about; or consider writing their own curriculum
materials so they could adapt to their own students' needs and
interests. After visiting the Project 2061 Web site and finding
the Algebra Textbook Evaluation report, they do the following:
(a) The group uses
the information in "Project 2061 Analysis Procedure," which provides
details about the textbook analysis procedure, rating criteria,
and background research. They use this material to study, reflect
on, and discuss the key content learning goals in mathematics,
the instructional criteria, and the research on teaching and learning
relevant to these criteria.
(b) The group agrees
that their students should understand concepts well by the end
of an Algebra I course and that Category II (Building on Student
Ideas about Mathematics) and Category V (Promoting Student Thinking
about Mathematics) are especially important to them.
(c) They see the Typical
Sightings charts in the report and decide to become more familiar
with the Project 2061 ratings. They use the link to publishers'
Web sites to order examination copies of textbook series. They
focus on textbooks that vary on coverage of the idea sets and
on instructional Categories II and V.
(d) When the examination copies of the textbooks arrive, the group
browses all of the data sets for each of the books they have decided
to review. They use the Typical Sightings charts to find lessons
in each of the books that address the instructional criteria,
learning what characteristics contribute to high and low ratings,
especially on the content and instructional criteria of most interest
to them.
(e) After several weeks of study, the group is ready to make some
recommendations to the district on the key criteria and strategies
that should be followed in the coming textbook adoption.

Scenario
2: Narrowing the List of Textbook Adoption Choices 

A district mathematics
textbook adoption committee has been charged with selecting three
top textbooks for secondary mathematics. The early part of the
adoption process has produced a set of basic requirements for
the books: strong development of algebra content, support for
teachers, adaptability for diverse students, alignment with national
standards, strong assessment, and a high level of student engagement.
The committee has reviewed
the Project 2061 Web site. Their work to narrow the list of candidate
textbook series includes the following:
(a) The committee studies
the individual content reports and the Instruction Highlights
graphs for each of the 12 textbooks. In addition to all of the
algebra ideas examined in the content reports, they focus on instructional
Category III (Engaging Students in Mathematics), Category VI (Assessing
Student Progress in Mathematics), and Category VII (Enhancing
the Mathematics Learning Environment), in which they have the
most interest. They note which textbooks have acceptable content
for the Functions, Operations, and Variables Idea Sets, and relatively
high ratings on instructional Categories III and VI, and strong
supportive commentary in the Instructional Analysis reports for
Category VII.
(b) The preliminary
review has identified four or five textbooks that have some strengths
related to the committee's requirements. They decide to use the
data sets to do a more indepth review of the criteria. Using
the Compare feature, the committee members do a comparison of
the Content Analysis of algebra idea sets for each of five textbooks.
This results in the elimination of one of the textbooks, which
does not develop the ideas in the Variables Idea Set sufficiently.
(c) The committee members
continue with the remaining four series, comparing instructional
Categories III, VI, and VII using both the Instructional Analysis
and the Graphs. These data help them to select their three finalist
textbooks.
(d) In addition to
having a great deal of evidence to support their selections, the
committee has developed a set of recommendations for selecting
the best textbooks for their students, and has a tentative ranking
of the three finalists just in case anyone is interested.

Scenario
3: Choosing Mathematics Textbooks 

A school mathematics
textbook adoption committee has been charged with guiding the
selection of secondary mathematics textbooks. They have narrowed
the list to three top candidates and plan to continue having wide
engagement by the teachers and community. The committee has already
reviewed the Project 2061 report. They continue to focus on key
ideas and instructional criteria but also decide to consider other
strengths and weaknesses in key content and in the 24 instructional
criteria. Their work to make a final selection includes the following:
(a) In order to align the content with state and district standards,
the committee adds standards statements from their state and district
frameworks to the comparison chart in "Project 2061 Analysis
Procedure." They note that some of the ideas from the three
idea sets are more closely aligned than others with their own
standards. They decide to pay close attention to these particular
parts of the idea sets when looking at the content reports of
the textbooks.
(b) The committee decides
to use the Sightings by Idea Set charts to guide an indepth examination
of the books. They print the sightings, then three twoperson
teams—one for each idea set—volunteer to compare the
three books on each idea set. They agree to focus on the ideas
in the idea set that are best aligned with the district and state
standards. They pay special attention to how the concepts are
developed and make notes. Afterward, the teams meet and compare
their notes and compare their own findings with Project 2061's
Content Analysis reports. They use the Compare feature, viewing
the Content Analyses side by side for each of the three books.
Finally, the committee prepares a ranking of the three books on
the content of the ideas contained in their district and state
standards. One of the books appears stronger than originally thought,
because even though it does not address all of the ideas in each
idea set, many of those ideas are not included in the district
framework. The committee does decide to recommend a study by the
framework committee on whether these ideas should be included
in the district mathematics framework.
(c) Using the Compare feature, the committee compares the Instructional
Analyses for the three textbooks, taking into account their content
findings. They keep track of the ranking of the three textbooks
for each of the 21 criteria that was rated, and make special note
of instructional strength on key mathematics content ideas. At
the end of this process, two of the textbooks appear to be stronger.
(d) Using the Compare
feature, the committee examines each of the four Graphs side by
side for the top two textbooks. They pay special attention to
criteria that are important for their students and teachers, and
to the graphs of ratings on individual idea sets that are most
important. They keep track of the ranking of the two textbooks,
accumulated across all of the comparisons.
(e) At a summary meeting, the committee discusses their data and
evidence. A clear consensus is reached on the toprated textbook.
The committee not only has made an evidencebased decision but
also has compiled a great deal of information about the textbook
that they are eager to share with their colleagues and the community.
Further, they have a list of areas that they believe will require
supplementing, even for the toprated book. They also know there
are some key areas for which teachers will need professional development
in order to implement the textbook successfully.

Scenario
4: Planning Staff Development for Using a Textbook Series 

A district has just
adopted a secondary mathematics series. The mathematics curriculum
specialist is responsible for providing staff development for
the teachers who will be using the books, and for considering
additional materials that might be needed both for staff development
and for student use. She knows that some of the teachers may need
assistance in order to implement the conceptual development approach
and emphasis on applications in the new series. Other teachers
need work on instructional strategies, especially taking account
of student knowledge, using embedded assessments, and helping
students to learn problem solving and to reflect on what they
have learned. Working closely with the teachers, she does the
following:
(a) To identify the main areas of content that require attention,
she examines the report of the textbook they have adopted, focusing
on the content analysis. From the Content Analysis reports, she
notes that some ideas in the Operations Idea Set have minimal
content coverage. She decides to examine this by printing copies
of the Sightings by Idea Set charts for this idea set. With the
aid of the information about the content analysis procedure, she
checks some other ideas, trying to find lessons that address them.
She concludes that, indeed, there is a need for additional conceptual
development of some of the ideas in that idea set. She sets up
a meeting with the mathematics teachers to explore possible supplementary
units that might be used, including units from some of the textbooks
that were rated highly by AAAS.
(b) Turning to the
instructional criteria, she examines the Instruction Highlights
graph. She notes that many of the criteria are less than satisfactory,
but is most concerned about the very low ratings for the criteria
in Category II (Building on Student Ideas about Mathematics) and
Category V (Promoting Student Thinking about Mathematics), as
well as Criterion VI.3 (Using Embedded Assessment).
(c) In order to understand
just what is involved in the instructional criteria she identified,
she studies the instructional analysis procedure, then uses the
Typical Sightings chart to find examples of the criteria in the
textbook. She also studies the Instructional Ratings files to
see the indicators for the criteria and the explanations that
the analysts gave for assigning the low scores.
(d) With all of this information, and a better understanding of
the instructional weaknesses of the textbook her district is using,
she plans a series of workshops to prepare teachers to use the
supplementary units that will be needed. She sets up a meeting
with the assistant superintendent to present her findings and
discuss support for her plan.

Scenario
5: A Unit for an Undergraduate or Graduate Mathematics Methods Course 

A professor of secondary
mathematics methods has read the report and decides that the analysis
procedure, especially the instructional analysis, would make an
important unit in her course. It would replace the previous unit
in which she had students look through and use checklists to review
textbooks as a way to familiarize themselves with adoption procedures.
The objectives would include learning about the content and instructional
analysis, reviewing some of the relevant research, and learning
to use the report to review mathematics textbooks. She decides
that a good strategy for having students understand the instructional
criteria and the report would be to have them do at least some
part of the analysis themselves. The following are some of the
activities she does with the students:
(a) As background reading, she assigns selected pages from NCTM's
Principles and Standards for School Mathematics, and AAAS's
Science for All Americans and Benchmarks for Science
Literacy that deal with the ideas in one of the idea sets
(representing and modeling with functions) used in the Project
2061 analysis. In class, she employs the Project 2061 clarification
process, guiding the students in an examination and discussion
of the idea set. The comparison chart helps to identify related
benchmarks and standards. She provides a chapter from a textbook
and a printout of the Sightings by Idea Set for the idea set they
have clarified. As an assignment for the next class, she asks
the students to find five sightings of the idea set and write
a brief justification of why the sightings align with the ideas.
She also asks them to review the analysis procedure, with special
attention to instructional Categories II and V.
(b) At the next class meeting, the students discuss their justifications
of content sightings. The professor focuses their attention on
the substance of the content alignment and on the specific ideas
of the idea set that are addressed. She then assigns instructional
Categories II through V to groups and asks the students in each
group to identify the instructional criteria from their category
that are addressed by the content sightings. When they are finished,
the groups make brief presentations to the class, then answer
questions to justify their decisions. They reconvene to consider
this input and make changes. The professor assigns them the task
of calculating the rating for each instructional criterion, based
on the content sightings they have decided on, using the indicators
and scoring guide in "The Procedure in Detail" section
of "Project 2061 Analysis Procedure."
(c) At the third class meeting, the professor asks the students
to reconvene in their groups, then hands each group a printout
for their instructional criterion from the Instructional Ratings
files. She asks each group to compare their own ratings with one
another and with the ratings on the printout, then reconcile the
differences. At the conclusion, she leads a discussion focused
on probing their understanding of the instructional criteria.
(d) As a final unit
project, the professor assigns pairs of students to use another
idea set to review a textbook on content and on instructional
Categories II, III, IV and V.

