Appendix C
METHODOLOGY
Evaluation Methods
Report Preparation: 
The following Project 2061 staff members were
responsible for supervising the development of the procedure,
training analysts, compiling data, and preparing the summary
reports.
Gerald Kulm, Program Director
Kathleen Morris, Senior Program Associate
Laura Grier, Project Coordinator
These staff members completed the final editing of the
reports and prepared them for publication.
Mary Koppal, Communications Director
Susan Shuttleworth, Senior Editor
The following staff members designed and produced the Web
version of the report and the CDROM.
Francis Molina, Electronic Resources
Manager
Samuel Kim, Technology Specialist
Thelxi Proimaki, Technology Specialist
Alan Stonebraker, Technology Specialist
Brian Sweeney, Technology Specialist
Christina Bautista, Intern
Vinod Rawat, Technology Assistant/Intern
Moojan ZareParsi, Intern

Reviewers: 
The analysts who reviewed and rated the texts
were experienced mathematics teachers and university mathematics
and mathematics education faculty trained in the use of the
Project 2061 procedure. The reviewers were evenly divided
between experienced, practicing classroom teachers and mathematics
educators who were knowledgeable about research on mathematics
learning and teaching. All of the reviewers were highly capable
in mathematics content. Reviewers were formed into 12 teams
of two persons each, at least one of whom was an experienced
classroom teacher. The reviewers were:
Donna Cook, Crossett Brook Middle School (VT)
Joe Cook, Paige Academy (NY)
Michelle Crowley, Northern Kentucky University
Peg Darcy, Louisville Schools (KY)
Mark Deegan, Springfield (VA) High School
Sassan Dehghan, University of Maryland
Kay Dighans, Quaw Elementary School (MT)
Florence Fasanelli, Washington, DC
Ben Flora, Morehead State University (KY)
Mary Ellen Foley, Louisiana State University – Shreveport
Marshall Gordon, The Park School (MD)
Linda Hackett, American University, Washingtin, DC
Steve Hays, Largo High School (MD)
Willis Johnson, University of Kentucky
Bill Kunnecke, Marshall County (KY) Schools
Jan McDowell, Noe Middle School (KY)
Alice Mikovch, Western Kentucky University
Joan Montgomery, Pilgrim Christian Academy (NY)
Linda Montgomery, Berea County (KY) Schools
Sue Reehm, Eastern Kentucky University
Kay Reinke, Oklahoma State University
Patty Schumacher, Telford County (PA) Schools
Faye Stevens, Cadiz County (KY) Schools
Diane Surati, Crossett Brook Middle School (VT)
James Telese, University of Texas – Brownsville
Sri Viswanathan, Fairfax County (VA) Schools
Connie Widmer, Northern Kentucky University
Grace Wood, Fort Peck Community College (MT)

Training: 
About half of the reviewers had one or more
previous experiences with the Project 2061 analysis procedure
in reviewing and rating middle grades mathematics materials.
The persons who had not had previous reviewing experience
attended an intensive threeday training session immediately
before the analysis. In the session, they learned the analysis
procedure and applied it to the analysis of a middle grades
mathematics textbook that had been reviewed earlier as part
of the reliability study. Their ratings on each criterion
were discussed and compared with the ratings that had been
assigned by experienced reviewers. After the training, each
of these reviewers was teamed with an experienced reviewer.

Design: 
Each team rated two mathematics benchmarks,
one conceptual and one skill, from one of the three mathematics
strands: Number, Geometry, and Algebra. Each team reviewed
a total of five sets of books, using the same two benchmarks
to review the textbooks over a period of two days. Most of
the materials consisted of three textbooks, one for 6^{th},
7^{th}, and 8^{th} grades. Two of the sets
of materials consisted of a number of units, usually three
or four per grade, that were relevant to a particular pair
of benchmarks. Two teams completed a review for each textbook
independently.

Analysis: 
Preliminary sightings for each benchmark were
identified by Project 2061 staff prior to the analysis. After
clarifying each benchmark, the reviewers checked the sightings
in the textbook, making necessary additions, deletions, or
revisions. Reviewers then aligned sightings with the relevant
instructional criteria and used indicators and rating guides
to rate how well the textbook addressed each criterion. After
completing the analysis for a benchmark, teams met to reconcile
their ratings and receive necessary clarifications from Project
2061 staff if ratings differed significantly. Teams submitted:
(1) the content sightings for each benchmark; (2) the sightings
and justifications for each instructional criterion; (3) the
rating of each instructional indicator [Met, Not Met, Unsure];
and, (4) the rating for each instructional criterion [High,
Medium, Low, None] including supporting notes.

Reports 
Project 2061 staff took these data plus notes derived from
debriefing the teams and prepared detailed reports on each
textbook. Summaries of these reports were published on Project
2061's Web site and in book form. The detailed reports and
all the analysis data were made available on CDROM.

The Project 2061 Analysis Procedure for Mathematics
Curriculum Materials
Introduction
Deciding which curriculum materials to use is one of the most important
professional judgments that educators make. Textbook adoption committees make
recommendations that influence instruction for years to come, and the daily decisions that
teachers make about which teaching units or chapters to use and how to use them largely
determine what and how students will be expected to learn.
Such important decisions require a valid and reliable method for evaluating the quality
of curriculum materials. Even an indepth review of the topics covered by a textbook or a
teaching unit may not be sufficient to determine whether the material will actually help
students learn that content. What is needed is a manageable process for examining
curriculum materials that gets below the surface by focusing intensely on the
appropriateness of content and the utility of instructional design.
With funding from the National Science Foundation and in collaboration with hundreds of
K12 teachers, curriculum specialists, teacher educators, scientists, and materials
developers, Project 2061 of the American Association for the Advancement of Science (AAAS)
has been developing a process for analyzing curriculum materials. Field tests suggest that
Project 2061's curriculumanalysis procedure will not only serve the materials adoption
needs of the schools but also help teachers revise existing materials to increase their
effectiveness, guide developers in the creation of new materials, and contribute to the
professional development of those who use it.
Specific Learning Goals Are Key
Until recently, there was nothing against which to judge appropriateness of
content and utility of instructional design. Now, as a result of the standardsbased
reform movement in education, these judgments can be made with a high degree of
confidence. In mathematics, for example, the appearance of Science for All Americans
(AAAS, 1989), Curriculum and Evaluation Standards for School Mathematics (NCTM,
1989), and Benchmarks for Science Literacy (AAAS, 1993) has made it possible to
make more thoughtful decisions about curriculum materials than ever before.
Although the Project 2061 curriculumanalysis
procedure was developed using the learning goals in Benchmarks and the mathematics
and science standards, subsequent work has indicated that some state education frameworks
also can be used. Indeed, the process would seem to apply to any K12 school subject for
which specific learning goals have been agreed upon. These goals must be explicit
statements of what knowledge and skills students are expected to learn, and they must be
precise. Vague statements such as "students should understand fractions" are not
adequate. Instead, consider this benchmark dealing with the meanings of fractions that
students should know by the end of the eighth grade:
Students should know that:
"The expression a/b can mean different things: a parts of size 1/b
each, a divided by b, or a compared to b."
At its simplest level, the Project 2061 curriculumanalysis procedure involves the
following five steps:
 Identify specific learning goals to serve as the intellectual basis for the analysis.
This is done before beginning to examine any curriculum materials. The source for
appropriate goals can be national standards or documents such as those mentioned above,
state or local standards and curriculum frameworks, or sources like them. To be useful,
the goals must be precise in describing the knowledge or skills they intend students to
have. If the set of goals is large, a representative sample of them should be selected for
purposes of analysis.
 Make a preliminary inspection of the curriculum materials to see whether they are likely
to address the targeted learning goals. If there appears to be little or no
correspondence, the materials can be rejected without further analysis. If the outlook is
more positive, go on to a content analysis.
 Analyze the curriculum materials for alignment between content and the selected learning
goals. The purpose here is to determine, citing evidence from the materials, whether the
content in the material matches specific learning goals, not just whether the topic
headings are similar. At the topic level, alignment is never difficult, since most topics,
proportions, equations, graphing, and so forth, lack specificity making them easy to
match. If the results of this analysis are positive, then reviewers can take the next
step.
 Analyze the curriculum materials for alignment between instruction and the selected
learning goals. This involves estimating the degree to which the materials (including
their accompanying teacher's guides) reflect what is known generally about student
learning and effective teaching and, more important, the degree to which they support
student learning of the specific knowledge and skills for which a content match has been
found. Again, evidence from the materials must be shown.
 Summarize the relationship between the curriculum materials being evaluated and the
selected learning goals. The summary can take the form of a profile of the selected goals
in terms of the content and instruction criteria, or a profile of the criteria in terms of
the selected goals. In either case, a statement of strengths and weaknesses should be
included. With this information in hand, reviewers can make more knowledgeable adoption
decisions and suggest ways for improving the examined materials.
In addition to its careful focus on matching content and instruction to very specific
learning goals, the Project 2061 procedure has other features that set it apart. For
example, its emphasis on collecting explicit evidence (citing page numbers and other
references) of a material's alignment with learning goals adds rigor and reliability to
decisions about curriculum materials. Similarly, the Project 2061 procedure calls for a
team approach to the analytical task, thus providing opportunities for reviewers to defend
their own judgments about materials and to question those of other reviewers. These and
other characteristics help make participation in the analytical process itself a powerful
professional development experience.
The Project 2061 CurriculumAnalysis Procedure in Detail
To provide a better sense of how the procedure works, the following describes in
more detail each step in the procedure. The description pays particular attention to the
various criteria used to evaluate the instructional effectiveness of materials.
Identify specific learning goals to serve as the intellectual basis for the
analysis. After reviewers have agreed upon a set of learning goals as a framework for
the analysis, the task is then to choose specific learning goals that will serve as the
focus of further study.
When evaluating standalone curriculum units that
cover a relatively short period of time, it might be possible and worthwhile to analyze
all of the learning goals that appear to be targeted by the material. However, in the
evaluation of yearlong courses or multiyear programs, this becomes impractical.
Therefore, a crucial step in the analysis procedure is the sampling of a few learning
goals that will lead to valid and reliable generalizations about the material.
Sampling of standards should be representative of the whole set of goals
specified in the framework or standards being applied and should reflect the reviewers'
needs. For example, if the review committee's task is to select a course in high school
Algebra that is aligned with a state mathematics framework or NCTM Standards, it
might identify a sample of learning goals from important topic areas (e.g., number
systems, equations, graphs, functions) and include learning goals that reflect different
types of knowledge (e.g., skills, conceptual understanding, problem solving). When
examining elementary or middle school mathematics materials, one would probably want to
broaden the range of learning goal statements examined to include important strands in
mathematics (e.g., number, geometry, algebra, and statistics).
Make a preliminary inspection of the curriculum materials to see whether they are
likely to address the targeted learning goals. Once learning goal statements have been
selected, the next step is to make a first pass at the materials to identify those whose
content appears to correspond reasonably well to the learning goals. Materials that do not
meet these initial criteria are not analyzed further.
Reviewers then examine materials on the shortened list more carefully to locate and
record places where each selected learning goal seems to be targeted (e.g., particular
readings, experiments, discussion questions). If several sightings are found for some or
all of the sample learning goals in the material, then these sightings will be looked at
more carefully in subsequent steps of the analysis. If, on the other hand, sightings
cannot be found for a significant number of the sample learning goals, then the material
is dropped from the list.
Analyze the curriculum materials for alignment between content and the selected
learning goals. This analysis is a more rigorous examination of the link between the
subject material and the selected learning goals and involves giving precise attention to
both ends of the match; the precise meaning of the learning goal on one end, and the
precise intention of the material on the other.
With respect to each of the sampled learning goals, the material is examined using such
questions as:
 Does the content called for in the material address the substance of a specific learning
goal or only the learning goal's general "topic"?
 Does the content reflect the level of sophistication of the specific learning goal, or
are the activities more appropriate for targeting learning goals at an earlier or later
grade level?
 Does the content address all parts of a specific learning goal or only some? (While it
is not necessary that any particular unit would address all of the ideas in a learning
goal or standard, the K12 curriculum as a whole should do so. The purpose of this
question is to provide an account of precisely what ideas are treated.)
In addition, an attempt is made to estimate the degree of overlap between the
material's content and the set of learning goals of interest. Thus, this step in the
analysis is designed to answer questions regarding the material's inclusion of content
that is not required for reaching mathematics literacy and the extent to which the
material distinguishes between essential and nonessential content. (While distinguishing
content essential for literacy from nonessential content in material might seem to be a
luxury, it assists teachers in determining the range of students for which the material
can be used. Identifying the nonessential material makes it easier for the teacher to
direct better students to enrichment activities and allows students themselves to avoid
overload from ideas that go beyond what is vital.)
Analyze the curriculum materials for alignment between instruction and the selected
learning goals. The purpose here is to estimate how well material addresses targeted
learning goals from the perspective of what is known about student learning and effective
teaching. The criteria for making the judgments in the instructional analysis are derived
from research on learning and teaching and on the craft knowledge of experienced
educators. In the context of mathematics literacy, these are summarized in Chapter 13,
"Effective Learning and Teaching," of Science for All Americans and in
Chapter 15, "The Research Base," of Benchmarks for Science Literacy. From
these and other sources, seven criteria categories (shown below) have been identified to
serve as a basis for the instructional analysis.
Category I: Identifying a Sense of Purpose
Part of planning a coherent curriculum involves deciding on its purposes and on
what learning experiences will likely contribute to achieving those purposes. But while
coherence from the designers’ point of view is important, it may be inadequate to
give students the same sense of what they are doing and why. This category includes
criteria to determine whether the material attempts to make its purposes clear and
meaningful to the student and genuinely relates lessons to the unit purpose.
I.1 Conveying Unit Purpose: Does the material convey an overall sense of
purpose and direction that is understandable and motivating to students? 
Clarification:
This criterion involves examining whether the material begins with (or early on
presents) an overarching question or problem to be addressed by the unit (e.g., How can a
graph help to make predictions?), or a representation of what will be learned (e.g., a
concept map of the main ideas that will be explored), or otherwise identifies a purpose
for the unit or chapter for the students (e.g., a clear statement of objectives, using
know terms). The problem, question, representation or purpose provided by the material
should be explicit and comprehensible by the students, and it should be plausible that it
would be interesting and/or motivating to them. A material that begins with abstractions
or phenomena outside students’ range of perception, understanding, or knowledge does
not adequately meet the criterion. However, a material that starts with an unfamiliar but
highly interesting phenomenon that is likely to motivate students may meet the criterion.
Providing students with a sense of purpose for a whole unit or chapter is not always
possible (for example, there may not be a single question or problem that is broad enough
to foreshadow all learning goals in the unit) or even desirable (for example, providing a
purpose on a large scale can lead to a complex sequence of activities that is too
demanding on the memory of younger students). In such cases, it may be sufficient for the
material to frame sections within a unit rather than the whole unit or chapter.
Indicators of meeting Criterion I.1:
1. The purpose is presented to students explicitly (or implicitly through a problem,
question, or representation).
2. The purpose is likely to be comprehensible to students.
3. The purpose is likely to be interesting and/or motivating to students.
4. Students are given an opportunity to think about and discuss the purpose.
5. Most activities or lessons are consistent with the stated purpose.
6. The material returns to the stated purpose at the end of the unit or chapter.
Scoring Scheme:
High: The material meets all 6 indicators.
Medium: The material meets indicator 1, along with 3 of the other 5 indicators.
Low: The material meets indicator 1, along with 1 of the other 5 indicators.
None: The material does not meet any of the indicators.
I.2 Conveying Lesson Purpose: Does the material convey the purpose of each
activity or lesson and its relationship to others? 
Clarification:
The question is whether the purpose of individual activities or lessons (as opposed to
the whole unit) is made apparent to the students and whether there are logical transitions
and connections between activities or lessons. If a classroom visitor asked students what
they were doing and why, is there reason to think they would know?
The purpose of individual activities or lessons could be brought out through the text,
teacher comments (suggested in the material), and/or student responses to questions. For
example, the purpose of gathering data about the heights of students in the class (to
learn about dispersion of data) might be brought out by text explanation, teacher
explanation, or by the students coming up with a description of the spread of data.
Indicators of meeting Criterion I.2:
1. The material conveys or prompts teachers to convey the purpose of each activity or
lesson to students.
2. Each activity encourages each student to think about the purpose of the activity or
lesson.
3. The material conveys or prompts teachers to convey to students how each activity or
lesson relates to the other activities.
4. The material periodically engages students in thinking about what they have learned so
far and what they need to learn/do next.
Scoring Scheme:
High: The material meets all 4 indicators.
Medium: The material meets 3 of the 4 indicators.
Low: The material meets 1 of the 4 indicators.
None: The material does not meet any of the indicators.
I.3 Justifying Sequence of Activities: Does the material involve students in a
logical or strategic sequence of activities (versus a collection of activities) that build
toward understanding of the ideas in the unit or chapter purpose? 
Clarification:
The issue here is whether there is a logical or strategic sequence of activities in
the material and whether this logic or strategy is made explicit to the teacher or just
inferred by the reviewer. A rationale or implicit reason for the sequence of activities
should be clear, providing the teacher and students with a sense of making progress toward
the purpose of the unit or chapter.
Indicators of meeting Criterion I.3:
1. The material provides a rationale for the overall sequence of activities or
lessons.
2. If no rationale for the overall sequence of activities or lessons is provided, the
reviewer can identify one.
3. The sequence of activities reflects the stated or inferred rationale or purpose.
Scoring Scheme:
High: The material meets indicators 1 and 3.
Medium: The material meets indicators 2 and 3.
Low: The material meets indicator 1 or 2.
None: The material does not meet any of the indicators.
Category II: Building on Student Ideas about Mathematics
Fostering better understanding in students requires taking time to attend to the
ideas they already have, both ideas that are incorrect and ideas that can serve as a
foundation for subsequent learning. Such attention requires that teachers be informed
about prerequisite ideas/skills needed for understanding a benchmark and what their
students’ initial ideas are—in particular, the ideas that may interfere with
learning mathematics. Moreover, teachers can help address students’ ideas if they
know what is likely to work. This category examines whether the material contains specific
suggestions for identifying and addressing student ideas.
II.1 Specifying Prerequisite Knowledge. Does the material specify and address
prerequisite knowledge/skills that are necessary to the learning of the benchmark? 
Clarification:
This criterion refers to (a) prerequisites to concepts or skills in the benchmark
examined, and (b) prerequisites to activities used in the material to teach the concepts or
skills in the benchmark examined.
(a) Understanding the ideas in benchmarks often requires that students first understand
some other "prerequisite" concepts or skills. For example, knowing what a prime
factor is is prerequisite to learning how to find the common denominator of fractions.
(b) In addition to prerequisites to specific ideas in benchmarks, additional
prerequisites may arise from the specific activities used to teach them. For example,
consider the benchmark: "Spreading data on a number line helps to see what the
extremes are, where they pile up, and where the gaps are. A summary of the data includes
where the middle is and how much spread is around it." There are no benchmarks that
are prerequisite to the ideas in this benchmark. However, as the students work through a
curriculum material, they may be engaged in investigations that target this benchmark and
involve measurement. In these cases, the material should not take for granted that
students will have developed measuring skills. Teachers should be alerted to this
prerequisite (measurement skills) and encouraged to support their students in developing
measurement skills.
Responding to the "prerequisites" criterion involves (a) making a list of
prerequisite concepts and/or skills, (b) examining whether the material alerts to any
prerequisite ideas on the list and if so which ones, (c) examining whether the material
has in fact adequately addressed the prerequisites in the same or earlier units, and (d)
examining whether the material helps students make connections between benchmarks and
their prerequisites. While a standalone unit should not be faulted for not addressing
prerequisite ideas or skills, it should be expected to make connections between benchmarks
and their prerequisites.
Indicators of meeting Criterion II.1:
1. The material makes explicit what the specific prerequisite ideas or skills are, if
any exist.
2. The material addresses the identified prerequisites in the same unit or in earlier
units.
3. The material makes connections between benchmark ideas and their prerequisites (even if
the prerequisites are addressed elsewhere).
Scoring Scheme:
High: The material meets all 3 indicators.
Medium: The material meets 2 out of the 3 indicators.
Low: The material meets 1 of the 3 indicators.
None: The material does not meet any of the indicators.
II.2 Alerting Teacher to Student Ideas. Does the material alert teachers to
commonly held student ideas (both troublesome and helpful) such as those described in Benchmarks
for Science Literacy Chapter 15: The Research Base? 
Clarification:
Researchers have identified ideas that students have in several content areas. The
issue here is whether the material informs teachers about students’ commonly held
ideas in the topic areas the material addresses. This information can help teachers (a)
understand better their own students’ ideas, (b) decide what ideas to build on and
what changes to promote, or (c) if the material is already designed in ways that build on
or attempt to change students’ commonly held ideas, to better understand the
rationale and purpose behind designed strategies and activities.
Responding to this question involves examining (1) whether there is research on
commonly held student ideas in the topic area/s that the material addresses, (2) whether
the material alerts teachers to such ideas, and (3) whether the material accurately
represents research findings. Summaries of research on students ideas in mathematics (such
as those included in Benchmarks Chapter 15: The Research Base or the NCTM Research
Ideas for the Classroom series) will be helpful to reviewers who will want to know what
ideas students typically have about the topics that the curriculum material they are
examining addresses. If there is no research on student ideas in the topic area/s that the
material addresses, the material should not be faulted for not addressing this criterion.
Indicators of meeting Criterion II.2:
1. The material lists, conveys, or identifies specific commonly held ideas that are
relevant to the benchmark (rather than just to relevant difficult topics).
2. The material clarifies/explains commonly held ideas.
3. The material explains or refers to commonly held ideas in an accurate way.
Scoring Scheme:
High: The material meets all 3 indicators.
Medium: The material meets indicators 1 and 2.
Low: The material meets indicator 1.
None: The material does not meet any of the indicators.
II.3 Assisting Teacher in Identifying Ideas. Does the material include
suggestions for teachers to find out what their students think about familiar situations
related to a benchmark before the mathematical ideas are introduced? 
Clarification:
Teachers need guidance in identifying students’ ideas in
these unresearched areas. But even in areas in which there is research
on commonly held student ideas, teachers need help in identifying
what proportion of their own students hold these ideas as well as
other more idiosyncratic ideas.
Responding to this criterion involves examining not only whether the material
encourages teachers to find out students’ ideas but also whether it provides
specific suggestions for how to do so. Suggestions may include providing (a) tasks in
which students make predictions and give their own descriptions and explanations of
concepts or skills; (b) tasks in which students are asked to represent their
understandings in drawings; (c) tasks in which students are asked to interpret information
(for example the solution to an algorithm related to ideas in the benchmark), discuss
connections with related topics, or discuss alternative solutions, or justifications; or
(d) tasks which ask students about the meaning of specific terms and/or probe for
understanding of important relationships between concepts.
Responding to the criterion also involves examining the quality of the suggestions
provided. Tasks should not focus exclusively on identifying students’ meaning for
terms. While including such tasks is useful, it is important to look for tasks in which
students make predictions and/or give explanations of concepts or procedures. It is
important that tasks make sense to students who have never studied the topic and include
questions posed in ways meaningful to the students who are not familiar with the
mathematical vocabulary. It is also important that the material encourages teachers to use
probing questions to clarify what students mean or to get more information about
students’ thinking.
Indicators of meeting Criterion II.3:
1. The material includes specific questions or tasks to assist the teacher in
identifying the ideas students have before they study the benchmark.
2. The questions or tasks are likely to be comprehensible by students before they become
familiar with the concepts, procedures, or vocabulary.
3. The material includes questions or tasks that ask students to make predictions or give
explanations of concepts or procedures (vs. focus primarily on identifying students’
meaning for terms).
4. The material suggests how teachers can use questions or tasks to understand students'
thinking and level of understanding.
Scoring Scheme:
High: The material meets all 4 indicators.
Medium: The material meets indicators 13.
Low: The material meets indicator 1.
None: The material does not meet any of the indicators.
II.4 Addressing Misconceptions. Does the material explicitly address commonly
held student ideas? 
Clarification:
The issue here is whether the material includes questions or activities that address
students’ commonly held ideas (both concepts or skills that are incorrect and those
that can serve as a foundation for subsequent learning). For example, the material may
include experiences that help students change their ideas by providing activities that
challenge students’ predictions or explanations, or prompt students to react to
commonly held misconceptions and resolve differences between these misconceptions and the
correct ideas. Alternatively, the material may include experiences that extend common
student concepts or skills that have limited scope. Pointing out misconceptions and
telling students that they should avoid them does not adequately address this criterion.
Serious difficulties, either with concepts or with skills, are not generally successfully
addressed by telling students they are wrong and providing them with the "right
answer."
In addition to providing specific suggestions to teachers about how to address commonly
held student ideas reported in the research literature, materials can be helpful by
including suggestions to teachers about how to take into account their own students’
ideas. Addressing this aspect of the criterion may involve suggesting general strategies
that teachers can use to build on or change students’ ideas, and providing examples
of how these strategies can be implemented in the classroom. For example, teachers can be
encouraged to probe students’ ideas further, juxtapose them with other students’
ideas, encourage students to compare their ideas on a topic before and after instruction
on the topic, etc.
Indicators of meeting Criterion II.4:
1. The material explicitly addresses commonly held ideas related to the benchmark (if
there is research on these ideas).
2. The material includes questions, tasks, or activities that are likely to help students
progress from their initial ideas, for example, by:
(a) explicitly challenging students’ ideas, for example, by comparing their
predictions to what actually happens.
(b) prompting students to contrast commonly held ideas and the correct concept or
procedure, and resolve differences between them.
(c) extending correct commonly held ideas that have limited scope.
3. The material suggests general strategies for addressing student ideas related to the
benchmark.
Scoring Scheme:
High: The material meets all 3 indicators.
Medium: The material meets indicators 1 and 2.
Low: The material meets any one indicator.
None: The material does not meet any of the indicators.
Category III: Engaging Students in Mathematics
Much of the point of mathematics is finding patterns and modeling ideas and
relationships in terms of a small number of generalizations or ideas. For students to
appreciate the power of mathematics, they need to have a sense of the range and complexity
of ideas and applications that mathematics can explain or model. "Students need to
get acquainted with the things around them—including devices, organisms, materials,
shapes, and numbers—and to observe them, collect them, handle them, describe them,
become puzzled by them, ask questions about them, argue about them, and then try to find
answers to their questions." (Science for All Americans, p. 201)
III.1 Providing Variety of Contexts. Does the material provide experiences with
mathematics in multiple, different contexts? 
Clarification:
Mathematicians and others construct and use mathematical knowledge to describe,
explain, predict, and design realworld objects, systems, or events as well as abstract
relationships. Therefore, mathematical ideas need to be connected to meaningful problems,
situations, and the real world. The question is whether the material provides a sufficient
number of problems, experiences, or events in a variety of contexts to support the ideas
put forth in benchmarks. The material can provide experiences with problems, situations,
systems, or events directly through handson activities or demonstrations (firsthand
experiences) or indirectly, through the use of text, graphs, diagrams, computer screens,
videos, pictures, models, etc.
Indicators of meeting Criterion III.1:
1. The experiences with objects, applications, and materials are "right on
target" in addressing the content of the benchmark.
2. The material provides an appropriate variety of experiences with objects, applications,
and materials.
Scoring scheme:
High: The material meets indicators 1 and 2.
Medium: The material meets indicator 1.
Low: The material meets indicator 2.
None: The material does not meet any of the indicators.
III.2 Providing Firsthand Experiences. Does the material include
activities that promote firsthand experiences with the benchmark ideas, when practical? 
Clarification:
Students can learn more readily about things that are tangible and accessible to their
senses; thus students, especially younger ones, will benefit most from firsthand
experiences with the objects, problems, or events to which the mathematical knowledge in a
benchmark refers. Providing students with some firsthand experiences (e.g., handson
activities, problem solving, or measuring) is important, provided such experiences are
practical. When such experiences are not practical (for example, providing firsthand
experiences with measuring the height of a mountain), students can encounter objects and
events indirectly, through the use of videos, pictures, models, etc.
However, it is neither necessary nor optimal that all experiences provided are
firsthand. (For example, once students have had some firsthand experience with flipping
coins to find probabilities, providing them with examples of other events that have finite
outcomes would likely be adequate.) If all experiences provided to students were
firsthand, it would limit the number of examples that could be provided (see previous
criterion, Providing Variety of Contexts). Moreover, students should not be asked to
reason only about ideas they see firsthand, when in real life they will also encounter
problems indirectly.
Indicators of meeting Criterion III.2:
1. The activities, whether firsthand or not, provide experiences (e.g., text,
pictures, video) that give students meaningful connections of the concept or skill to
their knowledge.
2. An appropriate number of experiences with ideas are firsthand experiences.
(The number of firsthand experiences that are appropriate depends
on the age level of the students and the difficulty of the benchmark.)
3. The firsthand experiences are efficient when compared to other firsthand experiences
that could be used. (Efficiency of an experience is judged by the time and cost of the
experience in relation to its value.)
Scoring Scheme:
High: The material meets all 3 indicators.
Medium: The material meets indicator 1, along with indicator 2 or 3.
Low: The material meets indicator 1.
None: The material does not meet any of the indicators.
Category IV: Developing Mathematical Ideas
Science for All Americans includes in its definition of mathematics literacy
a number of important yet quite abstract ideas—e.g., symbolic representation,
patterns and relationships, summarizing data. Such ideas cannot be readily discovered in
the real world; the ideas themselves were developed over many hundreds of years as a
result of considerable discussion and debate about the existence and logic of laws of
mathematics and proofs of theorems. Mathematics literacy requires that students see the
link between concepts and skills, see mathematics itself as logical and useful, and become
skillful at using mathematics. This category includes criteria to determine whether the
material expresses and develops ideas in ways that are accessible and intelligible to
students, and to demonstrate the usefulness of the concepts and skills in varied contexts.
IV.1 Justifying Importance of Benchmark Ideas. Does the material suggest ways
to help students develop a sense of the importance and validity of mathematical concepts
or procedures? 
Clarification:
This criterion highlights the importance of including some instances in the curriculum
where an argument is developed in support of the concepts, skills, or strategies
presented. There are both logical and psychological reasons for expecting a material to
provide students with a sense of why ideas make sense and of why mathematicians are
interested in them.
First, an understanding of the link between hypotheses and argument is itself a
literacy goal. Science for All Americans includes in its definition of science
literacy a basic knowledge of the nature of mathematics—both its logic and
creativity—and its central role in human endeavor. Learning about the nature of
mathematics can be studied in the context of learning about nearly any discipline,
including students’ own discoveries.
Given the time it takes to properly develop an argument for ideas and the increased
level of sophistication required for understanding both the evidence and the arguments,
there is a limited number of ideas for which an evidencebased argument is required for
literacy. However, it is possible that some concepts or procedures themselves are
sufficiently difficult for students to understand—e.g., long division, infinite
decimals—that a case needs to be made for students to find them plausible. The case
might involve examining whether the concept fits well with other concepts, explains
several relationships, and predicts new observations, and how it compares to other
explanations of the same observations. If such a case is likely to be too difficult for
most students to understand, then students should at least be informed that they are being
asked to take an idea on faith. When a material does not attempt to make a case, reviewers
should comment on (a) whether or not a case ought to have been made and, if so, why and (b)
whether or not the material makes explicit that a case is not being built.
Indicators of meeting Criterion IV.1:
1. The material builds a case for the mathematical importance of the benchmark.
2. The material builds a case for the validity of the mathematical ideas.
3. The material builds a case for the benchmark that is likely to be comprehensible to
students.
4. The material engages students in considering a case for the validity and importance of
benchmark concepts or skills.
Scoring Scheme:
High: The material meets all 4 indicators.
Medium: The material meets indicator 1 or 2, along with indicator 3 or 4.
Low: The material meets indicator 1 or 2.
None: The material does not meet any of the indicators.
IV.2 Introducing Terms and Procedures. Does the material introduce terms and
procedures only in conjunction with experience with them and only as needed to facilitate
thinking and promote effective communication? 
Clarification:
Understanding, rather than simply memorizing vocabulary or algorithms, should be the
main purpose of mathematics teaching. In mathematics, many terms refer to concepts. For
students to understand a concept, they should be able to describe its properties, give
examples and nonexamples of it, and eventually give a definition. Algorithms are
important in themselves, as well as providing efficient ways to solve problems. Students
should have opportunities to apply the concepts or procedures in problems and reasoning.
Indicators of meeting Criterion IV.2:
1. The material limits the use of terms and procedures.
2. The material introduces mathematical vocabulary or procedures in conjunction with
experiences, rather than having students simply memorize definitions or procedures.
3. The material provides appropriate examples or meaningful applications of the terms or
procedures.
Scoring Scheme:
High: The material meets all 3 indicators.
Medium: The material meets indicator 1, along with indicator 2 or 3.
Low: The material meets indicator 1.
None: The material does not meet any of the indicators.
IV.3 Representing Ideas Accurately. Does the material include accurate and
comprehensible representations of mathematical concepts, procedures, and relationships? 
Clarification:
This question highlights the importance of using accurate representations to make
(abstract) ideas intelligible to all students. Different representations highlight
different aspects of an idea and provide a variety of opportunities for the idea to
connect to other students’ ideas and become embedded in a student’s knowledge
system. Possible modes of representation include drawings, diagrams, graphs, images,
analogies and metaphors, models and simulations, and roleplaying. Representations need to
be clear so that students can understand fairly quickly what ideas are being represented
and how. In addition, because representations typically highlight only some aspects of an
idea, care must be taken that they represent the real thing as accurately as possible (or
they involve students in considering which aspects of the real thing are represented by
the model and which are not).
Indicators of meeting Criterion IV.3:
1. The material includes accurate representations.
2. The material includes comprehensible representations (depending on the students’
grade level and the difficulty of the benchmark).
3. The material includes an appropriate number and variety of accurate and comprehensible
representations.
Scoring Scheme:
High: The material meets all 3 indicators.
Medium: The material meets indicators 1 and 2.
Low: The material meets indicator 1.
None: The material does not meet any of the indicators.
IV.4 Connecting Benchmark Ideas. Does the material explicitly draw attention to
appropriate connections among benchmark ideas? 
Clarification:
This criterion emphasizes connections among benchmark ideas, concepts,
and skills (rather than connections between activities, which are
examined in Category I). Some kinds of connections can be classified
as belonging to one of the following general types:
(a) One concept or skill may be an instance of a generalization (e.g., the sum of the
angles of a rectangle illustrates the general sum of the angles of a polygon).
(b) One concept or skill might be analogous to another idea (e.g., adding rational
fractions is like adding rational expressions in that they both require a common
denominator).
(c) A concept or skill may show up in several fields or contexts (e.g., the number pi
can be the ratio of the circumference to diameter, the radian measure of a semicircle, or
the sum of a series).
Other connections are more unique to particular ideas (e.g., linking slope of a line to
the tangent of a curve).
Responding to this criterion involves looking to see whether any general
or unique connections are essential, which requires identifying what
such connections might be. Growth of Understanding maps provide a
rich source of potential connections. A set of maps on about 50 topics
will be available in Atlas of Science Literacy. When a map
is not available for a topic, the "AlsoSee" box in Benchmarks
for Science Literacy may be helpful in identifying conceptual
connections among ideas. Reading Science for All Americans
may also be helpful in identifying conceptual connections, since several
important connections among ideas are made in the prose.
Indicators of meeting Criterion IV.4:
1. The material notes connections among specific benchmark ideas (rather than just
among general topics).
2. The material adequately explains or develops the identified connections.
3. The material engages students in making and/or explaining the identified connections.
Scoring Scheme:
High: The material meets all 3 indicators.
Medium: The material meets indicators 1 and 2.
Low: The material meets 1 of the 3 indicators.
None: The material does not meet any of the indicators.
IV.5 Demonstrating/Modeling Procedures. Does the material demonstrate/model (or
include suggestions for teachers on how to demonstrate/model) skills or the use of
knowledge? 
Clarification:
Among the ways literate adults use their knowledge and skills are to describe and
explain phenomena, to solve practical problems, and to consider alternative positions on
issues. Hence students should learn to use their knowledge and skills in these ways. In
order for students to know the type and level of performance expected for a skill or an
application of conceptual knowledge, they need to see examples. This is particularly
important for complex behaviors such as explaining how to solve problems, developing a
generalization, argument, or proof, or carrying out a complex procedure or algorithm.
Demonstrating or Modeling a skill involves (a) an expert's demonstrating or modeling the
skill, (b) providing running commentary about important aspects to note about the
performance or demonstration, and (c) providing criteria for judging a good performance.
Demonstrating or Modeling how knowledge might be used, for example, to solve problems
or construct a proof, is similar.
Responding to this criterion involves examining whether (a) demonstrating/modeling is
carried out by the text or other accompanying materials (e.g., software, video), or (b) the
material includes suggestions to teachers about how to go about demonstrating/modeling
skills or use of knowledge in their classrooms.
Indicators of meeting Criterion IV.5:
1. The material demonstrates (or instructs teachers how to demonstrate) the expected
procedure or performance.
2. The demonstration is clear and comprehensible.
3. The material provides commentary that points to particular aspects of the demonstration
and/or provides justifications or explanations for the steps taken.
Scoring Scheme:
High: The material meets all 3 indicators.
Medium: The material meets indicators 1 and 2.
Low: The material meets indicator 1.
None: The material does not meet any of the indicators.
IV.6 Providing Practice. Does the material provide tasks or questions for
students to practice skills or use knowledge in a variety of situations? 
Clarification:
An important part of learning mathematics consists of giving students many
opportunities to use their skill or knowledge, in particular giving them opportunities to
practice using mathematical knowledge and skills in describing objects, relationships, and
events, solving problems, and applying knowledge in new situations or contexts. Moreover,
literacy means that people will be able to draw upon and use their understanding of
mathematics when they encounter events that do not come with labels such as
"algebra," "geometry, " or "statistics" but in political
arguments, discussions of literature, and walks on the beach. Therefore, students will
need practice in making connections to new situations. Providing students with
opportunities to practice only calculating answers to predictable exercises does not
adequately address this criterion.
Indicators of meeting Criterion IV.6:
1. The material includes appropriate practice exercises and tasks on the benchmark.
2. The material provides an appropriate number of practice exercises and tasks.
3. The material includes a variety of contexts, including everyday tasks and contexts and
novel as well as familiar practice tasks.
Scoring Scheme:
High: The material meets all 3 indicators.
Medium: The material meets indicator 1, along with either 2 or 3.
Low: The material meets indicator 1.
None: The material does not meet any of the indicators.
Category V: Promoting Student Thinking about Mathematics
No matter how clearly materials may present ideas, students (like all people) will
devise their own meaning, which may or may not correspond to targeted learning goals.
Students need to make their ideas and reasoning explicit, hold them up to scrutiny, and
recast them as needed. This category includes criteria for whether the material suggests
how to help students express, think about, and reshape their ideas to make better sense of
the world.
V.1 Encouraging Students to Explain Their Reasoning. Does the material
routinely include suggestions for having each student express, clarify, justify, and
represent his/her ideas and how to get feedback from peers and the teacher? 
Clarification:
It is important to provide opportunities for students’ thinking to become overt
to themselves, the teacher, and other students. By stating (clarifying, justifying, and
representing) their ideas, in writing, drawing, or speaking, students become more aware of
what they think. This may stimulate making explicit connections between their ideas and
the ideas presented by the text or the teacher, and/or questioning of their ideas (if
relevant). Exchange of ideas in small groups or a large group discussion may make students
aware of the range of ideas that exist and may provoke students to reconsider their own
ideas in light of others. Feedback from the teacher or other peers is necessary to help
students understand their mistakes and improve the quality of their descriptions,
explanations, or designs.
Responding to the first part of the criterion involves examining whether the material
prompts (or encourages teachers to prompt) students to express their ideas either orally
or in writing. It also involves examining whether the material has opportunities for each
student to express his or her ideas. Responding to the second part of the criterion
includes examining whether the material includes specific suggestions to help the teacher
provide explicit feedback, includes text that directly provides students with feedback on
the adequacy of their ideas, or provides teachers with strategies they can use to ensure
that each student in the class receives feedback.
Indicators of meeting Criterion.V.1:
1. The material encourages students to express their ideas about the benchmark.
2. The material encourages students not only to express but also to clarify, justify,
interpret, or represent benchmark ideas.
3. The material provides (or includes suggestions to help the teacher to provide) explicit
feedback to students about their ideas.
4. The material includes suggestions to the students or teacher on how to use student
responses to diagnose errors or difficulties, address errors or difficulties, or further
develop students’ ideas about the benchmark.
Scoring Scheme:
High: The material meets all 4 indicators.
Medium: The material meets indicator 1, along with 2 of the other 3 indicators.
Low: The material meets at least 1 of the indicators.
None: The material does not meet any of the indicators.
V.2 Guiding Interpretation and Reasoning. Does the material include tasks
and/or question sequences that guide student interpretation and reasoning about benchmark
concepts, skills, and relationships? 
Clarification:
Experiences with handson materials, problems, and examples of mathematical ideas are
useful but not sufficient. Students need time, opportunities, and guidance to make sense
of these experiences. If the students are turned loose to do exercises or problems on
their own, very little happens except for a small number of students. The activities need
to be guided with sequences of questions that lead students to make relevant
generalizations and understand relationships. Similarly, students need time,
opportunities, and guidance to make sense of things they read and ideas they are
introduced to.
Responding to this criterion involves examining whether the material includes (in the
teacher’s guide or student books) specific, carefully chosen and sequenced tasks or
questions that are likely to support students’ thinking about exercises, problems,
and investigations. Good tasks and questions frame important issues, help students relate
their previous experiences to the mathematical ideas, anticipate common student
difficulties or misconceptions, and focus on important generalizations and procedures.
Indicators of meeting Criterion V.2:
1. The material includes specific and relevant tasks and/or questions for activities
related to the benchmark.
2. The material includes connected sequences (rather than only collections) of questions
or tasks.
3. The questions or tasks guide student interpretation and reasoning through approaches
such as:
(a) framing, introducing, or developing important ideas,
(b) helping students to relate their own experiences to mathematical ideas, or
(c) anticipating or eliciting common difficulties or student misconceptions.
Scoring Scheme:
High: The material meets all 3 indicators.
Medium: The material meets 2 of the 3 indicators.
Low: The material meets 1 of the 3 indicators.
None: The material does not meet any of the indicators.
V.3 Encouraging Students to Think about What They’ve Learned. Does the
material suggest ways to have students check their own progress? 
Clarification:
This criterion highlights the importance of having students look back at the progress
of their thinking and learning. Monitoring one’s understanding and realizing which
ideas one does not understand can shift some of the responsibility for learning to the
students and may elicit their attempts to understand as a result.
Responding to this criterion involves examining whether the material includes questions
or tasks that prompt students to monitor their understanding, or includes suggestions to
teachers on how to encourage students to do so. For example, What was confusing to you
today? How does the new knowledge compare with what you used to think? What do you think
you understand and where do you need to work more? Encouraging students to monitor their
understanding should also include (when appropriate) questions on how and why students
changed their ideas. For example, Did you change any of your ideas today? What evidence
convinced you to do so?
Indicators of meeting Criterion V.3:
1. The material engages (or provides specific suggestions for teachers to engage)
students in monitoring their progress toward the benchmark.
2. The material asks students to think about how their ideas have developed or changed.
3. The material gives students an opportunity to revise their initial ideas about the
benchmark based on what they have learned.
Scoring Scheme:
High: The material meets all 3 indicators.
Medium: The material meets 2 of the 3 indicators.
Low: The material meets 1 of the 3 indicators.
None: The material does not meet any of the indicators.
Category VI: Assessing Student Progress in Mathematics
Assessment provides information to students about what is important and to teachers
about what has been learned. Just as important, assessment provides information to both
the students and the teacher about adjustments that should be made in learning and
instruction. Because assessment is so important to the teachinglearning process, it must
match the mathematics learning goal of the curriculum materials. Further, assessments must
address the range of skills, applications, and contexts that reflect what students are
expected to learn. All of this is possible only if assessment takes place throughout
instruction, not only at the end of a chapter or unit.
VI.1 Aligning Assessment. Are assessment items or tasks included that match the
ideas, concepts, or skills of the benchmark? 
Clarification:
This criterion highlights the necessity of including assessment items for each
benchmark that is important in the material. To judge whether the items provided match the
standard, the same procedure should be followed as in examining the content match between
an activity and a benchmark. That is, examine whether the assessment item addresses the
substance rather than only the topic of the benchmark, the level of sophistication of the
benchmark, and what part of the benchmark is assessed.
Indicators of meeting Criterion VI.1:
1. The material provides at least one assessment task that addresses the specific
ideas of the benchmark (assessment item should not be answerable by reading comprehension,
general intelligence, or testwiseness alone).
2. The assessment items that do address the benchmark require no other, more
sophisticated, ideas.
3. If the material provides a test that is given to the students, an appropriate number of
assessment items are contentmatched to the benchmark. If the material provides a bank of
assessment items that teachers select from, an appropriate proportion of assessment items
are contentmatched to the benchmark.
Scoring Scheme:
High: The material meets all 3 indicators.
Medium: The material meets indicators 1 and 2.
Low: The material meets indicator 1.
None: The material does not meet any of the indicators.
VI.2 Assessment through Applications. Does the material include assessment
tasks that require application of benchmark ideas, concepts, or skills and avoid allowing
students a trivial way out, like using a formula or repeating a memorized term or rule
without understanding? 
Clarification:
Rather than checking whether students have memorized certain bits of information,
assessment needs to test students’ mathematical understanding, reasoning, and the
application of knowledge. In addition, it needs to include tasks that engage students in
activities similar to those they will engage in in their lives outside the classroom.
Literate persons use mathematical knowledge to describe, explain, and predict real world
phenomena, solve a practical problem, or discuss issues. Accordingly, assessment tasks
need to engage students in descriptions, explanations, predictions, design, and discussion
of issues. This, however, does not necessarily dictate the format that the assessment
should include. For example, assessments of students’ use of knowledge to explain a
concept could include a "multiple choice" or "constructed response"
format.
Indicators of meeting Criterion VI.2:
1. The material provides assessment tasks that focus on application of benchmark
ideas.
2. The material includes assessment tasks that are familiar as well as tasks that are
novel or nonroutine.
3. If the material provides a test that is given to the students, an appropriate number of
assessment items focus on application. If the material provides a bank of assessment items
that teachers select from, an appropriate proportion of items or tasks focus on
application.
Scoring Scheme:
High: The material meets all 3 indicators.
Medium: The material meets 2 of the 3 indicators.
Low: The material meets 1 of the 3 indicators.
None: The material does not meet any of the indicators.
VI.3 Using Embedded Assessment. Are some assessments embedded in the curriculum
along the way, with advice to teachers as to how they might use the results to choose or
modify activities? 
Clarification:
This criterion highlights the need for assessment to be in the service of instruction
to guide teaching and learning. The criterion requires that materials include assessments
that can be used as diagnostic or formative instruments, which help determine
learners’ needs, rather than largely as instruments for grading students at the end
of a unit or chapter.
Responding to this question involves examining whether the material (a) provides
assessment tasks only at the end of a unit of study to help grade student achievement, or
also along the way to help monitor student progress, (b) encourages and provides guidance
to teachers about how to probe beyond students’ first response to clarify and further
understand student answers, and (c) encourages teachers to use the information from the
assessments to make instructional decisions about what ideas need to be addressed by
further activities with the whole group or smaller groups of students.
Indicators of meeting Criterion VI.3:
1. The material uses embedded assessment as a part of the instructional strategy or
design.
2. The material includes assessments that provide opportunities, encouragement, or
guidance for students on how to further understand benchmark ideas.
3. The material includes suggestions or guidance for teachers on how to probe
students’ understanding of benchmark ideas.
4. The material provides specific suggestions to teachers about how to use the information
from the embedded assessments to make instructional decisions about what ideas need to be
addressed by further activities.
Scoring Scheme:
High: The material meets all 4 indicators.
Medium: The material meets indicator 1, along with 2 of the other 3 indicators.
Low: The material meets indicator 1.
None: The material does not meet any of the indicators.
Category VII. Enhancing the Mathematics Learning Environment
Several other important considerations are involved in the selection of curriculum
materials—for example, the help they provide teachers in encouraging student
curiosity and creating a classroom community where all can succeed, or attractiveness.
These can influence student learning or even whether the materials are used appropriately
by the teacher and students. The criteria listed in this category provide reviewers with
the opportunity to comment on these and other important features.
VII.1 Providing Teacher Content Support. Does the material help teachers
improve their understanding of mathematics and its applications? 
Clarification:
The issue here is whether the material includes a "content background"
section or other features that help teachers develop their understanding of the
mathematical knowledge addressed in the material. Responding to this question involves
commenting on the quality of the support, not merely on whether such support is included.
Just providing teachers with a list of resources that may enhance their understanding of
mathematics does not adequately address this question. A minimum requirement is that such
lists are annotated to describe the resources and specify what can be learned from them.
Indicators of meeting Criterion VII.1:
1. The material provides content information or recommends resources for improving
specific skills or understanding of particular ideas.
2. The material provides content information that is in the form that is useful and
appropriate for teachers, no matter what their background knowledge.
3. The material indicates how the ideas or skills are relevant and important to teaching
the material to students.
Scoring Scheme:
High: The material meets all 3 indicators.
Medium: The material meets 2 of the 3 indicators.
Low: The material meets 1 of the 3 indicators.
None: The material does not meet any of the indicators.
VII.2 Establishing a Challenging Classroom. Does the material help teachers to
create a classroom environment that welcomes student curiosity, rewards creativity,
encourages a spirit of healthy questioning, and avoids rigidity? 
Clarification:
Responding to this criterion involves examining whether teachers are given guidance to
(a) encourage students to raise questions about the material being studied and suggest
productive ways for finding answers, (b) use activities in which students’ creativity
and imagination will pay off, (c) respect and value students’ ideas, and (d) avoid
conveying the impression that they themselves or the textbooks are absolute authorities
whose conclusions are always correct. In addition, the criterion involves examining
whether the materials give a vision of what the curriculum might look like in action
(i.e., teacher hints and suggestions, dialogue boxes, vignettes, or video clips that show
desirable student teacher interactions).
Indicators of meeting Criterion VII.2:
1. The material provides opportunities for students to express their curiosity or
creativity.
2. The material provides occasions for students to take risks and ask questions.
3. The material suggests how to encourage students to weigh and challenge their own and
others’ ideas.
4. The material avoids sending a message that mathematics consists only of rules and
single correct answers.
Scoring Scheme:
High: The material meets all 4 indicators.
Medium: The material meets 2 of the 4 indicators.
Low: The material meets 1 of the 4 indicators.
None: The material does not meet any of the indicators.
VII.3 Supporting All Students. Does the material help teachers to create a
classroom that encourages high expectations for all students, enables all students to
experience success, and provides all students a feeling of belonging in the mathematics
classroom? 
Clarification:
Several pedagogical criteria presented in previous categories highlight the need for
materials to incorporate principles of teaching and learning that are likely to promote
mathematics understanding for all students. This question highlights the importance of
reviewing curriculum materials for features that might distract or impede the progress of
females, minorities, students whose first language is not English, students with
disabilities, or others from the intended work. Further, the criterion requires that
materials provide specific suggestions and resources for encouraging all students to be
able to learn mathematics and express their competence and performances during instruction
and in assessment tasks.
Indicators of meeting Criterion VII.3:
1. The material avoids stereotypes or language that might be offensive to a particular
group.
2. The material illustrates the contribution or participation of women, minorities, and
persons with disabilities to mathematicsrelated fields.
3. The material suggests alternative formats for students to develop or express their
mathematics knowledge during instruction and assessment.
4. The material includes specific suggestions on how teachers can modify activities for
students with special needs, interests, or abilities.
Scoring Scheme:
High: The material meets all 4 indicators.
Medium: The material meets indicator 1, along with 2 of the other 3 indicators.
Low: The material meets indicator 1.
None: The material does not meet any of the indicators. 