Appendix B

A Research Base for the Instructional Criteria in Project 2061’s Mathematics Curriculum Materials Analysis Procedure

Project 2061’s instructional criteria represent a set of features that are characteristic of good instructional design. They require analysts to consider how effective a material is in: stating the purpose for learning the content; taking account of what students already know; engaging students in meaningful learning experiences that provide for the development and understanding of, and practice with mathematics ideas; providing opportunities for reflection and assessment of learning; and supplying the important aspects of a quality learning environment that provides for the success of all students.

These criteria were derived from research on learning and teaching and from the craft knowledge of experienced educators. The primary sources for the research background include: Chapter 13, "Effective Learning and Teaching," of Science for All Americans (American Association for the Advancement of Science [AAAS], 1989); Chapter 15, "The Research Base," of Benchmarks for Science Literacy (AAAS, 1993); Research Ideas for the Classroom: Middle Grades Mathematics (Owens, 1993); and Handbook of Research on Mathematics Teaching and Learning (Grouws, 1992).

Overall, the research tells us that effective instruction provides students with a carefully developed set of experiences. For example, one summary of research concluded that the five components used by successful teachers to help students develop mathematical ideas are: attending to prerequisites, developing relationships, employing representations, attending to student perceptions, and emphasizing the generality of mathematics concepts (Good, Grouws, & Ebmeier, 1983). In addition, Peterson (1988) found that "students learn efficiently when their teachers first structure new information for them and help them relate it to what they already know, then monitor their performance and provide corrective feedback recitation, drill, practice, or applications activity." The following summary of the research supporting Project 2061’s analysis of mathematics textbooks is intended to provide key examples of relevant findings, rather than a comprehensive review of the literature. (It is organized according to the seven categories of the Project 2061 criteria.)


Category I. Identifying a Sense of Purpose: This category includes criteria to determine whether the material attempts to make its purposes explicit and meaningful to students, either by itself or by instructions to the teacher.

The positive effects of providing students with a clear idea of the purpose, goals, and content of the lesson ahead of time is most closely associated with the work of Ausubel (1968). In addition to stating the purpose at the beginning, effective lessons include summaries that review the mathematical concept being studied (Madsen-Nason, 1988), helping students to close the loop to formalize what they have learned. The sequence of lessons is also important in accomplishing the stated purpose, since mathematical ideas often build on each other. For example, Mack (1990) determined that prior knowledge can influence the sequence of instruction. Therefore careful design of lesson flow is important.

Requiring that such actions be clear and meaningful to students is a common sense reminder that students themselves need to understand the intended purposes, and that how those purposes are communicated is important.


Category II. Building on Student Ideas about Mathematics: Fostering better understanding in students requires taking time to attend to the ideas they already have, both ideas that are incorrect and ideas that can serve as a foundation for subsequent learning. This category of criteria examines whether the material contains specific suggestions for identifying and relating to student ideas.

The importance of taking account of students’ ideas is captured in the statement by Ausubel (1968) that "the most important single factor influencing learning is what the learner already knows." There are many implications of this finding in mathematics teaching and learning that demand attention in curriculum materials. If students have narrow conceptions and representations of ideas or procedures that do not extend to other situations, their subsequent work can result in misconceptions (Fischbein, Deri, Nello, & Marino, 1985; Bell, Greer, Grimison, & Mangan, 1989). For example, students’ intuitions about number operations need to be revised when they move to expanded number systems (Graeber & Campbell, 1993). Students may decide, for instance, that when multiplying, the result is always larger than either of the two original numbers – a generalization that can lead to trouble when they move to working with numbers less than 1. Hart (1988) and Matz (1980) also found that prior knowledge from arithmetic leads to misconceptions when generalized to more advanced topics.

Helping teachers to understand students’ knowledge and thinking leads to using improved instructional strategies (Carpenter, Franke, Jacobs, Fennema, & Empson, 1998; Cobb et al., 1991). A number of strategies have been found that are effective in identifying and addressing prior knowledge. For example, a discussion of how students perceive the difference between two solutions to an exercise or problem can provide insights into student understanding (Cobb, 1988). Also, an assessment of how students extend procedures to other contexts and situations can determine misconceptions or lack of understanding (Hiebert & Wearne, 1986). Both of these strategies apply to identifying and addressing a wide range of mathematical ideas and procedures.

Instructional materials should provide opportunities for students to make connections between and among mathematical ideas and skills. Resnick (1987) concluded that without explicit assistance in connecting ideas, people do not usually learn concepts simply by building up pieces of knowledge. Unless materials attend to students’ prior knowledge and teachers are alerted to it, the sequence of activities might be inappropriate (Mack, 1990). Moreover, further misconceptions may develop or achievement will be diminished for many students who are unable to develop more sophisticated ideas, partially due to persistent errors (Brown & VanLehn, 1982; Matz, 1980).


Category III. Engaging Students in Mathematics: Mathematics involves finding patterns and modeling ideas and relationships. This category determines whether the material provides students with firsthand experiences with mathematics in a variety of contexts.

The simple use of hands-on materials is not sufficient to promote learning and understanding. However, an early review of research showed that when concrete materials were used appropriately (which took place in about half the studies), student achievement was better than when more abstract approaches were used (Suydam & Higgins, 1977). The key is the appropriate use of concrete materials, which includes a sufficient number of firsthand experiences that are provided in a variety of contexts. Research has shown that learning activities involving problem situations that vary along critical dimensions do promote connections and understanding (Case & Sandieson, 1988; Vergnaud, 1988).

Appropriate use of firsthand experiences also requires that the situations be embedded in problem situations that have meaning for students (Brown, Collins, & Duguid, 1989). Taking this somewhat further, recent research indicates that mathematical learning is enhanced when instruction emphasizes student engagement with tasks that are both meaningful and challenging (Stein & Lane, 1996). Tasks that are challenging are important in producing the kind of higher order thinking necessary for connections and understanding to take place.


Category IV. Developing Mathematical Ideas: This category includes criteria to determine whether the material expresses and develops ideas in ways that are accessible and intelligible to students, and provides demonstration and practice with concepts and skills in varied contexts.

The introduction of mathematical terms, symbols, and procedures is critical to developing skill and understanding. Students at all grade levels need to work consistently at developing their understanding of the ideas captured in conventional mathematics representations and symbols (Greeno & Hall, 1997). However, Wagner and Parker (1993) found that extensive work with symbolic manipulation before developing solid understanding results in inability to progress beyond mechanical manipulations.

Because representations of mathematical ideas are so important to conceptual development (Ball, 1988; Hiebert & Wearne, 1986), these representations should be carefully developed, and should connect with earlier informal and concrete experiences. According to Resnick (1982), making explicit the connections between concrete representations and their associated symbols helps students construct necessary relationships.

Student understanding of concepts leads to the ability to generate new connections (Mayer, 1989) and promotes remembering ideas so concepts and procedures can be applied to solving problems and in learning more advanced concepts (Bruner, 1960). In addition, strong connections between concepts enhances transfer to other contexts (Carpenter & Moser, 1984; Kieren, 1988).

For learning to become formalized and ready to use, appropriate and meaningful practice in a variety of contexts and applications is necessary (Peterson, 1988). However, Hiebert et al. (1997) found that excessive practice before attaining understanding can lead to difficulty in making sense of the procedures later.


Category V. Promoting Student Thinking About Mathematics: This category includes criteria for whether the textbook suggests how to help students express, think about, and reshape their ideas to make better sense of mathematics and the world.

Difficulties in mathematical problem solving are often caused by students’ ineffective use of what they already know (Schoenfeld, 1992). However, classroom discourse can exploit the use of language as a powerful tool for orienting and focusing attention and is crucial for constructing relationships (Greeno, 1988; Resnick & Omanson, 1987). Specifically, students who are expected to engage in communication about mathematics will have improved conceptions of the nature of mathematics (Lampert, 1989).

Work in pairs and small groups can be an effective tool for promoting student communication. For example, both Slavin (1989) and Webb (1989) found that work in small groups can enhance achievement through student interaction if the work is focused carefully on learning mathematical ideas. In addition, guidance of student interpretation and reasoning through classroom discourse and work in small groups can help students construct and formalize their ideas so they are more accessible.

Students need the opportunity for self-discovery, and activities that are unstructured enough to allow them to derive generalizations and invent their own procedures (Doyle, 1983). Questions in the lesson summary can also help students reflect on the mathematical concepts and help them establish linkages between mathematical topics (Madsen-Nason, 1988).


Category VI. Assessing Student Progress in Mathematics: This category includes criteria for evaluating whether the material includes a variety of aligned assessments that apply the concepts and skills taught in the material.

The alignment of mathematics learning goals with assessment items and tasks is a critical requirement that is not often accomplished well. Without this alignment, it is unlikely that assessment tasks can be used effectively to monitor learning or make instructional decisions.

Research shows that elementary mathematics teachers tend to use more diverse assessment techniques while secondary teachers emphasize tests (Gullickson, 1985) and rely heavily on published paper and pencil tests (Stiggins & Bridgeford, 1986). At both levels, assessment that is integrated or embedded with instruction is important for estimating the effectiveness of lessons and making decisions about individual and group progress. In addition, multiple types of assessments that are integrated with learning mathematics can lead to teachers’ understanding of student thinking and strategies that promote higher order thinking and problem solving (Kulm, 1994).

The convergence of assessment evidence from different sources is necessary in order to make instructional decisions (Stiggins, 1997), so a variety of applications and contexts as well as multiple types of assessment strategies should be used. For example, Zehavi, Bruckheimer, and Ben-Zvi (1988) found that the use of projects in and out of class for assessment enhances student mathematical achievement.

Category VII. Enhancing the Mathematics Learning Environment: The criteria listed in this category provide analysts with the opportunity to comment on features that enhance the use and implementation of the textbook by all students.

The keys to quality instruction are a well-prepared teacher, instructional materials that provide opportunities to learn important mathematics, and an environment with high expectations that encourages thinking. Research has revealed that the motivational climate in classrooms is enhanced through encouraging enthusiasm for learning, reducing anxiety, and inducing curiosity (Brophy, 1983), and that students perform better at problem solving in a supportive atmosphere (Thompson & Thompson, 1989).

Knowledge of mathematics content enables the teacher to provide rich and flexible mathematics instruction, but knowledge of mathematics alone is not sufficient—the knowledge must be well-connected with how students learn specific ideas. Some teachers draw the strategy and rationale for their approaches to teaching mathematics from their knowledge of the discipline of mathematics itself (Ball, 1991). Ultimately, teachers whose knowledge of mathematics is more connected and conceptual are also more conceptual in their teaching (Leinhardt, Putnam, Stein, & Baxter, 1991; Steinberg, Haymore, & Marks, 1985).

Not all students, even in the same classroom, receive the same quality of instruction. For example, the nature, frequency, and duration of mathematics teachers’ interactions with male and female students sometimes differ significantly (Leder, 1992; Meyer & Koehler, 1990). In addition, national assessments reveal a picture of racial and ethnic disparities in mathematics achievement (Dossey, Mullis, Lindquist, & Chambers, 1988) that has complex roots but can be traced to factors such as socio-economic status and language proficiency (Fernandez & Nielsen, 1986).

All children, including those who have been underserved, can and do learn mathematics when they have access to quality mathematics instruction (Campbell, 1995; Silver & Stein, 1996). Project 2061’s mathematics curriculum materials analysis procedure will help educators determine which curriculum materials will provide the quality instruction that research shows students need.


American Association for the Advancement of Science. (1989). Science for all Americans. New York: Oxford University Press.

American Association for the Advancement of Science. (1993). Benchmarks for science literacy. New York: Oxford University Press.

Ausubel, D. P. (1968). Educational psychology: A cognitive view. New York: Holt, Rinehart & Winston.

Ball, D. L. (1988). Unlearning to teach mathematics. (Issue Paper 88-1). East Lansing: Michigan State University, National Center for Research on Teacher Education.

Ball, D. L. (1991). What’s all this talk about discourse? Arithmetic Teacher, 39(3), 44-48.

Bell, A., Greer, B., Grimison, L., & Mangan, C. (1989). Children’s performance on multiplicative word problems: Elements of a descriptive theory. Journal for Research in Mathematics Education, 20(4), 434-449.

Brophy, J. (1983). Conceptualizing student motivation. Educational Psychologist, 18, 200-215.

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Education Research, 18(1), 32-42.

Brown, J. S., & VanLehn, K. (1982). Towards a generative theory of "bugs." In T. P. Carpenter, J. M. Moser, & T. A. Romberg (Eds.), Addition and subtraction: A cognitive perspective (pp. 117-135). Hillsdale, NJ: Erlbaum.

Bruner, J. S. (1960). The process of education. New York: Vintage Books.

Campbell, P. (1995). Project IMPACT: Increasing mathematics power for all children and teachers. (Phase I, Final Report). College Park, MD: Center for Mathematics Education, University of Maryland.

Carpenter, T. P., Franke, M. L., Jacobs, V. R., Fennema, E., & Empson, S. B. (1998). A longitudinal study of invention and understanding in children’s multidigit addition and subtraction. Journal for Research in Mathematics Education, 29(1), 3-20.

Carpenter, T. P., & Moser, J. M. (1984). The acquisition of addition and subtraction concepts in grades one through three. Journal for Research in Mathematics Education, 15, 179-202.

Case, R., & Sandieson, R. (1988). A developmental approach to the identification and teaching of central conceptual structures in mathematics and science in the middle grades. In J. Hiebert & M. Behr (Eds.), Number concepts and operations in the middle grades (pp. 236-259). Reston, VA: National Council of Teachers of Mathematics.

Cobb, P. (1988). The tension between theories of learning and instruction in mathematics education. Educational Psychologist, 23, 87-103.

Cobb, P., Wood, T., Yackel, E., Nicholls, J., Wheatley, G., Trigatti, B., & Perlwitz, M. (1991). Assessment of a problem-centered second-grade mathematics project. Journal for Research in Mathematics Education, 22(1), 3-29.

Dossey, J. A., Mullis, I. V., Lindquist, M. M., & Chambers, D. L. (1988). The mathematics report card: Are we measuring up? Trends and achievement based on the 1986 national assessment. Princeton, NJ: Educational Testing Service.

Doyle, W. (1983). Academic work. Review of Educational Research, 53, 159-199.

Fernandez, R. M., & Nielsen, F. (1986). Bilingual and Hispanic scholastic achievement: Some baseline results. Social Science Research, 15, 43-70.

Fischbein, E., Deri, M., Nello, M. S., & Marino, M. S. (1985). The role of implicit models in solving verbal problems in multiplication and division. Journal for Research in Mathematics Education, 6(1), 3-17.

Good, T. L., Grouws, D. A., & Ebmeier, H. (1983). Active mathematics teaching. New York: Longman.

Graeber, A., & Campbell, P. (1993). Misconceptions about multiplication and division. Arithmetic Teacher, 40(7), 408-411.

Greeno, J. G. (1988). The situated activities of learning and knowing mathematics. In M. J. Behr, C. G. Lacampagne, & M. M. Wheeler (Eds.), Proceedings of the Tenth Annual Meeting of PME-NA (pp. 481-521). DeKalb, IL: Northern Illinois University.

Greeno, J. G., & Hall, R. P. (1997, January). Practicing representation: Learning with and about representational forms. Phi Delta Kappan, 361-367.

Grouws, D. A. (Ed.). (1992). Handbook of research on mathematics teaching and learning. New York: Macmillan.

Gullickson, A. R. (1985). Student evaluation techniques and their relationship to grade and curriculum. Journal of Educational Research, 79, 96-100.

Hart, K. (1988). Ratio and proportion. In J. Hiebert & M. Behr (Eds.), Number concepts and operations in the middle grades (pp. 198-219). Reston, VA: National Council of Teachers of Mathematics.

Hiebert, J., Carpenter, T. P., Fennema, E., Fuson, K. C., Wearne, D., Murray, H., Olivier, A., & Human, P. (1997). Making sense: Teaching and learning mathematics with understanding. Portsmouth, NH: Heinemann.

Hiebert, J., & Wearne, D. (1986). Procedures over concepts: The acquisition of decimal number knowledge. In J. Hiebert (Ed.), Conceptual and procedural knowledge: The case of mathematics (pp. 199-223). Hillsdale, NJ: Erlbaum.

Kieren, T. E. (1988). Personal knowledge of rational number: Its intuitive and formal development. In J. Hiebert & M. Behr (Eds.), Number concepts and operations in the middle grades (pp. 162-181). Reston, VA: National Council of Teachers of Mathematics.

Kulm, G. (1994). Mathematics assessment: What works in the classroom. San Francisco: Jossey-Bass.

Lampert, M. (1989). Choosing and using mathematical tools in classroom discourse. In J.E. Brophy (Ed.), Advances in research on teaching (Vol. 1, pp. 223-264). Greenwich, CT: JAI Press.

Leder, G. C. (1992). Mathematics and gender: Changing perspectives. In D.A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 597-622). New York: Macmillan.

Leinhardt, G., Putnam, R. R., Stein, M. K., & Baxter, J. (1991). Where subject knowledge matters. In J. E. Brophy (Ed.), Advances in research on teaching: Teachers’ subject matter knowledge and classroom instruction (Vol. 2, pp. 87-113). Greenwich, CT: JAI Press.

Mack, N. K. (1990). Learning fractions with understanding: Building on informal knowledge. Journal for Research in Mathematics Education, 21(1), 16-32.

Madsen-Nason, A. (1988). A teacher’s changing thoughts and practices in ninth grade general mathematics classes: A case study. Dissertation Abstracts International 50, 666-A.

Matz, M. (1980). Towards a computational theory of algebraic competence. Journal of Mathematical Behavior, 3(1), 93-166.

Mayer, R. E. (1989). Models for understanding. Review of Educational Research, 59(1), 43-64.

Meyer, M. R., & Koehler, M. S. (1990). Internal influences on gender differences in mathematics. In E. Fennema & G. C. Leder (Eds.), Mathematics and gender (pp. 60-95). New York: Teachers’ College Press.

Owens, D. T. (Ed.). (1993). Research ideas for the classroom: Middle grades mathematics. New York: Macmillan

Peterson, P. L. (1988). Teaching for higher-order thinking in mathematics: The challenge for the next decade. In D. A. Grouws, T. J. Cooney, & D. Jones (Eds.), Perspectives on research for effective mathematics teaching (pp. 2-26). Hillsdale, NJ: Erlbaum.

Resnick, L. B. (1982). Syntax and semantics in learning to subtract. In T. P. Carpenter, J. M. Moser, & T. A. Romberg (Eds.), Add and subtraction: A cognitive perspective (pp. 136-155). Hillsdale, NJ: Erlbaum.

Resnick, L. B. (1987). Education and learning to think. Washington, DC: National Academy Press.

Resnick, L. B., & Omanson, S. F. (1987). Learning to understand arithmetic. In R. Glaser (Ed.), Advances in instructional psychology (Vol. 3, pp. 41-95). Hillsdale, NJ: Erlbaum.

Schoenfeld, A. H. (1992). On paradigms and methods: What do you do when the ones you know don’t do what you want them to? Issues in the analysis of data in the form of videotapes. Journal of the Learning Sciences, 2(2), 179-214.

Silver, E. A., & Stein, M. K. (1996). The QUASAR project: The revolution of the possible in mathematics instructional reform in urban middle schools. Urban Education, 30, 486-521.

Slavin, R. E. (1989). Cooperative learning and student achievement. In R.E. Slavin (Ed.), School and classroom organization (pp. 129-156). Hillsdale, NJ: Erlbaum.

Stein, M. K., & Lane, S. (1996). Instructional tasks and the development of student capacity to think and reason: An analysis of the relationship between teaching and learning in a reform mathematics project. Educational Research and Evaluation, 2(1), 50-80.

Steinberg, T., Haymore, J., & Marks, R. (1985). Teachers’ knowledge and structuring content in mathematics. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL.

Stiggens, R. J. (1997). Student-centered classroom assessment. Englewood Cliffs, NJ: Prentice Hall.

Stiggins, R. J., & Bridgeford, N. J. (1986). Classroom assessment: A key to effective education. Educational Measurement: Issues and Practice, 5(2), 5-17.

Suydam, M. N., & Higgins, J. L. (1977). Activity-based learning in elementary school mathematics: Recommendations from research. Columbus, OH: ERIC/SMEAC.

Thompson, A. G., & Thompson, P. W. (1989). Affect and problem solving in an elementary school mathematics classroom. In D. B. McLeod & V. M. Adams (Eds.), Affect and mathematical problem solving: A new perspective (pp. 162-176). New York: Springer-Verlag.

Vergnaud, G. (1988). Multiplicative structures. In J. Hiebert & M. Mehr (Eds.), Number concepts and operations in the middle grades (pp. 141-161). Reston, VA: National Council of Teachers of Mathematics.

Wagner, S. & Parker, S. (1993). Advancing algebra. In Research ideas for the classroom: High school mathematics (pp. 119-139). Reston, VA: National Council of Teachers of Mathematics.

Webb, N. M. (Ed.). (1989). Peer interaction, problem-solving, and cognition: Multidisciplinary perspectives [Special issue]. International Journal of Education Research, 13(1).

Zehavi, N., Bruckheimer, M., & Ben-Zvi, R. (1988). Effect of assignment projects on students’ mathematical activity. Journal for Research in Mathematics Education, 19(4), 421-438.

Back Arrow


Next Arrow