An electronic newsletter for the science education community

January/February 2005

Mapping for Curriculum Coherence
Online resources from Project 2061’s high school biology textbook evaluation

The importance of coherence to learning has been well documented; part of what characterizes expertise is having a large amount of interconnected knowledge and knowing when it can be applied to the situation at hand (Chi, Feltovich, & Glaser, 1981). As the National Research Council (NRC) reports in How People Learn, research has shown that organizing information into a conceptual framework helps students apply what they learn in new situations and to learn related information more quickly (NRC, 2000, p. 17). Project 2061 has long emphasized this need to make connections among ideas: Science for All Americans (AAAS, 1989) narrates a coherent story of the knowledge, skills, and habits of mind that constitute science literacy and Benchmarks for Science Literacy (AAAS, 1993) unpacks that coherent story into sequential grade-level steps that lead toward science literacy.

Thus, when Project 2061 applied its curriculum-materials analysis procedure to the evaluation of widely used high school biology textbooks, we looked to see not only whether the textbooks presented content that aligned with specific key ideas in important topics, but also whether they made connections among those ideas. We then looked to see how well the instructional strategies in the materials supported students’ learning of these coherent sets of ideas.

From this experience, we have learned that the use of conceptual strand maps—with their depiction of how K–12 ideas depend on and support one another in the progression of student understanding—can bring consistency to the evaluation of curriculum coherence and holds promise for aiding the coherent design of new curriculum materials. Now, with the first phase of our report High School Biology Textbooks: A Benchmarks-Based Evaluation published online, the entire set of maps created for the topic of Matter and Energy Transformations in living systems is available, along with detailed reports about the content and instructional findings in that topic for each textbook.

Using Maps to Analyze Coherence
For the purposes of our evaluation, we judged coherence in terms of how completely a textbook aligned with the set of key ideas for a topic and whether it made explicit connections among the key ideas and between them and their prerequisites and other, related ideas. To help our reviewers make more systematic judgments about the textbooks’ coherence, we adapted conceptual strand maps developed for Atlas of Science Literacy (AAAS, 2001). For each of four biology topics examined, we provided reviewers with a map that suggested an ideal of coherence by displaying three kinds of connections among ideas:

• Connections among the key ideas that served as the basis for the analysis (mapped as black lines)
• Connections between the key ideas and their prerequisites (mapped as red lines)
• Connections between the key ideas and other, related ideas that could strengthen students’ understanding of them (mapped as blue lines)

(See list of Topic Maps to view the “What the Reviewers Looked For” map for each topic.) We also provided reviewers with written clarifications of the connections among the ideas for each topic, which they used as guidelines and examples to inform their judgments.

Reviewers used the maps to keep track of the connections they found in each textbook, to facilitate discussion of their findings, and to resolve differences in their judgments. Once the evaluation was complete, Project 2061 created maps summarizing what the reviewers found in each textbook for each of the four topics (see sample textbook map) and what they found across all nine textbooks for each topic.

The set of key ideas on Matter and Energy Transformations is highly complex, spanning four levels of biological organization (molecular, cellular, organism, and ecosystem) and depending heavily on knowledge in physical science (e.g., energy forms and transformations among them, and recombination of atoms in chemical reactions). The risk of incoherence is strong on this topic because the learning goals and their component ideas, their prerequisite ideas, and other supporting ideas are often distributed over widely separated chapters. And given the way most textbooks are organized (from molecules to cells to organisms to ecosystems), different parts of the same key idea may be presented far apart from one another in the textbook.

Comparing the map “What the Reviewers Looked For” with the map “Composite of What the Reviewers Found” across all textbooks provides insight into what the textbooks typically do and don’t do in presenting this topic (see Topic Maps: Matter and Energy Transformations). While most key ideas are treated in most textbooks, at least in part, the composite map shows at a glance that the textbooks stop short of treating the culminating idea that living organisms “share with all other natural systems the same physical principles of the conservation and transformation of matter and energy....” (Idea “e” on the map). The map also helps us see that only four connections between key ideas are treated in most of the textbooks, while many other connections are only treated in some texts or in one text at best. Treating only part of the story or failing to make important connections among the parts of the story could leave students with fragmented knowledge. We recognize that there is no way that all possible connections can be noted in a textbook. Judgments must be made about which connections are the most important and powerful for giving students a sense of coherence rather than fragmentation. By explicitly defining coherence and using detailed conceptual strand maps to identify and record the treatment of ideas and the important connections among them, Project 2061’s biology textbook study was able to bring consistency to the evaluation of coherence.

Missed Opportunities
The Matter and Energy Transformations maps now available online are best read in conjunction with the detailed content analysis reports also available for each textbook, and we recommend printing out the maps to have on hand as you browse the reports. Each report describes how completely the textbook addresses the set of key ideas for the topic and goes on to assess how well the textbook makes connections among those ideas.

Reading the reports for several textbooks provides a sense of the missed opportunities to tell a more coherent story, but also shows the potential for improvement. By carefully analyzing what important connections are made and not made, and then mapping the findings in a convenient visual summary, we can pinpoint neglected areas and identify what is needed. Incorporating this kind of mapping strategy into the design or revision of curriculum materials could help prevent the fragmentation of content in textbooks and increase the likelihood that teachers and students will encounter a coherent science story in their materials.

# # #

This article draws on a reliability study, conducted as part of Project 2061’s high school biology textbook evaluation, that examined the consistency of reviewers’ judgments about the textbooks’ content coherence for the Matter and Energy Transformations topic. Details about the design and results of that reliability study are forthcoming.

In addition to the maps and detailed evaluation reports for Matter and Energy Transformations, High School Biology Textbooks: A Benchmarks-Based Evaluation provides information about the evaluation, details about Project 2061’s analysis procedure, and background reading on Project 2061’s science textbook evaluations. Detailed reports for the other three topics evaluated (Cell Structure and Function, Molecular Basis of Heredity, and Natural Selection and Evolution) will be published at a later date.

For more information about Project 2061’s science textbook evaluations and research on curriculum coherence, please contact:

Principal Investigator: Dr. Jo Ellen Roseman, (202) 326-6752

---

[Table of Contents]