Project 2061 LogoAAAS Project 2061
AAAS  :: Project 2061  :: Textbook Evaluations


Middle Grades Science Textbooks: A Benchmarks-Based Evaluation

Science Insights. Addison-Wesley Publishing Company, 1996 and 1997
Earth Science Life Science Physical Science

1.
About this Evaluation Report
2.
Content Analysis
3.
Instructional Analysis
  Categories
I. [Explanation] This category consists of criteria for determining whether the curriculum material attempts to make its purposes explicit and meaningful to students, either in the student text itself or through suggestions to the teacher. The sequence of lessons or activities is also important in accomplishing the stated purpose, since ideas often build on each other.
II. [Explanation] Fostering understanding in students requires taking time to attend to the ideas they already have, both ideas that are incorrect and ideas that can serve as a foundation for subsequent learning. This category consists of criteria for determining whether the curriculum material contains specific suggestions for identifying and addressing students’ ideas.
III. [Explanation] Much of the point of science is to explain phenomena in terms of a small number of principles or ideas. For students to appreciate this explanatory power, they need to have a sense of the range of phenomena that science can explain. The criteria in this category examine whether the curriculum material relates important scientific ideas to a range of relevant phenomena and provides either firsthand experiences with the phenomena or a vicarious sense of phenomena that are not presented firsthand.
IV. [Explanation] Science literacy requires that students understand the link between scientific ideas and the phenomena that they can explain. Furthermore, students should see the ideas as useful and become skillful at applying them. This category consists of criteria for determining whether the curriculum material expresses and develops the key ideas in ways that are accessible and intelligible to students, and that demonstrate the usefulness of the key ideas and provide practice in varied contexts.
V. [Explanation] Engaging students in experiences with phenomena (category III) and presenting them with scientific ideas (category IV) will not lead to effective learning unless students are given time, opportunities, and guidance to make sense of the experiences and ideas. This category consists of criteria for determining whether the curriculum material provides students with opportunities to express, think about, and reshape their ideas, as well as guidance on developing an understanding of what they experience.
VI. [Explanation] This category consists of criteria for evaluating whether the curriculum material includes a variety of aligned assessments that apply the key ideas taught in the material.
VII. [Explanation] The criteria in this category provide analysts with the opportunity to comment on features that enhance the use and implementation of the curriculum material by all students.
  References

I. Providing a Sense of Purpose

Conveying unit purpose (Rating = Poor)

Both units and chapters open with photographs and questions. In the case of the units, no purpose is stated, and no link is made between the opening photograph and what is to be studied in the unit. In the case of the chapters, there is a stated purpose, but it is brief and uses abstractions that are not likely to be familiar to students as they start the chapter (for example, “As you read this chapter, you will learn about matter, phases of matter, and changes in matter” [p. 134s], or “As you read this chapter, you will learn about atoms, elements, compounds, and mixtures” [p. 156s]). Hence, the purposes are not likely to be comprehensible or interesting to students. Although the lessons in each chapter are consistent with the purpose stated, the purpose is not returned to at the end of the chapter.

Conveying lesson/activity purpose (Rating = Poor)

Each section in a chapter starts with a list of objectives for students. These are brief statements of what students will do (but not why) that often use terms that they probably will not know before they have studied the section (for example, “Make models illustrating the gas laws” [p. 140s]). Hence, the objectives are unlikely to convey a purpose in an understandable way for the text segments that follow. The Teacher’s Edition gives a modest purpose for the few activities included (“To help students understand some of the properties of matter” [p. 135t], “To help students understand changes in matter” [p. 147t]), but teachers are not instructed to convey such purposes to students.

Justifying lesson/activity sequence (Rating = Poor)

Sections in a chapter or text segments in a chapter section do not follow a student-centered strategic sequence. For example, the properties of matter (which are familiar to or readily observable by students) are introduced after the particle model of matter (which is not directly observable by students) has been described. Nevertheless, the sections appear to be sequenced logically (e.g., Chapter 6: Properties of Matter includes the sections Matter, Phases of Matter, and Changes in Matter), although a clear rationale for the sequence cannot be inferred always.




II. Taking Account of Student Ideas

Attending to prerequisite knowledge and skills (Rating = Poor)

Science Insights: Exploring Matter and Energy does not alert teachers to the ideas that students need to know before they can understand the kinetic molecular theory.

Before the particle theory of matter is described, matter is defined and an activity is suggested to demonstrate that air is matter. Before phases of matter are explained with the particle model, students are asked to think about how ice, steam, and liquid water are similar and how they are different. But, overall, not enough experiences with relevant phenomena (such as the characteristics of states and the changes of state) are given before the kinetic molecular theory is introduced. Although a few statements are included that deal with prerequisite information, none of them try to make explicit connections between the prerequisite information and the key ideas.

Alerting teachers to commonly held student ideas (Rating = Poor)

Although there is a feature called Misconceptions, the only item listed that is relevant for the key ideas states, “Students may identify matter as only things that they can see” (p. 135t). All of the other items focus on peripheral issues, such as, “Students may identify steam as a gas” (p. 140t). Teachers are not warned about any of the important areas of difficulty with regard to the kinetic molecular theory noted in research studies. For example, research on student understanding of the structure of matter reveals that many students think that particles (atoms or molecules) are in substances and/or that there is something (e.g., air) between the particles, rather than that substances consist of nothing except molecules with empty spaces between them (Brook, Briggs, & Driver, 1984; Nussbaum, 1985). Research also reveals that, often, students are confused about the observable properties of substances versus the properties of the molecules themselves. For example, students may think that the molecules themselves become hot or cold, or that the molecules themselves expand, causing substances to expand (Johnston & Driver, 1989; Lee, Eichinger, Anderson, Berkheimer, & Blakeslee, 1993). Teachers are not alerted to these widely believed student ideas.

Assisting teachers in identifying their students’ ideas (Rating = Poor)

Students are never given specific tasks or asked questions that would reveal their prior understanding of the structure of matter and the kinetic molecular theory before the ideas are introduced. Although there is a component called Prior Knowledge, it does not appear in Chapter 6: Properties of Matter, the main chapter in which the kinetic molecular theory is presented.

Addressing commonly held ideas (Rating = Poor)

None of the major misconceptions related to the structure of matter and the kinetic molecular theory are addressed. In the only instance in which students are alerted to a relevant misconception—that they may identify matter as only things that they can see—teachers are told: “Stress that there are two ways to identify matter: Does it take up space and have mass?” (p. 135t). Teachers are then told: “Demonstrate that air is matter by squeezing a sealed plastic bag to show how it moves when displaced” (p. 135t). However, this activity may not help students who associate the existence of air only with the sensation felt when it moves (Driver et al., 1994). In addition, the text does not suggest that teachers demonstrate that air has mass.




III. Engaging Students with Relevant Phenomena

Providing variety of phenomena (Rating = Poor)

While a variety of phenomena are provided to support the idea that increased temperature means greater molecular motion, so that most substances expand when heated (Idea d), very few are provided to support the other key physical science ideas. Students read about the increase of pressure in spray cans when they are heated (p. 144s) and the expansion of the liquid in thermometers (p. 210s); and they are asked to discuss the expansion of air in balloons that are heated (p. 225t) and the increase of pressure in gasoline tanks that are left in the sun (p. 144t). The arrangement and interaction of particles of solids are linked to the properties of shape, volume, and hardness, but the links are not always clear and explicit. One example of melting and one of boiling are associated with a molecular explanation (albeit minimally) but no examples are given of freezing or condensation. There are hardly any phenomena that support the ideas that all matter is made of particles (Idea a), that particles are too small to see even with magnification (Idea b), that particles are constantly in motion (Idea c), or the different arrangement, motion, and interaction of the particles of liquids and gases (Idea e).

Providing vivid experiences (Rating = Poor)

There are no first-hand experiences with relevant phenomena. All of the relevant phenomena are mentioned in passing or are described briefly in the Student Edition. Hence, it is unlikely that middle grades students will be able to form a mental picture of these phenomena.


IV. Developing and Using Scientific Ideas

Introducing terms meaningfully (Rating = Fair)

Typically, Science Insights: Exploring Matter and Energy does not provide experiences with phenomena and then develop definitions of the terms needed to interpret the experiences. In several instances, terms are introduced before students have had any experience to help them understand the terms (for example, the introduction of the terms “particle” [p. 135s], “liquid” [p. 141s], and “evaporation” and “condensation” [p. 141s]). However, usually the key physical science ideas are presented without the introduction of excessive technical terms.

Representing ideas effectively (Rating = Poor)

Overall, there are only a few relevant representations, and most of them are not helpful in making the ideas of the kinetic molecular theory intelligible to students. Some of the drawings of molecules in the Student Edition are hard to interpret and misleading. For example, one drawing is supposed to represent air molecules in a balloon with small black balls, but it may not be clear to students whether the balls are molecules that make up the balloon or molecules that make up air. Moreover, the drawing may reinforce the commonly held student misconception that the air contains molecules (rather than consists of molecules) (Figure 6.9, p. 144s). The representation of the liquid phase is misleading in several drawings. The molecules are spread far apart and look almost like the gaseous phase, with more molecules per unit volume (e.g., see Figure 6.4, p. 140s). The motion of the molecules appears to be represented with lines and/or brackets, but the lines and brackets are not clear, and students are not told whether and how molecular motion is represented in the drawings (e.g., Figure 6.4, p. 140s, or Figure 6.9, p. 144s).

Demonstrating use of knowledge (Rating = Poor)

The material does not model, or include suggestions for teachers to model, the use of the kinetic molecular theory in explaining phenomena or solving problems. The Student Edition rarely presents explanations of specific phenomena. None of the few explanations included are identified for the benefit of students or teachers as examples of modeling, and the textbook does not address what good explanations are like.

Providing practice (Rating = Poor)

There are a few opportunities for students to practice the key physical science ideas. Most of the questions can be answered verbatim from the text and do not require students to use knowledge (for example, “Contrast the movement of particles in solids, liquids, and gases” [p. 154s], or “How does heat energy cause matter to change phases?” [p. 226s]). There are no questions or tasks for students to practice the ideas that particles are too small to see even with magnification (Idea b), and that particles are constantly in motion (Idea c). Question sets do not increase in complexity and there are no suggestions for teachers to provide feedback to students.

V. Promoting Students' Thinking about Phenomena, Experiences, and Knowledge

Encouraging students to explain their ideas (Rating = Poor)

Very few of the questions provide opportunities for students to express, clarify, or justify their ideas about the key physical science ideas. Most of the questions (in components such as Skills WarmUp, Portfolio, Cooperative Learning, Discuss, Explore Visually, and Directed Inquiry) do not focus on the structure of matter or the kinetic molecular theory. The introduction to the Teacher’s Edition does not describe the purpose of these components or any other features of the program as intending to have students explain their own ideas.

Guiding student interpretation and reasoning (Rating = Poor)

Occasionally, there are questions on student readings (e.g., In-Text Questions in the Student Edition, Discuss questions in the Teacher’s Edition). These questions focus on having students respond with the “correct” answers (mostly based upon information in the text) and are not likely to help them interpret or reason about the phenomena and ideas presented in the text (for example, see the In-Text Questions on page 141s and the Discuss questions on page 144t). Relevant questions rarely accompany the few activities that focus on the key physical science ideas (e.g., p. 135t, Themes in Science). Furthermore, none of the questions have helpful characteristics. Questions from readings and activities do not frame important issues, anticipate student misconceptions, or help students relate their experiences to scientific ideas. The question sets (such as Explore Visually, and Check and Explain) are collections of questions rather than strategic sequences of questions that develop understanding of the key ideas.

Encouraging students to think about what they have learned (Rating = Poor)

No opportunities are suggested for students to revise their initial ideas or to monitor their progress in other ways.



VI. Assessing Progress

Aligning assessment to goals (Rating = Poor)

For the end-of-instruction assessment, Science Insights: Exploring Matter and Energy provides two tests for each chapter and one test for each unit. They are given in a separate Assessment Program booklet. The applicable components of chapters 6 and 9—the chapters that treat the key physical ideas most extensively—have been examined in terms of the first two assessment criteria.

While this material includes some tasks that target the key physical science ideas, the number of assessment items applicable to each key idea varies. Most of the seven key ideas are not adequately assessed. Two of them—particles are extremely small (Idea b) and changes of states have a molecular explanation (Idea f)—are not assessed at all. The idea that particles are perpetually in motion (Idea c) is targeted by two items. In one item, students are asked to choose the phrase that best completes the statement, “According to the particle model of matter, all matter is _______.” Their four options are (a) “too small to see,” (b) “made up of tiny particles that are in constant motion,” (c) “made up of one type of particle,” and (d) “particles that are the same size.” (Assessment Program, p. 25, Test 6A, item 10; the answer is b). In the other item, students are asked to determine whether the following statement is true, and if it is not, to change the underlined words to make it true: “According to the particle model of matter, all matter is made up of tiny particles that are in constant motion” (Assessment Program, p. 39, Unit Test 2, item 41; the statement is true). Note that the first item also focuses on the related idea of the particulate nature of matter (Idea a).

The idea that increased temperature means greater molecular motion (Idea d) is targeted by three items. Students are to complete the statement, “Increasing the _______ of a substance will cause the particles of the substance to move faster” (Assessment Program, p. 37, Unit Test 2, item 4; the answer is “temperature” [also see p. 42, Test 9A, item 22 for a similar question]). They are also to decide whether or not it is true to state, “As the temperature of a substance increases, the movement of the particles that make up the substance decreases” (Assessment Program, p. 26, Test 6A, item 18; the statement is true). They are also to decide (and explain their decision) which of three drawings depicting particles shows the substance with the highest temperature (Assessment Program, p. 43, Test 9B, item A, question 1).

More assessment items target the idea that solids, liquids, and gases differ in their molecular motion and arrangement (Idea e), but they typically focus on the motion aspect. For example, students are to complete the statement, “The particles of a liquid move faster than the particles of a _____” (Assessment Program, p. 25, Test 6A, item 1; the answer is “solid”). Also they are to choose the best word to complete the statement, “Particles of matter move the slowest in a (a) solid, (b) liquid, (c) gas, (d) plasma” (Assessment Program, p. 26, Test 6A, item 13; the answer is a [see also p. 39, Unit Test 2, item 31 for a similar question]). Similarly, they are to complete the statement, “You can compress a gas because (a) there is much empty space between its particles, (b) it is made up of elements, (c) it is made up of small particles, (d) its particles are too large” (Assessment Program, p. 39, Unit 2 Test, Item 33; the answer is a [see also p. 26, Test 6A, item 14 for a similar question]). Also they are to decide whether or not it is true to state, “The particles of a solid are the most tightly bound of any phase of matter” (Assessment Program, p. 26, Test 6A, item 25; the statement is true). Also, they are to draw models to show the particles of one substance in its solid, liquid, and gas phase (Assessment Program, p. 28, Test 6B, item C, question 2), and they are prompted to add arrows to their drawings to indicate the range of motion.

Testing for understanding (Rating = Poor)

The provided assessment items do not ask students to use the key physical science ideas to explain phenomena, identify examples for the general principle, or make predictions. Of the relevant assessment items described above under “Aligning assessment to goals,” only two require application of the key ideas. Students are shown three drawings of particles with arrows in different lengths indicating how fast each particle is moving and are asked to decide (and explain their decision) which drawing shows the substance with the highest temperature (Assessment Program, p. 43, Test 9B, item A, question 1). Also, they are to draw models to show the particles of one substance in its solid, liquid, and gas phase (Assessment Program, p. 28, Test 6B, item C, question 2).

Using assessment to inform instruction (Rating = Poor)

Science Insights: Exploring Matter and Energy does not make explicit claims about assessment items that could be used to find out where students are and modify instruction accordingly. Nonetheless, reviewers examined whether questions throughout the relevant chapters could help a well-informed teacher to identify students’ remaining difficulties. While most relevant items in the evaluate and the Chapter Review sections of chapters 6 and 9 are based on regurgitation of statements in the text, a few items require application of the key ideas. For example, students draw a diagram to show the spacing of particles in a solid, liquid, and gas (p. 146s, item 2) and explain sublimation in molecular terms (p. 154s, Check Your Understanding, item 6). Unfortunately, the material does not include suggestions about how to interpret students’ responses and make informed modifications in the instruction. Given the extensive literature on students’ naive ideas related to the kinetic molecular theory, this is a serious flaw.

VII. Enhancing the Science Learning Environment

Providing teacher content support (Minimal support is provided.)

The material provides minimal support in alerting teachers to how ideas have been simplified for students to comprehend and what the more sophisticated versions are. Content background notes only briefly summarize the student text for units (e.g., p. 132t, Unit Overview) and chapters (e.g., p. 134At, Overview; p. 208At, Overview).

The material rarely provides sufficiently detailed answers to questions in the Student Edition for teachers to understand and interpret various student responses. Most answers are brief and require further explanation (for example, “Ending masses are the same” [p. 145t, SkillBuilder, item 3]); often, they emphasize factual recall of information from the student text (for example, “As heat energy increases, molecules move faster” [p. 212t, Check and Explain, item 1]).

The material provides minimal support in recommending resources for improving the teachers’ understanding of key ideas. While the material lists references at the beginning of each chapter that could help teachers improve their understanding of key ideas (e.g., “Berger, Melvin. Solids, Liquids, Gases. New York: G. P. Putnam’s Sons, 1989” [p. 134Bt]), the lists lack annotations about what kind of information the references provide or how they may be helpful.

Encouraging curiosity and questioning (Minimal support is provided.)

The material provides no suggestions for how to encourage student questions and guide their search for answers.

The material provides a few suggestions for how to respect and value students’ ideas. Introductory teacher’s notes about concept mapping state that “each student connects concepts differently” and that working with other students in constructing concept maps “gives students valuable experience in comprehending and communicating the meanings of scientific concepts and terms” (p. T–39). In addition to concept mapping, the material explicitly elicits and values students’ ideas in some activities. For example, teacher notes state that answers may vary (e.g., p. 141t, Skills Development) for selected tasks. Also, each chapter begins with a feature entitled “What do you see?” which consists of a quote from an actual student about the chapter opening photograph (e.g., p. 208s).

The material provides a few suggestions for how to raise questions such as “How do we know? What is the evidence?” and “Are there alternative explanations or other ways of solving the problem that could be better?” But it does not encourage students to pose such questions themselves. Specifically, the material includes a few tasks that ask students to provide evidence or reasons in their responses (e.g., p. 136t, Explore Visually, item 3; p. 149st, Skills WorkOut; p. 220st, Check and Explain, item 2).

The material provides a few suggestions for how to avoid dogmatism. The first chapter portrays the nature of science as a durable yet dynamic human enterprise in which students can participate (pp. 2–10s). The material later illustrates changes over time in scientific thinking leading to the current concept of heat (p. 210s, Historical Notebook). However, the material also contributes to dogmatism by presenting most of the text in a static, authoritative manner with little reference to the work of particular practicing scientists and by expecting single specific responses for most student tasks.

The material does not provide examples of classroom interactions (e.g., dialogue boxes, vignettes, or video clips) that illustrate appropriate ways for teachers to respond to student questions or ideas. However, a limited sense of desirable student-student interactions may be gained from procedural directions for laboratory and cooperative group activities (e.g., p. T–37; p. 221st, Activity 9; p. 224t, Cooperative Learning; Laboratory Manual, pp. T–x, T–xi).

Supporting all students (Some support is provided.)
The material generally avoids stereotypes or language that might be offensive to a particular group. For example, photographs include a diverse cultural mix of students and adults (e.g., pp. 150s, 172s, 258s), but the number of photographs that include people are few throughout the material. In addition, the material’s use of the short story genre (e.g., pp. 204–205st, 252–253st) related to the traditional expository text may support the language use of particular student groups.

The material provides some illustrations of the contributions of women and minorities to science and as role models. Introductory notes highlight multicultural perspectives and suggest that teachers “Tell students about the contribution to science and technology of people from diverse ethnic and cultural backgrounds” (p. T–36). Throughout the material, however, most of the contributions of women and minority scientists appear in special features. Science and Society sections relate chapter content to human activities and sometimes focus on the contributions of particular cultural groups. For example, one Science and Society section discusses natural medicine practices of various cultures including some African tribes’ use of particular plants to treat snakebite (pp. 172–173s). The Historical Notebook feature emphasizes historical contributions of particular cultural groups (e.g., p. 289s). The Career Corner feature briefly describes a scientific occupation related to the chapter content and includes a photograph of a scientist; in some instances the scientist is a woman or minority (e.g., p. 150s). Multicultural Perspectives are general directions in teacher’s notes for projects related to the chapter content in which students often research particular characteristics of a cultural group (e.g., p. 218t). All of these sections highlighting cultural contributions are interesting and informative but may not be seen by students as central to the material because they are mostly presented in sidebars and teacher’s notes.

The material suggests multiple formats for students to express their ideas during instruction, including individual investigations (e.g., p. 135st, Skills WarmUp), journal writing (e.g., p. 132t, Writing About the Photograph), laboratory investigations (e.g., p. 221st, Activity 9), cooperative group activities (e.g., p. 224t, Cooperative Learning), whole class discussion (e.g., p. 142t, Skills Development), essay questions (e.g., p. 139st, Check and Explain, item 1), creative writing (e.g., p. 208t, Writing Connection), making models (e.g., p. 139st, Check and Explain, item 4), and visual projects (e.g., p. 136t, Art Connection). In addition, multiple formats are suggested for assessment, including essay (e.g., p. 154st, Check Your Understanding, item 6), concept mapping (e.g., p. 155st, Make Connections, item 1), portfolio (e.g., p. 151t, WrapUp), creative writing (e.g., p. 139t, WrapUp), and visual projects (e.g., p. 212t, WrapUp). However, the material does not usually provide a variety of alternatives for the same task in either instruction or assessment.

The material does not routinely include specific suggestions about how teachers can modify activities for students with special needs. However, the Teacher’s Edition and supplementary materials (including reinforcement, enrichment and review worksheets, activities and laboratory investigations) provide additional activities and resources for students of specific ability levels. Teacher’s notes at the beginning of each chapter provide additional activities for limited English proficiency, at-risk, and gifted students (e.g., p. 134At, Individual Needs), and give ability level designations (core, standard, and enriched) for all chapter components (e.g., p. 208Bt, Chapter 9 Planning Guide). For Spanish speakers, there is a Spanish Supplement, which translates chapter summaries and the glossary, and Spanish Section Reviews (worksheets). However, the placement of some supplemental resources in individual booklets separate from the main text may discourage their use, and the special needs codes at the beginning of chapters may discourage teachers from using those activities with all students when appropriate.

The material provides some strategies to validate students’ relevant personal and social experiences with scientific ideas. Some text sections intersperse brief references to specific personal experiences students may have had that relate to the presented scientific concepts (e.g., p. 209s). In addition, some tasks ask students about particular, personal experiences they may have had or suggest specific experiences they could have. For example, a Skills Workout asks students to list the items they used that day which were in the gas phase as well as things that were solid and liquid. The activity then asks students if they used the same substance in two different phases, and if so, how that was possible (p. 142st). However, the material rarely encourages students to contribute relevant experiences of their own choice to the science classroom, and sometimes it does not adequately link the specified personal experiences to the scientific ideas being studied (e.g., p. 208t, Writing Connection). Overall, support is brief and localized.