| Earth Science | Life Science | Physical Science |
About this Evaluation Report Content Analysis Instructional Analysis
| Categories | |
| I. | [Explanation] This category consists of criteria for determining whether the curriculum material attempts to make its purposes explicit and meaningful to students, either in the student text itself or through suggestions to the teacher. The sequence of lessons or activities is also important in accomplishing the stated purpose, since ideas often build on each other. |
| II. | [Explanation] Fostering understanding in students requires taking time to attend to the ideas they already have, both ideas that are incorrect and ideas that can serve as a foundation for subsequent learning. This category consists of criteria for determining whether the curriculum material contains specific suggestions for identifying and addressing students’ ideas. |
| III. | [Explanation] Much of the point of science is to explain phenomena in terms of a small number of principles or ideas. For students to appreciate this explanatory power, they need to have a sense of the range of phenomena that science can explain. The criteria in this category examine whether the curriculum material relates important scientific ideas to a range of relevant phenomena and provides either firsthand experiences with the phenomena or a vicarious sense of phenomena that are not presented firsthand. |
| IV. | [Explanation] Science literacy requires that students understand the link between scientific ideas and the phenomena that they can explain. Furthermore, students should see the ideas as useful and become skillful at applying them. This category consists of criteria for determining whether the curriculum material expresses and develops the key ideas in ways that are accessible and intelligible to students, and that demonstrate the usefulness of the key ideas and provide practice in varied contexts. |
| V. | [Explanation] Engaging students in experiences with phenomena (category III) and presenting them with scientific ideas (category IV) will not lead to effective learning unless students are given time, opportunities, and guidance to make sense of the experiences and ideas. This category consists of criteria for determining whether the curriculum material provides students with opportunities to express, think about, and reshape their ideas, as well as guidance on developing an understanding of what they experience. |
| VI. | [Explanation] This category consists of criteria for evaluating whether the curriculum material includes a variety of aligned assessments that apply the key ideas taught in the material. |
| VII. | [Explanation] The criteria in this category provide analysts with the opportunity to comment on features that enhance the use and implementation of the curriculum material by all students. |
| References |
I. Providing a Sense of Purpose
Conveying
unit purpose (Rating = Poor)
Conveying lesson/activity
purpose (Rating = Poor)
Justifying lesson/activity
sequence (Rating = Poor) There is no sequential organization of the readings
and activities that the reviewers could determine for
course 3, chapter 7. For example, it is not clear why
the chapter starts with the kinetic molecular theory
in the context of solids and liquids (Section 7–1:
Solids and Liquids) and then introduces the kinetic
theory of gases (Section 7–2: Kinetic Theory of
Gases). Course 2 introduced the kinetic molecular theory
in the context of gases, so one could expect the material
to start where it left off. Moreover, section 7–1
contains a subsection on evaporation and condensation
and their molecular explanations before the
kinetic theory of gases is introduced in section 7–2.
II. Taking Account of Student Ideas
Attending to prerequisite
knowledge and skills (Rating = Poor) Science Interactions does not alert teachers
to specific prerequisite ideas, nor does it point explicitly
to the earlier chapters in which prerequisites are addressed.
The feature called Tying to Previous Knowledge often
directs teachers to review topics from previous chapters,
but, generally, the importance of reviewing these topics
is not mentioned to them. Which specific ideas from
these general topics need to be reviewed is not mentioned,
the topics to be reviewed are not identified clearly
as prerequisites, they are not linked specifically to
chapter topics, and the question of why such review
is important is not answered. For example, in course
2, chapter 14, Section 14-1: How Do Gases Behave? the
Tying to Previous Knowledge portion suggests to teachers,
“Review the three physical phases of matter—solid,
liquid, and gas” (p. 427t). It does not state
explicitly what about solids, liquids, and
gases they should review. In course 3, chapter 7, at
the beginning of a section on the kinetic molecular
theory, Tying to Previous Knowledge instructs teachers
to review the concepts of kinetic and potential energy
(p. 211t). Not only is it not explained why students
need to review these concepts, but also the issue is
confused by including potential energy (which is not
relevant for the section that follows).
Alerting teachers to commonly
held student ideas (Rating = Poor)
Assisting teachers in identifying
their students’ ideas (Rating = Poor) Taken together, the questions in these features focus
mostly on finding out the status of students’
knowledge with respect to some prerequisites to the
key physical science ideas. For example, in course 2,
chapter 14, the teacher is told to “Have students
list all properties of gases that they can think of”
(Tying to Previous Knowledge, p. 424t); in course 3,
chapter 7, the teacher is told to write on the board
the headings “gases,” “liquids,”
and “solids” and have the class list under
each heading anything they associate with it (Tying
to Previous Knowledge, p. 208t). The rest of the questions
are related only peripherally to the key ideas, and
most could be answered at the macroscopic level: “Why
[does] a hairspray can [feel] cold when you use it?”
(course 2, chapter 14, p. 424s); “Why [do] you
feel cooler if you splash yourself with water?”;
“Why [does] the tar [squeeze] out of street cracks
in warm weather?” (course 3, chapter 7, p. 208s).
Teachers are not asked to probe students’ responses
to determine any microscopic ideas they may have; however,
their doing so would be particularly relevant for the
last two questions from course 3 because molecular ideas
have already been introduced in course 2. In summary,
although a few questions are included that could help
to identify students’ ideas (if teachers were
to use them for this purpose), they are insufficient
to elicit the numerous misconceptions and to identify
the various areas of difficulty that we know from research
studies students have with the topic of the kinetic
molecular theory.
Addressing commonly held
ideas (Rating = Poor) However, it should be noted that the prerequisite idea
that air is matter is addressed long after
the idea that all matter (including air) is made of
particles has been introduced and the properties of
air have been explained by using with the kinetic molecular
theory (in course 2).
Students may have difficulty appreciating
that intangible, invisible air has mass and substance
just as liquids and solids do. Have students weigh an
uninflated balloon on a sensitive balance scale. Then
have them blow up the balloon, tie it off, and weigh
it again. They should note that the added mass shown
on the scale is that of the air trapped within the balloon.
[p. 223t, Uncovering Preconceptions]
III.
Engaging Students with Relevant Phenomena
Providing variety of phenomena
(Rating = Satisfactory)
Providing vivid experiences
(Rating = Satisfactory)
IV. Developing and Using Scientific Ideas
Introducing terms meaningfully
(Rating = Satisfactory)
Representing ideas effectively
(Rating = Poor) Very few of the representations are likely to be helpful
in making the ideas of the kinetic molecular theory
intelligible to students. For example, in course 3,
chapter 7, Figure 7–2 illustrates the arrangement
and motion of molecules in a solid (p. 211s). This diagram
shows eight balls held in position by springs. Captions
state that the balls represent the particles of which
the solid is made, while the springs represent forces
between these particles. While springs hold the balls
in position, the balls can vibrate around their rest
positions. Still, the text does not discuss explicitly
how the representation is unlike what it is supposed
to demonstrate. Furthermore, since students often think
that something is between particles—rather than
a void (Brook,
Briggs, & Driver, 1984; Nussbaum,
1985), this representation could reinforce that
erroneous idea.
Demonstrating use of knowledge
(Rating = Poor)
Providing practice (Rating
= Fair)
V. Promoting Students' Thinking about Phenomena, Experiences, and Knowledge
Encouraging students to
explain their ideas (Rating = Fair) For the most part, students are not instructed to discuss
topics in small groups, nor are teachers directed to
use a style of discussion that involves everyone in
the class. Furthermore, the teacher is not instructed
explicitly to provide feedback to students, nor are
suggestions made about how feedback could be given to
help students diagnose their errors or develop their
ideas more fully.
Guiding student interpretation
and reasoning (Rating = Poor)
Encouraging students to
think about what they have learned (Rating = Poor) At the end of each chapter, before students are asked
to review their responses to the Did you ever wonder…
questions, they are asked to review statements about
the major ideas presented in the chapter and to answer
accompanying questions. Then, they are asked to write
a paragraph about how their understanding of the major
ideas has changed. Although the ideas and the questions
posed relate well to the key physical science ideas
(e.g., “Kinetic-molecular theory states that the
particles that compose all forms of matter are in constant
motion”; “Temperature of a gas is a measure
of the particles’ average kinetic energy. The
greater the temperature, the faster the particles move
and the harder they collide” [course 3, chapter
7, p. 234s]), it is not clear whether students will
be able to respond effectively to the question about
how their understanding of the major ideas has changed
without having been asked to consider questions related
to these big ideas before. As noted above, the Did you
ever wonder… questions do not address the main
ideas; hence, reviewing the responses to these questions
will not help. Another feature, Flex Your Brain, has students write
what they know about a topic, research the topic further,
write about what they found, and compare their new knowledge
to their original statement. It is possible that students
will monitor their learning by using this feature. However,
only one Flex Your Brain activity is included in the
relevant physical science chapters. Furthermore, the
topic given is vague, namely, solids and liquids (course
3, chapter 7, p. 217t). It is uncertain what students
will investigate, and there is no guidance or indication
that the key physical science ideas will be explored.
Aligning assessment to
goals (Rating = Fair) Science Interactions includes several assessment
items that focus on the key physical science ideas.
However the number of questions provided is inconsistent
across the set of key physical science ideas, and most
of the key physical science ideas are not adequately
assessed. Several assessment items focus on parts of
the key idea that delineates the different arrangement
and motion of particles in solids, liquids, and gases
(Idea e). For example, students respond to this true/false
question: “Molecules in liquids have stronger
attractive forces than do molecules in solids”
(Review and Assessment, course 3, Chapter 7 Review,
p. 41, item 10) and complete the sentence: “When
molecules are in a fixed arrangement, the state of matter
must be _______” (Review and Assessment,
course 3, Chapter 7 Test, p. 43, item 10). For most
of the other key physical science ideas (Ideas a, c,
d, f), one or two assessment items are provided. For
example, two questions focus on the molecular explanations
of changes of states (Idea f). In one, students explain
what happens to a material’s molecules when a
solid becomes a liquid (Review and Assessment,
course 3, Chapter 7 Test, p. 44, item 20). The other
asks why drops of liquid appear on the outside of a
cold glass on a hot summer day (Review and Assessment,
course 3, Chapter 7 Test, p. 45, item 25). For the idea
that all matter is made up of particles (Idea a), one
item is provided. Students complete this sentence: “The
idea that matter is composed of small (particles)
called atoms is explained by the atomic theory of matter”
(Review and Assessment, course 2, Chapter 14
Review, p. 83, item 9). (While two questions focus on
evidence for the particulate nature of matter from the
law of definite proportions, Benchmarks for Science
Literacy [American
Association for the Advancement of Science, 1993]
and National Science Education Standards [National
Research Council, 1996] consider the law of definite
proportions to be too sophisticated for middle grades
students). The idea that particles of matter are extremely
small (Idea b) is not assessed.
Testing for understanding
(Rating = Fair)
Using assessment to inform
instruction (Rating = Poor)
Providing teacher content
support (Minimal
support is provided.) The material rarely provides sufficiently detailed
answers to questions in the student text for teachers
to understand and interpret various student responses.
Most answers are brief and require further explanation
(e.g., “The temperature of a gas corresponds to
the average kinetic energy of its molecules” [course
3, p. 230t, Check Your Understanding, item 2]); some
emphasize a “right-answer” approach (e.g.,
“A straight line graph should result” [course
2, p. 434t, Conclude and Apply, item 1]). The material provides minimal support in recommending
resources for improving the teacher’s understanding
of key ideas. While the material lists references in
the introductory notes of the Teacher Wraparound
Edition (e.g., “Laidler, Keith J. The
World of Physical Chemistry. New York: Oxford University
Press, 1993” [course 1, p. 47T]), National Geographic
resources at the beginning of each chapter (e.g., course
2, p. 424Bt, National Geographic Teacher’s Corner),
and websites throughout the book (e.g, course 2, p.
413t, interNET CONNECTION) that could help teachers
improve their understanding of key ideas, the lists
lack annotations about what kinds of information the
references provide or how they may be helpful.
Encouraging curiosity
and questioning (Minimal
support is provided.) The material provides a few suggestions for how to
respect and value students’ ideas. Introductory
teacher’s notes about cooperative learning state
that students will “recognize…the strengths
of others’ [perspectives],” be presented
with “the idea that there is no one, ‘ready-made’
answer” (courses 1–3, p. 22T), and “respect
other people and their ideas” (courses 1–3,
p. 33T). Introductory teacher’s notes also state
that student responses may vary in concept mapping tasks.
Teachers are thus instructed to “[l]ook for the
conceptual strength of student responses, not absolute
accuracy” (courses 1–3, p. 26T). A special
feature, Teens in Science, describes specific students
conducting experiments and activities related to the
chapter content (e.g., course 2, p. 478st). In addition,
Design Your Own Investigation and Investigate! activities
(e.g., course 2, pp. 430–431st, Investigate!)
are structured to be open-ended, allowing students to
pursue a laboratory task in various ways. However, teacher’s
notes often give specific, expected outcomes for these
activities that may limit their intended open-ended
nature (e.g., course 2, pp. 442–443t, Design Your
Own Investigation). The material provides a few suggestions for how to
raise questions such as, “How do we know? What
is the evidence?” and “Are there alternative
explanations or other ways of solving the problem that
could be better?” But it does not encourage students
to pose such questions themselves. Specifically, the
material includes a few tasks that ask students to provide
evidence or reasons in their responses (e.g., course
2, p. 426t, Assessment; course 2, p. 439st, Check Your
Understanding, item 3). The material provides a few suggestions for how to
avoid dogmatism. Introductory teacher’s notes
state, “Science is not just a collection of facts
for students to memorize” but instead “a
process of applying those observations and intuitions
to situations and problems, formulating hypotheses,
and drawing conclusions” (courses 1–3, p.
23T). The first chapter portrays the nature of science
as a human enterprise that proceeds by trial and error
and uses many skills familiar to students (course 1,
pp. 2–17st). However, most of the text is generally
presented in a static, authoritative manner with little
reference to the work of particular practicing scientists,
and single specific responses are expected for most
student tasks. The material does not provide examples of classroom
interactions (e.g., dialogue boxes, vignettes, or video
clips) that illustrate appropriate ways to respond to
student questions or ideas. However, a limited sense
of desirable student-student interactions may be gained
from procedural directions for laboratories and cooperative
group activities (e.g., course 2, pp. 430–431st,
Investigate!; course 3, pp. 222st, Find Out!; Cooperative
Learning in the Science Classroom resource book).
Supporting all students
(Some
support is provided.) The material provides some illustrations of the contributions
of women and minorities to science and as role models.
While the introductory teacher’s notes state the
goal of multicultural education as being to promote
“the understanding of how people from different
cultures approach and solve the basic problems all humans
have in living and learning” (courses 1–3,
p. 25T), most of the contributions of women and minorities
appear in special features. Science Connections emphasize
associations among the various science disciplines and
society. Some of these essays describe scientific contributions
of women and minorities (e.g., course 3, p. 233st, Technology
Connection). In addition, Multicultural Perspectives
teacher’s notes highlight specific cultural contributions
related to chapter topics (e.g., course 2, p. 438t).
A separate Multicultural Connections resource
book contains short readings and questions about individual
scientists or groups addressing text-related issues
in many parts of the world. For example, the book includes
a reading activity about Walter E. Massey, an African
American theoretical physicist who worked to improve
the science teaching of minority students (e.g., Multicultural
Connections, p. 11). All of these sections highlighting
cultural contributions are interesting and informative
but may not be seen by students as central to the material
because they are presented in sidebars, supplemental
materials, and teacher’s notes. The material suggests multiple formats for students
to express their ideas during instruction, including
individual investigations (e.g., course 2, p. 434st,
Find Out!), journal writing (e.g., course 2, p. 424s,
Science Journal), cooperative group activities (e.g.,
course 2, p. 427t, Activity), laboratory investigations
(e.g., course 3, pp. 212–213st, Investigate!),
whole class discussions (e.g., course 2, p. 437t, Discussion),
essay questions (e.g., course 2, p. 453st, Critical
Thinking, item 4), concept mapping (e.g., course 3,
p. 235st, Developing Skills, item 1), and visual projects
(e.g., course 3, p. 233t, Teaching Strategy). In addition,
multiple formats are suggested for assessment, including
oral discussion (e.g., course 3, p. 230t, Discussion),
essay (e.g., Computer Test Bank Manual, course
3, pp. 7–8, item 12), performance (e.g., course
2, p. 437t, Assessment), and portfolio (e.g., course
2, p. 446t, Activity). However, the material does not
usually provide a variety of alternatives for the same
task (except in rare instances for special needs students). The material does not routinely include specific suggestions
about how teachers can modify activities for students
with special needs. However, the Teacher Wraparound
Edition and supplemental Program Resources provide
additional activities and resources for students
of specific ability levels. At the beginning of each
chapter, teacher’s notes link the various chapter
activities to different learning styles (e.g., course
3, p. 208t, Learning Styles), and each activity is coded
according to ability level (courses 1–3, p. 33T).
Each chapter also includes a Meeting Individual Needs
feature, which provides activities specifically designated
for students with special needs (e.g., course 2, p.
430t, Meeting Individual Needs). For Spanish speakers,
there are English/Spanish audiocassettes, which summarize
the student text in both languages, and a Spanish
Resources book, which translates chapter vocabulary
terms and definitions. However, the placement of supplemental
resources in individual booklets separate from the main
text may discourage their use, and the special needs
codes within chapters may discourage teachers from using
those activities with all students when appropriate.
The material provides some strategies to validate students’
relevant personal and social experiences with scientific
ideas. Many text sections begin with a brief reference
to a specific personal experience students may have
had that relates to the presented scientific concepts
(e.g., course 2, p. 433s). In addition, some tasks—particularly
Science at Home (e.g., course 1, p. 181t), Across the
Curriculum, Daily Life, and How It Works resource
book work sheets—ask students about particular
personal experiences they may have had or suggest specific
experiences for them to have. For example, teacher’s
notes ask students to make lists of items in their homes
that are solids, liquids, and gases (course 1, p. 139t,
Assessment). However, the material rarely encourages
students to contribute relevant experiences of their
own choice to the science classroom and sometimes does
not adequately link the specified personal experiences
to the scientific ideas being studied (e.g., course
3, p. 234t, Science at Home). Overall, support is brief
and localized.
