Earth Science | Life Science | Physical Science |
1.About this Evaluation Report 2.Content Analysis 3.Instructional Analysis
Categories | |
I. | [Explanation] This category consists of criteria for determining whether the curriculum material attempts to make its purposes explicit and meaningful to students, either in the student text itself or through suggestions to the teacher. The sequence of lessons or activities is also important in accomplishing the stated purpose, since ideas often build on each other. |
II. | [Explanation] Fostering understanding in students requires taking time to attend to the ideas they already have, both ideas that are incorrect and ideas that can serve as a foundation for subsequent learning. This category consists of criteria for determining whether the curriculum material contains specific suggestions for identifying and addressing students’ ideas. |
III. | [Explanation] Much of the point of science is to explain phenomena in terms of a small number of principles or ideas. For students to appreciate this explanatory power, they need to have a sense of the range of phenomena that science can explain. The criteria in this category examine whether the curriculum material relates important scientific ideas to a range of relevant phenomena and provides either firsthand experiences with the phenomena or a vicarious sense of phenomena that are not presented firsthand. |
IV. | [Explanation] Science literacy requires that students understand the link between scientific ideas and the phenomena that they can explain. Furthermore, students should see the ideas as useful and become skillful at applying them. This category consists of criteria for determining whether the curriculum material expresses and develops the key ideas in ways that are accessible and intelligible to students, and that demonstrate the usefulness of the key ideas and provide practice in varied contexts. |
V. | [Explanation] Engaging students in experiences with phenomena (category III) and presenting them with scientific ideas (category IV) will not lead to effective learning unless students are given time, opportunities, and guidance to make sense of the experiences and ideas. This category consists of criteria for determining whether the curriculum material provides students with opportunities to express, think about, and reshape their ideas, as well as guidance on developing an understanding of what they experience. |
VI. | [Explanation] This category consists of criteria for evaluating whether the curriculum material includes a variety of aligned assessments that apply the key ideas taught in the material. |
VII. | [Explanation] The criteria in this category provide analysts with the opportunity to comment on features that enhance the use and implementation of the curriculum material by all students. |
References |
I. Providing a Sense of Purpose
Conveying
unit purpose (Rating = Fair)
Conveying lesson/activity
purpose (Rating = Satisfactory)
Justifying lesson/activity
sequence (Rating = Satisfactory) In addition, when the order of the activities is examined
closer, several issues can be raised with regard to the sequence of particular
sections or activities within sections. For example, Chapter 9: Scientific
Explanations Are Ancient History focuses on ancient explanations related to
the concept of "elements," but this topic seems extraneous because it is not
continued in the rest of the unit. The next chapter, chapter 10, introduces
part of the particle model, specifically, just the idea that "matter is made
up of particles" (Idea a). Then students are asked to explain a variety of
phenomena that cannot be explained only with this idea. Other ideas such as
that particles are in constant motion (Idea c) and that they move farther
apart when heated (Idea d) are needed to make sense of why a balloon gets
larger when heated and why food coloring spreads out in water (pp. 201-205s).
In the next few pages, the particle model is revised, and the revised model
is shown to explain these phenomena (pp. 206-207s). This sequence is likely
to confuse students and is an inaccurate representation of the scientific
process. A logical sequence should show that the model has success
in explaining some phenomena before showing that it cannot explain other phenomena.
Otherwise, models that do not explain phenomena are discarded rather than
revised. This happens again when students are asked to explain some chemical
reactions from an earlier chapter (p. 207s). They are to conclude that the
model is incomplete before they develop confidence in their model by showing
that it can explain further observations (such as the "Coin-Dancers" and the
"Rainmakers") (pp. 208−212s). With regard to this sequencing criterion,
one review team gave the material a "satisfactory" rating, but the other
team gave it an "excellent" rating. Although the second team acknowledged
weaknesses in unit 2's sequencing of activities, it felt that the material
meets both indicators for this criterion (namely, that the material provides
a rationale for a logical or strategic sequence of activities, and that the
sequence of activities reflects the stated rationale) and hence, should receive
an "excellent" rating. Given the weaknesses in the material's sequence of
activities described above and also that the sequence of the sections does
not reflect the rationale of the five E's instructional model, the Project
2061 staff recommends a satisfactory rating.
II. Taking Account of Student Ideas
Attending to prerequisite
knowledge and skills (Rating = Poor)
Alerting teachers to commonly
held student ideas (Rating = Poor)
Assisting teachers in identifying
their students’ ideas (Rating = Poor)
Addressing commonly held
ideas (Rating = Poor)
III.
Engaging Students with Relevant Phenomena
Providing variety of phenomena
(Rating = Poor) Some phenomena are provided
that support the idea that increased temperature means
greater particle motion (Idea d); however, phenomena
are not provided that can be explained in terms of the
expansion of solids or liquids. In addition, phenomena
are presented for the idea that there are attractive
forces among particles in liquids (part of Idea e2).
However, there is little variety, and most phenomena
focus on surface tension. Very few experiences are presented
to support the idea that particles are perpetually in
motion (Idea c) or to illustrate changes of state (Idea
f). No phenomena are included on the arrangement and
motion of particles in solids, liquids, and gases (Idea
e). The choice of phenomena
is a general weakness in level B's Unit 2: Why Are Things Different? Most
phenomena included are too complex for middle school students (for example,
the chemical reactions presented in chapter 9, or the surface tension phenomena
presented in chapter 11).
Providing vivid experiences
(Rating = Satisfactory)
IV. Developing and Using Scientific Ideas
Introducing terms meaningfully
(Rating = Very Good) With regard to this introducing-terms
criterion, one review team gave the material an "excellent" rating, but the
other team gave it a "very good" rating. Given the weakness in the material's
introduction and use of the term "particle," the Project 2061 staff recommends
a "very good" rating.
Representing ideas effectively
(Rating = Poor)
Demonstrating use of knowledge
(Rating = Poor) With regard to this criterion about demonstrating
use of knowledge, one review team gave
the material a "poor" rating, but the
other team gave it an "excellent" rating.
According to the second team, "The material
does an outstanding job of demonstrating
how to construct and critique a scientific
explanation. This includes demonstrations
of how to design an investigation, how
to use ancient explanations to explain
properties of materials, how to use observations
as evidence in support of a model, how
to revise a model on the basis of new
information, how to make predictions on
the basis of a model, how to construct
if-then statements to express predictions
made on the basis of a model, and how
to critique a scientific explanation." In response to this statement, the first team pointed out
that students are asked to explain various
phenomena in terms of the particle model,
but they will not know how; rather, they
are left to arrive at their own explanations.
This evaluation focuses on whether Diversity
and Limits provides instructional
support for the idea that "all matter
is made of particles" and not for
students' ability to develop and use models.
Consequently, to satisfy this criterion,
the material needs to "demonstrate how
to explain different phenomena using the
particle model." Although Diversity
and Limits attempts to demonstrate
how to use models and points to what a
good explanation or model is in general,
it does not show students explicitly good
examples of explanations using the particle
model. Therefore, the Project 2061 staff
recommends a "poor" rating for this criterion.
Providing practice (Rating
= Poor)
V. Promoting Students' Thinking about Phenomena, Experiences, and Knowledge
Encouraging students to
explain their ideas (Rating = Satisfactory)
Guiding student interpretation
and reasoning (Rating = Satisfactory)
Encouraging students to
think about what they have learned (Rating = Fair) The rating given above for
this encouraging-students criterion is the average of the ratings of the two
review teams (the individual team ratings were "poor" and "satisfactory").
The teams debated the question of what students are asked to monitor in Diversity
and Limits: their understanding of the particle model ideas (key ideas)
or the usefulness of their models. The first team pointed out that students
do not evaluate their ideas against what is known, that the progression of
the particle model that the material suggests is not necessarily the one that
the students would use, and that giving them the steps takes the self-monitoring
out of their hands. The second team objected that the clarification of the
criterion that was available to reviewers does not say explicitly that students
need to check their own progress against "what is known." This evaluation focuses
on whether Diversity and Limits provides instructional support for
the idea that "all matter is made of particles" and not for students'
ability to develop and use models. Therefore, to satisfy this criterion, the
material needs to engage students in monitoring their progress in understanding
the particle model. While it is true that the clarification document
does not state explicitly that materials need to encourage students to check
their progress against "what is known," students appear to monitor mostly
whether their models fit the observations at hand, and hence, whether the
process they used to build their models was adequate (rather than whether
the actual models, and hence, their understanding of the particle model are
adequate). Therefore, the Project 2061 staff recommends a "fair" rating for
this criterion.
Aligning assessment to
goals (Rating = Poor) None of the other key physical
science ideas-such as the idea of different arrangement and motion of molecules
in the solids, liquids, and gases (Idea e)-are assessed.
Testing for understanding
(Rating = Poor)
Using assessment to inform
instruction (Rating = Poor) A few tasks could be used
to assess students' progress in understanding the particle nature of matter
(Idea a). One question asks students to describe the evidence scientists have
for the particle model (level B, p. 241s, item 2). Other questions ask students
to use the particle model to explain phenomena such as a coin placed on the
rim of a bottle that moves when the bottle is placed in hot water (but not
when placed in cold water) (p. 210s, item 14) and what happens when a flask
filled with cold colored water is placed in hot water (pp. 210-211s, item
8). For questions like these, found in the Evaluate sections, the introductory
material encourages teachers to ask questions such as "Why do you think.?
What evidence do you have? What do you know about x? How would you explain
x?" (p. xviiit). Unfortunately, teachers are not reminded of these questions
in the Evaluate sections throughout the chapters, so this advice may go unheeded.
Occasionally, teachers are reminded to motivate the students to provide supporting
statements for their answers (e.g., p. 210t). However, overall, the material
does not include suggestions for teachers about how to interpret students'
responses, nor does it include specific suggestions about how to use students'
responses to make decisions about instruction.
Providing teacher content
support (Some
support is provided.) The material generally provides sufficiently
detailed answers to questions in the student text for teachers to understand
and interpret various student responses (e.g., pp. 204-205t, Wrap Up, item
2). However, there are some limitations to
the responses provided in the teacher's notes, which occasionally are incomplete
(e.g., p. 207bt, Wrap Up, items 5, 7). The
material provides minimal support in recommending resources
for improving the teacher's understanding of key ideas.
The Teacher's Edition includes a list of "Websites
of Interest" with brief, bulleted descriptions and recommendations
at the beginning of each chapter (e.g., pp. 191c-191dt)
and a list of "Educational Technology Resources" (software,
CD-ROMs, videos, and laser discs) with recommendations
but not descriptions at the beginning of each unit (e.g.,
pp. 161b-161dt). A reference list subdivided by unit
without annotations is also provided at the end of the
Teacher's Edition (pp. 417-418t). Limited descriptions
of references in the Teacher's Edition identify
topics addressed, but none of the references are explicitly
linked to specific text sections or key ideas. In addition,
the Teacher's Resource Book has a Background
Resources section that includes commentary on science
standards documents and general references on learning
and assessment strategies.
Encouraging curiosity
and questioning (Some
support is provided.) The material provides some
suggestions for how to respect and value students' ideas. Teacher's notes
often stress that multiple student answers should be acceptable for questions
(e.g., p. 184t, Wrap Up). However, this may have implications for other criteria
(see segment above entitled "Providing practice"). The skills identified as
"Be open to others' ideas" (e.g., p. 192s, Working Cooperatively) and "Let
others finish without interrupting" (e.g., p. 208s, Working Cooperatively)
are stated in multiple places in the student text and teacher's notes as new
cooperative group skills. However, other than stating the skills, the material
does not usually provide students with any further support in how to enact
the skills within their cooperative groups. For the particle model of
matter, the material provides many suggestions for how to raise questions
such as, "How do we know? What is the evidence?" and "Are there alternative
explanations or other ways of solving the problem that could be better?" (e.g.,
p. 198s). However, the material does not routinely encourage students to pose
such questions themselves. The material provides
some suggestions for how to avoid dogmatism. For example, the student text
portrays the nature of science as a human enterprise in which students may
participate (e.g., pp. 172-174s, Investigation: How Can I Learn More about
That Property?). In addition, the material includes the work of particular,
practicing scientists (e.g., pp. 185-188s, Reading: Elements of Explanations),
and describes changes over time in scientific thinking about explanations
of differing material characteristics (pp. 195-198s, Reading: Another Explanation).
The material provides a
few examples of classroom interactions (e.g., dialogue boxes, vignettes, or
video clips) that illustrate appropriate ways to respond to student questions
or ideas, etc. Examples of desirable student-teacher dialogues are provided
for discussions of the "bounceability" of balls (pp. 166-166at, Process and
Procedure, part A) and the movement of food coloring in water (p. 203t, Process
and Procedure, part A). Two sample student dialogues are provided for the
development of scientific models of the particle model (pp. 232-233t, Students'
Models). Some sense of other desirable student interactions may be gained
from procedural directions and descriptions of student roles and social skills
in cooperative group activities (e.g., pp. 192-194s, Investigation: Mystery
Box).
Supporting all students
(Some support is provided.) The material provides some
illustrations of the contributions of women and minorities to science and
as role models. The illustrations sometimes occur in the main text (e.g.,
pp. 168s, 186-187s). They are also presented in sidebars that are interesting
and informative, but may not be seen by students as central to the material
(e.g., p. 166s, Sidelight on Technology). Children doing science in the video
materials as well as cartoon characters of European scientists in the text
are intended as role models for students. It is unclear, however, if students
will relate to the cartoon animal figures, and their dialogues may appear
extraneous to the main content of the text (e.g., p. 212s). The material suggests multiple
formats for students to express their ideas during instruction, including
individual investigations and notebook writing (e.g., pp. 201-205s, Investigation:
How Well Does the Particle Model Work?), cooperative group activities and
lab investigations (e.g., pp. 183-185s, Investigation: Strange Phenomena,
Part B-Puddle Mystery), and whole class discussion and drawing (e.g., p. 207bt, item 6). Introductory teacher's notes
suggest using a wide variety of classroom activities for assessment, including
in-class discussion, notebook entries, portfolios, and evaluate activities
(Teacher's Resource Book, p. 33). However, the material does not usually
provide a variety of alternatives for the same task in either instruction
or assessment. The material does not routinely
include specific suggestions about how teachers can
modify activities for students with special needs. However,
the Teacher's Edition and Teacher's Resource
Book provide some additional activities and
resources for students. Within each chapter, the Teacher's
Edition provides Further Opportunities for Learning,
in which students may further study a related interest
(e.g., p. 210t), sometimes using chapter and unit resources
including websites (e.g., p. 179bt) and educational
technology resources (software, CD-ROMs, videos, and
laser discs [e.g., pp. 161b-161dt]). The Teacher's
Resource Book includes a few extension activities
similar in complexity to those in the student text (e.g.,
pp. 183-184). The
material provides a few strategies to validate students'
relevant personal and social experiences with scientific
ideas. A few text sections include a brief reference
to a specific, personal experience students may have
had that relates to the presented scientific concepts
(e.g., p. 164s, Investigation: Bounce That Ball). In
addition, a few tasks ask students about particular
personal experiences they may have had or suggest specific
experiences they could have. For example, after reading
about the properties of translucence and hardness, students
are asked to identify products in their homes with those
properties (pp. 170-171s, Stop & Think, items 6,
9). However, the material rarely encourages students
to contribute relevant experiences of their own choice
to the science classroom, and sometimes it does not
adequately link the specified personal experiences to
the scientific ideas being studied (e.g., p. 171s, Stop
& Think, items 11-12). Overall, support is brief
and localized. Endnote Diversity and Limits describes
its use of the five E's instructional model as follows: "Each chapter goes
through a cycle of activities, and each activity exemplifies one of the "E"
words. First the students are engaged by an event or question related to the
concept that the teacher plans to introduce. Then the students participate
in one or more activities to explore the concept. This exploration provides
students with a common set of experiences from which they can initiate the
development of their understanding of the concept. In the explain phase, the
teacher clarifies the concept and defines relevant vocabulary terms. Next,
the students elaborate and build on their understanding of the concept by
applying it to new situations. Finally, the students complete an activity
that will help them and the teacher evaluate their understanding of the concept"
(level B, p. xvii).