Project 2061

Identifying Curriculum Materials for Science Literacy: A Project 2061 Evaluation Tool

This report describes an early version of Project 2061's curriculum-materials evaluation procedure. The report is based on a paper prepared for the Colloquium "Using the National Science Education Standards to Guide the Evaluation, Selection, and Adaptation of Instructional Materials," which was held at the National Research Council, November 10-12, 1996.

Introduction

With Project 2061’s publication of Science for All Americans (1989) and Benchmarks for Science Literacy (1993) and the National Research Council’s release of the National Science Education Standards (1996), there now exists a strong national consensus among educators and scientists on what all K-12 students need to know and be able to do in science, mathematics, and technology. The overwhelming similarity between Benchmarks and the NSES means that curriculum materials that support students’ learning benchmarks will likewise promote learning science standards. Valid identification of such curriculum materials is of great interest to educators nationwide, even more so since state and district frameworks are drawing heavily on the national documents.

Beginning in 1991, Project 2061 sought to quickly develop a database of reviewed materials that could be used in making local adoption decisions. Because the pool of existing materials was large there was no way that as few people as the Project 2061 staff could analyze them all. To engage a large number of people in analysis, it would be necessary to have an analysis procedure that could be used, with a reasonable amount of training, to give valid and reliable results. By valid, we mean that the conclusions reached would derive from accurate interpretations of benchmarks and sound principles of effective teaching. By reliable, we mean that independent analysts would reach similar conclusions and cite similar evidence for them. Unfortunately, no procedure existed for judging whether or how well curriculum materials matched specific learning goals like benchmarks. We found that, by using only the impressionistic or check-list procedures that are common in curriculum evaluation, neither other people’s judgments nor our own yielded consistent or defensible results. We therefore began a major effort to develop an adequately valid and reliable procedure, and we have made considerable progress.

Premises

The central proposition in the Project 2061 procedure for analyzing curriculum materials is that they are to be judged primarily in terms of how well they are likely to contribute to the attainment of specific learning goals. While it is certainly important that materials are scientifically accurate, age appropriate, and motivating, if they do not also contribute significantly to students’ learning important and agreed upon ideas and skills, they are not suitable for adoption. Hence the Project 2061 procedure concentrates on examining curriculum materials in the light of a coherent set of learning goals. The particular learning goals used here are those found in Benchmarks and in the content component of National Science Education Standards, but in principle any well thought out goal statements could be used as long as they are learning goals and as long as they are specific.

A second premise of the Project 2061 approach to the analysis of curriculum materials is that account has to be taken of both the content and instructional properties of the materials under examination. It does little good for materials to simply include the content of specific learning goals if the instructional strategies recommended in the material are not consistent with what is known about how students learn. To bring home this point, consider that Benchmarks for Science Literacy is a perfect content match to benchmarks—it contains explicit and specific material that is relevant to all benchmarks and does not contain any material that goes beyond them—yet no one would argue that Benchmarks should be used as a student textbook. Mere statement of the goals is far from enough to help students to achieve them. For materials to be credible, it must be evident how students could actually learn what is intended from them.

The Procedure in Brief

The analysis of curriculum materials described here involves four steps, described briefly here and in greater detail in Appendix A:

We have found this four-step procedure is helpful to those doing the analysis for the first time. More experienced users of the procedure tend to combine some of the steps. For example, as knowledge of benchmarks and standards increases, the preliminary examination may be combined with the content analysis.

Features

There are several important features of the 2061 procedure that are key to its design. The first four are strategic elements of the procedure itself, the last four characterize the tools available to users. Taken together, these features distinguish the 2061 analysis procedure from other evaluation forms.

1. Specific Learning Goals. Both Benchmarks and NSES specify what students should know or be able to do at a fairly fine grain size. The 2061 analysis procedure examines the alignment of materials to specific learning goals—benchmarks and fundamental understandings—rather than to section heading or general topics. For example, within the section "The Earth," Benchmarks specifies precisely what students should know about the water cycle by the end of grades 2, 5, 8, and 12. By the end of grade 2 students should know that "Water left in an open container disappears, but water in a closed container does not disappear." By the end of grade 5 students should know the (more sophisticated) idea that "When liquid water disappears it turns into a gas…" and by the end of grade 8 students should be able to explain evaporation in terms of molecules. A first grade activity that has students comparing yearly rainfall patterns around the world addresses the topic "water cycle" but not the substance of the K-2 benchmark "Water left in a closed container disappears, but water in an open container does not disappear." But having students observe the water loss in their classroom fish tank after a holiday and investigating whether covering it solves the problem is more to the point. Benchmarks and NSES provide a set of developmentally appropriate goals based on learning research, making it unnecessary for reviewers to intuit what is appropriate for students in different grades.
   
2. Instruction Tied to Learning Goals. The 2061 procedure examines how well the instructional and assessment strategies in the material contribute to students’ learning the specific benchmarks. For example, criteria that probe whether or not the material "includes activities that provide first-hand experiences with phenomena" and "includes question sequences to guide student interpretation and reasoning about phenomena" ask evaluators to examine whether the material contains activities and calls for reflection on each benchmark to be learned. Similarly, the assessment tasks are examined for their match to specific benchmarks. It is important to note that in looking at many materials in terms of many different benchmarks, we have seen that a material may treat one benchmark quite well and another quite poorly. That makes it difficult to form good global judgments. It’s possible, of course, to consider whether a material employs generally supportive instructional strategies such as hands-on activities and cooperative groups, independent of what learning goals they may contribute to. But the instructional strategies might turn out to be aimed at only some of the learning goals of interest—or none at all. (This is the most central proposition of our work: The desirability of instruction and content cannot be considered separately!)
   
3. Evidence-based Arguments. The 2061 analysis procedure requires that judgments are supported by evidence-based arguments and take account of both the quantity and the quality of the evidence. The 2061 criteria are not used as a check list, nor can analysts get by with giving unsupported opinions. Analysis reports include descriptions of the supporting evidence and page number references—benchmark by benchmark—so that readers can check them and form independent judgments (and judge the credibility of the reviewer). In coming to conclusions about the quality of evidence, a reviewer might argue that a material that includes carefully sequenced questions to prompt student reflection on a particular phenomenon is deserving of a higher rating than a material that simply instructs teachers to "encourage students to discuss their ideas about the motion of molecules." We have found that when reviewers are required to justify their conclusions, the quality and reliability of their judgments improves.
   
4. Feedback. The 2061 procedure involves teams of educators who examine and comment on one another’s judgments and supporting evidence. For example, one team member may question whether activities in a curriculum material that claimed to match a particular benchmark really do. Another may question whether the supporting evidence included in a report truly responds to a criterion. Holding judgments and evidence up to such scrutiny improves the quality of the analysis reports, and having successfully defended their claims builds reviewers’ confidence in their judgments. Team members have indicated both how much they value good feedback when it is given and miss it when it is not.
   
5. Clarification of Learning Goals. The meaning of any learning goal, however specific and clearly written it may be, is nonetheless subject to the interpretation of users. We have observed that simply reading a benchmark is often insufficient to grasp its intent, because users typically couch it in their own idiosyncratic understanding. This causes them to both overestimate and underestimate what a benchmark is expecting students to know or be able to do (Roseman, 1997). Before attempting to match activities in a curriculum material to the content of any benchmark, review teams study Science for All Americans and Benchmarks for Science Literacy to clarify the benchmark’s meaning. Science for All Americans provides a narrative context that clarifies where the benchmark is aiming. Other benchmarks before or after a benchmark in a K-12 list clarify the level of sophistication intended in the one under consideration. A strand map from Benchmarks on Disk provides still more context in terms of what goes into understanding a benchmark and where it will lead. Benchmarks essays describe difficulties students may have with the benchmark topic and offer some suggestions for helping students achieve the benchmark. Research summaries suggest likely limitations in student understanding of the benchmark and provide rationale for its grade-level placement.
   
6. Specific Criteria. The 2061 procedure uses highly specific analysis criteria. For example, materials are examined for how well they alert teachers to commonly held student ideas (both troublesome and helpful) such as those described in Benchmarks Chapter 15: The Research Base and then for how well the material explicitly addresses those commonly held student ideas. In contrast, other procedures ask for more general impressions: For example, "Do the materials reflect current knowledge about effective teaching and learning practices based on research related to science education?" The more general the criterion, the more open it is to varied interpretations and sampling variations and hence the less likely it is that different analysts will reliably interpret it and respond accordingly.
   
7. Clarification of Criteria. To help users interpret the criteria, the 2061 analysis procedure gives the rationale for the analysis criteria and elaborates what a response to each should include. For example, the question "Does the material alert teachers to commonly held student ideas?" is clarified with the following paragraphs:

   Students usually have ideas about how the world works even before instruction. Some ideas are intuitions that are in basic agreement with the scientists’ views, others (labeled often as misconceptions) are in disagreement/conflict with currently accepted scientific theories. Some of the students' misconceptions work fairly well in familiar contexts and are highly resistant to change. Knowing the ideas that students typically have helps teachers decide what ideas to build on and what changes to promote…

   Responding to this question involves examining a) whether there is research on commonly held student ideas in the topic area(s) that the material addresses, b) whether the material alerts teachers to such ideas, c) whether the material accurately represents research findings, and d) what proportion of commonly held ideas identified by the research are described in the material. Summaries of research on students ideas in science (such as those included in Benchmarks Chapter 15: The Research Base or Making Sense of Secondary Science: Research Into Children’s Ideas, by Rosalind Driver, Ann Squires, and Valerie Wood-Robinson. New York: Routledge, 1994) will be helpful to reviewers who want to know what ideas students typically have about the topics that the curriculum material they are examining addresses. If there is no research on student ideas in the topic area(s) that the material addresses, the material should not be faulted for not addressing this criterion.

8. Concrete Examples of Applying Criteria. The 2061 analysis procedure employs examples of applying the criteria to specific goals and specific materials. Case study reports of fully analyzed materials are provided to illustrate the use of the criteria, what a good argument consists of, and what evidence justifies each rating—high, medium, and low. To illustrate a "high" score on the criterion "Reflecting on Activities" the following example is used. The material in question, Matter and Molecules (Berkheimer, et. al, 1988), is attempting to teach the idea that "All matter is composed of molecules that are constantly moving." During an activity in which students place hard candy in both hot and cold water, they are asked to make some predictions:

As was evident in comparing our first cycle to our second cycle of development (described below), including all of the features described above greatly increases the validity and reliability of the analysis procedure. Furthermore, because of the specificity and detail provided in the analysis reports, results of the 2061 procedure can facilitate attempts to better align curriculum materials with benchmarks and standards.

Developing and Testing the Procedure

In developing and trying out the analysis procedure, Project 2061 has involved over 100 K-12 teachers, teacher educators, materials developers, cognitive researchers, and scientists. In two cycles of materials evaluation 3-person teams used the procedure to analyze a few materials, reported their findings, and suggested modifications to the procedure. In each cycle, the reliability of the analysis reports was tested by having two teams analyze each curriculum material independently. A similar strategy was used at 6 national sites—both statewide and school-district—to test the procedure under field conditions. A list of materials examined is provided in Appendix B.

We have learned that the procedure can be reliably used only if the following conditions are met:

• Review teams have more than a superficial understanding of the content to be learned.
• Review teams understand what constitutes an appropriate level of sophistication of that content for K-12 students. (It is not enough that analysts are familiar with the content, they should also know what of that content contributes to literacy at various grades—for Benchmarks this has been spelled out for K-2, 3-5, 6-8, and 9-12; for Standards, for K-4, 5-8, and 9-12.)
• Review teams are knowledgeable about reported difficulties that students have learning that content. (Benchmarks Chapter 15: The Research Base summarizes research on students’ ideas that contributed to the substance and grade-level placement of benchmarks. The new Project 2061 tool Resources for Science Literacy (AAAS, 1997) supplements Benchmarks with descriptions of over 100 articles, reports, books, and videos that summarize difficulties students have with Benchmarks ideas)
• Review teams have undergone considerable training in the use of the analysis procedure that includes practice with feedback. (A minimum of 4 days is needed to teach the procedure but this needs to be followed by practice with feedback to ensure learning.)
• The materials to be analyzed are not too dissimilar in type of content from the materials used as examples in training. (Case study materials illustrate the application of the analysis criteria to benchmarks in commonly taught topics in life, earth, and physical science. Analysts have difficulties transferring the procedure to benchmarks less commonly taught—for example, to benchmarks in the nature of science or common themes. We are planning to develop additional case studies to illustrate how the procedure applies to less familiar topics.)

Reviewers were enthusiastic about their involvement in the analysis work. Over 80% indicated on a follow-up questionnaire that they are interested in reviewing other materials according to the 2061 procedure.

Results

Results of using the procedure are often surprising. Analysis teams find that quick judgments about alignment to benchmarks or content standards are frequently contradicted by a more rigorous analysis. This held for single units or across several units in a program. A superficial examination most often overestimates what a material can be expected to accomplish. For example, when educators were asked how well River Cutters (a grade 6-9 curriculum module) addressed benchmarks, their initial judgments were far more optimistic than their judgments after completing the 2061 analysis. After initially listing 22 benchmarks that a cursory read led them to suspect were addressed in River Cutters, they found actual sightings for only 12 of them. After studying the meaning of the benchmarks carefully and revisiting the sightings with this more sophisticated understanding, they found that only 6 had a respectable content match. And on considering the instructional strategy of the material, only 1 was found to be instructionally well-supported, as shown below.

These results are meant less as a criticism of River Cutters (which could certainly be modified to address more benchmarks) than to illustrate how one can be easily mislead by a superficial analysis. A similar pattern is obtained from analysis of a variety of K-12 curriculum materials. To the extent that other evaluation procedures’ and developers’ claims about their own materials are made at the suspected or at the "sightings" level, their reports will not be credible.

The good news is that, through the use of the procedure, we have identified highly credible materials that are likely to support students’ learning of benchmarks and standards. For example, stand-alone units developed over a period of years by the Institute for Research on Teaching and by the Michigan Department of Education are well-aligned in terms of content and instruction. These units, though not commercially polished, are readily available at relatively low cost and could be used as is by those eager to get started. And some selected units within elementary, middle, and high-school courses can be used as is or readily modified to be better aligned with national learning goals. Even a small module like River Cutters could contribute to students’ understanding of the utility of models by, for example, including question sequences to guide student interpretation of their river runs and their reasoning about the usefulness of their river cutters in understanding how real rivers shape the earth.

Because of the effort required by the procedure, it would be helpful if large-scale curriculum materials could be evaluated by means of sampling a few typical units. Unfortunately, we have found some year-long and grade-range materials to be quite uneven in their treatment of benchmarks and standards. Hence, the analysis results from single units, whether favorable or unfavorable, cannot be generalized to whole programs. Project 2061 is currently exploring sampling techniques for grade-range curriculum materials that do not compromise the reliability and validity of the analysis procedure and that still yield a fairly accurate picture of the material. In addition, we are identifying content and instructional analysis criteria that are especially important for the analysis of programs.

An important consequence of involving a variety of educators in the development and testing of the 2061 analysis procedure is the creation of a pool of reformers who understand what alignment involves and who can bring that knowledge to bear on their work. A cadre of K-12 educators now exists with increased knowledge of specific learning goals in Benchmarks and NSES and with skills to evaluate curriculum materials for their fit to these goals. Moreover, teacher educators are adapting the procedure for use in preservice and professional development programs, which will produce a larger cadre still.

Additional Considerations

Empirical Verification.

It is important to note that the 2061 analysis procedure, however meticulous, produces judgments of the likelihood of effectiveness. The potential of curriculum activities can be estimated by examining whether they address specific, important learning goals (content match), and whether they are based on effective principles of teaching and learning (instructional match) for these goals. However, the "implemented" curriculum will depend on teachers’ interpretations and use of the materials, and the "achieved" curriculum will depend on individual students’ skills, interests, and prior knowledge. Until activities are tried out with students, there will not be hard evidence for what is actually learned. Yet sound studies of effects on student learning are expensive and difficult to do. Such data are available for only a handful of materials and, where available, they correspond well to the results of the 2061 analysis.

Cost and support.

As noted earlier, the Project 2061 procedure concentrates on only some aspects of materials analysis: their alignment with specified learning goals and learning psychology. There are other important variables—such as affordability and availability of publisher support—that influence the usefulness of materials and whether or not they will even be used at all. Other evaluation procedures currently in use focus on these other variables.

Uses of the Project 2061 Procedure

The discussion to this point might seem to suggest that the only purpose of the Project 2061 analysis procedure is to improve decisions about the selection of curriculum materials. That might indeed be its initial use. But the procedure has other important uses, including:

• Identifying shortcomings in existing curriculum materials and suggesting ways to improve them.

• Increasing teachers’ knowledge of characteristics of well-aligned materials and developing skills to distinguish materials that are well-aligned from those that are not.

• Stimulating the development of and the market for well-aligned materials.

But while it may be important for every teacher to have some experiences evaluating materials, to have every teacher evaluating every material is not an effective use of their time or the nation’s dollars. We recommend three levels of involvement:

The first attends to immediate practicality. Some choices of materials have to be made now in order to sustain the momentum of the systemic reform movement. In the short run, the evaluation procedure we have researched is too demanding for processing all of the materials that would fairly have to be included in a resource pool. Yet people who are necessarily going to be using a simpler procedure should be aware of what a more thorough evaluation would be like and what sort of results it produces. That raised sensitivity should improve the quality of simpler judgments that are made. A couple of days of tutored practice—soon—would be enough for this beginning.

The second level looks ahead over the next couple of years. The number of qualified evaluators has to grow enough to mount a more searching analysis and build a base of reviews of well evaluated materials. A fairly large number of practitioners should be trained over a period of a week or so and brought back together periodically to share conclusions and assess reliability. A number of these more experienced evaluators should be curriculum developers, so that they can plan new projects (and revise old ones) that are aligned with standards from the start.

The third level of involvement continues to improve the method itself, preparing the way for more efficient evaluation of more diverse materials in the future. This R&D will also provide updates to the second-layer work that can improve and simplify it along the way. Since we believe that Project 2061 (thanks to foresighted NSF funding) is well ahead on this, we see ourselves as playing a leading role in that work, and we would welcome collaborators.

A Possible Approach: The Philadelphia Story

The Philadelphia School District had already committed itself to teaching toward specific learning goals in Benchmarks and NSES and was searching for suitable materials. Faced with the need to select materials fairly quickly, it is using a strategy that combines the selection, fixing-up, and professional development benefits of the 2061 analysis procedure.

Selecting materials.

A curriculum review committee (about a dozen K-12 teachers and teacher educators) was trained to use the 4-step analysis procedure, used it to produce draft analysis reports, compared their reports to others on the same material, and attempted to reconcile, or at least account for, differences. This group will review existing materials and recommend promising alternatives for district use. Due to time constraints, their recommendations will be based on results from only the first step in the 2061 procedure—eliminating materials that do not appear to focus a significant amount of instruction on specific learning goals. Although they acknowledge the desirability of subjecting materials to both a content and instructional analysis before using them with students, these more rigorous steps in the analysis procedure will be postponed and used over time on the greatly reduced list that survives the preliminary inspection. The committee will rely on members’ first-hand experience with the more rigorous analyses to inform their preliminary judgments. Efforts will focus on the materials already at least partially examined and found promising by 2061 analysis teams.

Fixing-up.

To use the 2061 procedure to improve existing materials, teachers will (a) undertake the more rigorous analysis of the content and instructional match of the chosen materials to benchmarks and (b) use the results of the analysis as a basis for modifying the materials to better align them. This could include such remedies as developing questions to focus students reflection on benchmark ideas, adding activities to address reported student learning difficulties, providing evidence-based arguments to foster student generalization of concepts, and/or explicitly demonstrating how benchmark ideas are useful for making sense of the students’ world outside the classroom.

Professional development.

At the same time, a larger and more diverse group of educators is becoming knowledgeable, through a series of workshops, about specific learning goals in Benchmarks and NSES and about the analysis criteria used to judge materials in light of these goals. This knowledge will help them to recognize both the strengths and weaknesses in existing curriculum materials in terms of their treatment of specific learning goals -- important because even relatively good materials still have some distance to go before they are well aligned. As new, better aligned, materials become available the District will have a cadre of informed consumers who can recognize and appreciate them.

Professional development of teachers is being extensively supported by USI, Eisenhower, and other funds. To help improve curriculum and instruction in both undergraduate and graduate college courses, the District will encourage the college faculty themselves to develop knowledge of science literacy as defined by Science for All Americans, to recognize the importance of specific learning goals, and to use them in designing professional development for Philadelphia teachers. In this way, both teachers pursuing graduate degrees and new teachers will have significant blocks of time focused on science literacy.

References Cited

American Association for the Advancement of Science. (1997). Resources for Science Literacy: Professional development. New York, NY: Oxford University Press.

American Association for the Advancement of Science. (1993). Benchmarks for Science Literacy. New York, NY: Oxford University Press.

American Association for the Advancement of Science. (1989). Science for All Americans. New York, NY: Oxford University Press.

Berkheimer, G.D., Anderson, C.W., Lee, O., and Blakeslee, T.D. with Eichinger, D. and Sands, K. (1988). Matter and Molecules: Teachers Guide: Science Book and Activity Book. East Lansing, MI: The Institute for Research on Teaching, College of Education, Michigan State University. Occasional paper No. 121.

Driver, R., Squires, A., and Wood-Robinson, V. (1994). Making sense of secondary science: Research into children’s ideas. New York, NY: Routledge.

National Research Council (1996). National Science Education Standards. Washington, DC: National Academy Press.

Roseman, J. (1997). Implementing benchmarks and standards: Lessons from Project 2061. The Science Teacher 64 (1), pp. 26-29.

Appendix A: A More Detailed Look at the Procedure

Preliminary Inspection.

Let’s assume for the moment that we are starting with materials that appear promising—the content doesn’t appear too far outside the scope of science literacy and the material includes lots of hands-on activities. The task becomes listing some specific learning goals on which the material appears to focus.

First, reviewers search fairly quickly through the material (both student material and teachers guides) to make a preliminary list of all the specific learning goals that would seem likely to be targeted. The material is then examined more carefully to locate and record all places where each learning goal is actually served—e.g., particular readings, experiments, discussion questions. (A sighting must be explicit in the material.) Then, based on the number and types of sightings, a decision is made about which benchmarks and standards warrant a more careful analysis.

Content Analysis.

This analysis is a more rigorous examination of the link between the subject material and the selected learning goals. This involves giving precise attention to both ends of the match – the precise meaning of the benchmark on one end, and the precise intention of the material on the other. The material is examined with respect to such questions as:

For the material as a whole an attempt is made to estimate the degree of overlap between its content and the learning goals of interest. Thus it strives to answer questions such as these:

Instructional Analysis.

The purpose of the instructional analysis is to estimate how well the material addresses targeted benchmarks from the perspective of what is known about student learning and effective teaching. The criteria for making such judgments are derived from research on learning and teaching and on the craft knowledge of experienced educators. In the context of science literacy, summaries of these have been formulated in CHAPTER 13: Effective Learning and Teaching in Science for All Americans, in Chapter 15: The Research Base of Benchmarks for Science Literacy, and of science education alone in CHAPTER 3: Science Teaching Standards in National Science Education Standards.

From those sources, seven criteria clusters have been identified to serve as a basis for the instructional analysis. (One could view these as standards for instructional materials.) A draft of the specific questions within each cluster is shown below. The proposition here is that (1) in the ideal all questions within each cluster would be well addressed in a material – they are not alternatives; and (2) this analysis has to be made for each benchmark separately – if we are serious about having science literate high school graduates then we want to focus effective instruction on every single one of the important ideas in Science for All Americans.

Summary Report.

Having analyzed both the content and the instruction aimed at that content, the final step in the process is to prepare a report that summarizes the material’s treatment of specific benchmarks and, drawing on that evidence, comments more generally on strengths and weaknesses of the material. Nonetheless, the report stops short of an overall recommendation. It is for educators to decide whether to use the material as is, to use it with modifications, or not use it at all. A goal-centered analysis of the kind developed by Project 2061 should help them make better decisions than would otherwise be possible.

Appendix B: Curriculum Materials Analyzed

K-5 units

Insights: Changes of State (EDC)

ESS: Where is the Moon? (EDC)

Science and Technology for Children: Food Chemistry (NSRC)

Nuffield Primary Science: Earth & Space (Collins Educational)

Nuffield Primary Science: Living Things in Their Environment (Collins Educational)

FOSS: Models & Designs (LHS, Britanica)

FOSS: The Structures of Life (LHS, Britanica)

Used Numbers: The Shape of the Data (TERC)

Grades 6-8 Units

Changes in Matter (Macmillan)

Food, Energy, and Growth (Michigan Department of Education)

GEMS: River Cutters (LHS)

Matter and Molecules (Institute for Research on Teaching, MSU)

Power Plant (Institute for Research on Teaching, MSU)

Science 2000 (D.C. Heath)

Science Focus: The Salters’ Approach: Drinks (Heinemann)

SciencePlus: Life Processes (Holt, Rinehart, Winston)

SEPUP: Issues, Evidence and You (LHS)

Technology Units

Materials World Modules: Composites (Northwestern University)

Nuffield Design and Technology (Collins Educational)

Introduction To Design & Technology: Control Technology Systems (Todd et. al., Taylor)

TSM Integration Project: Cabin Insulation (LaPorte and Sanders, Glencoe)

Middle School Program

SciencePlus (Holt, Rinehart, Winston)

Grades 9-12 Biology Units

Insights in Biology: The Matter of Life (EDC)

Biological Science: A Human Approach: Evolution (BSCS, Kendall Hunt)

Biological Science: An Ecological Approach (BSCS, Kendall Hunt)

Heath Biology: Unit II (D.C. Heath)

Grades 9-12 Chemistry Units

ChemCom: Conserving Chemical Resources (American Chemical Society, Kendall Hunt)

Chemistry that Applies (Michigan Department of Education)

Salters Chemistry: Burning & Bonding

Visualizing Matter: Atomic Structure (Holt)

Grades 9-12 Physics Units

Active Physics: Predictions (AAPT)

Project Star: Chapters 10-15 (Coyle et. al., Kendall Hunt)

Conceptual Physics: Chapters 2-6 (Hewitt, Addison-Wesley)

Roseman, J. E., Kesidou, S., and L. Stern (1997). Identifying Curriculum Materials for Science Literacy. A Project 2061 Evaluation Tool. Based on a paper prepared for the colloquium "Using the National Science Education Standards to Guide the Evaluation, Selection, and Adaptation of Instructional Materials." National Research Council, November 10-12, 1996.