Method of Comparison

METHOD OF COMPARISON

The comparison of the NRC and Project 2061 content recommendations was conducted on two levels. First, there was an examination of how their common territory is organized -- the eight NSES content standards in relation to the twelve Benchmarks chapters. Second, there was an intensive content analysis at the specific level of detail expressed in the NSES "fundamental concepts (or abilities)" and the Project 2061 "benchmarks."

Organization

At the organizational level, the task is not as straight-forward as it might seem at first glance. First, NSES divides K-12 into three bands: K-4, 5-8, and 9-12. Project 2061 uses four bands: K-2, 3-5, 6-8, and 9-12. There are arguments for each, but the different divisions complicate comparisons.

Second, NSES is organized by grade level whereas Benchmarks is organized by content. Thus NSES has separate sections for grade levels K-4, 5-8, and 9-12, with seven content areas ("Standards," e.g., Life Science) cycling through them. The seven areas are somewhat different in each grade level. (One exception is the Unifying Concepts and Processes standard, which is designated as K-12 and proposes a fixed set of topics to be treated at every grade level.) Benchmarks takes the opposite approach. It has 12 chapters of content, with grade levels K-2, 3-5, 6-8, and 9-12 cycling through each chapter, so that the same content areas are treated at each grade level. Table 2 illustrates the difference for one content area. NSES’ primary organization by grade levels might facilitate looking for connections across content areas within the same grade-range, whereas Benchmarks' primary organization by content area facilitates looking for the development of ideas over time.

Third, there is no simple match of NRC’s content standards to Benchmarks content chapters. Figure 2 depicts the main correspondences between the major divisions for NSES content standards and for Benchmarks. It can be seen that the content in the NSES Life Science standard, for example, is divided between two Benchmarks chapters, The Living Environment and The Human Organism. On the other hand, the ideas in the two NSES standards Physical Science and Earth & Space Science are found in a single Benchmarks chapter, The Physical Setting. There is little intrinsic importance to the sequence of content divisions in either document.

Judgments about this "correspondence" were not made merely by titles of standards or chapters. It was necessary to look at least at the next more specific level of content -- sections within each chapter of SFAA/Benchmarks, and what we will call topics within each NSES standard. (NSES has no consistent name for these.) Figure 3 compares NSES standard topics and Benchmarks sections for the NSES Life Science standard and the two chapters of SFAA/Benchmarks that correspond to it. Lines are drawn in Figure 3 to show the number of SFAA/Benchmarks sections that include content relevant to the single standard topic The Cell. Claiming this correspondence of course required looking inside the standard topic and the SFAA/Benchmarks section at the next lower level of organization -- the "benchmark" for Benchmarks and the "fundamental concept" for NSES. The correspondence at the section level is listed in Table 1.

NSES organization levels

Benchmarks organization levels

STANDARD (LIFE SCIENCE)

CHAPTER (LIVING ENVIRONMENT)

(Standard topic) (The Cell)

Section (Cells)

fundamental concept ("Cells have particular structures...")

benchmark ("Every cell is covered by...")

Specific Content

At the specific level, an attempt was made to match every "fundamental concept" (or sometimes "fundamental ability") in the NSES content standards with the most relevant benchmark or set of benchmarks from all chapters of Benchmarks for Science Literacy. In the reverse comparison, an attempt was made to match each benchmark with the most relevant "fundamental concepts" and/or "fundamental abilities" in NSES. Not every benchmark was included, since NSES does not purport to address content for mathematics, social science, and technology in a systematic way. Some topics related to these areas do appear in NSES, however. In such instances, corresponding benchmarks from The Nature of Mathematics, The Nature of Technology, Human Society, The Designed World, and the Mathematical World chapters were included in the match.

Correspondence was sought only between the itemized content sections of both NSES and Benchmarks. No account was made of the NSES s vignettes, essays, and example assessments or to Benchmarks essays, although comments about content are sometimes found in those sections. Occasionally reference is made to Benchmarks' parent report, Science for All Americans (SFAA), which presents science literacy goals for high school graduates in a coherent narrative form, rather than broken down by grade ranges as in Benchmarks. The organization of content in Benchmarks is the same as in SFAA.

In matching to Standards, the initial search for benchmarks was made within the same grade range as the NSES item. The search was extended to benchmarks in different grade ranges (typically one grade range above and one below) only when (a) a satisfactory correspondence with benchmarks from the same grade range could not be found, or (b) when the intended level of sophistication of the NSES item was not clear and so might be interpreted as corresponding to a benchmark at a higher or lower grade level. In matching to Benchmarks, a similar procedure was followed in identifying NSES items that correspond to each benchmark. As a consequence of these limited searches, the specific comparison of Benchmarks to NSES is not simply the reverse of the comparison of NSES to Benchmarks. For example, for this K-2 benchmark:

Chapter 5 The Living Environment Grades K-2
Some animals and plants are alike in the way they look and in the things they do, and others are very different from one another. the earliest matching fundamental concept is in grades 5-8: Content Standard C Life Science Grades 5-8
Millions of species of animals, plants, and micro-organisms are alive today. Although different species might look dissimilar, the unity among organisms becomes apparent from an analysis of internal structures, the similarity of their chemical processes, and the evidence of common ancestry. However, in the reverse comparison, this NSES fundamental understanding is more closely matched by a 6-8 benchmark: Chapter 1 The Living Environment Grades 6-8
Similarities among organisms are found in internal anatomical features, which can be used to infer the degree of relatedness among organisms. In classifying organisms, biologists consider details of internal and external structures to be more important than behavior or general appearance. Therefore, the K-2 benchmark above -- a precursor to the 6-8 benchmark-- was not explicitly included as a match.

The specific comparison between Benchmarks and NSES was complicated by three factors (in addition to the general organizational differences already described):

Aggregation. Both the "fundamental concepts" and "benchmarks" themselves are usually compound statements containing several distinct ideas. The ideas are seldom aggregated in exactly the same way. Often, therefore, no simple one-to-one correspondence is found even at this level. The detailed comparisons of content provide myriad instances of a single "fundamental concept" comprising ideas that are found in many, sometimes far-flung, "benchmarks" -- and, less often, vice-versa.

NSES’ Treatment of Unifying Concepts and Processes. A special complication occurs in the NSES treatment of the Unifying Concepts and Processes content standard, which a) includes fairly general prose instead of bulleted items, and b) does not divide its recommendations into grade level clusters. As a result, Benchmarks matches to this standard could be made only to the broad topic sections underlying the standard rather than to specific paragraphs that describe the fundamental concept, and NSES matches to Benchmarks Chapter 11 Common Themes were made only to benchmarks at the 6-8 and 9-12 grade level. When appropriate, reference was made to precursor ideas in earlier grades that appear in Benchmarks.

NSES’ Treatment of Scientific Inquiry and Technological Design. In searching for correspondences to NSES "fundamental abilities" of scientific inquiry and technological design, benchmarks that specify "what students should know" were included in addition to benchmarks that specify "what students should be able to do." One reason for this was some ambiguity about whether the NRC includes abilities of scientific inquiry within the content standards with the intention that all science-literate adults will subsequently be able to perform scientific investigations, or because hands-on experiences in inquiry are a necessary means to understand the nature of scientific inquiry. A second reason is that NSES fundamental abilities are themselves sometimes mixed statements of "what students should know" and "what they should be able to do." For example, consider the following fundamental ability of scientific inquiry:

Use mathematics in all aspects of scientific inquiry. Mathematics is essential to asking and answering questions about the natural world. Mathematics can be used to ask questions; to gather, organize and present data; and to structure convincing explanations (Content Standard A Science as Inquiry, Grades 5-8). A third reason is that, in many cases, "what students know" accompanies, or is even a prerequisite to "what they are able to do."

In some cases, the NSES fundamental-concept and fundamental-ability statements related to Scientific Inquiry and Technological Design were fairly broad, allowing potential matches to be made to a variety of benchmarks. For example, the NSES fundamental concept on designing and conducting scientific investigations (Content Standard A Science as Inquiry, Grades 5-8) states:

Design and conduct a scientific investigation. Students should develop general abilities, such as systematic observation, make accurate measurements, and identifying and

controlling variables. They should also develop the ability to clarify their ideas that are influencing and guiding the inquiry, and to understand how those ideas compare with current scientific knowledge. Students can learn to formulate questions, design investigations, execute investigations, interpret data, synthesize evidence into explanations, propose alternative explanations, and critique explanations and procedures.

Several specific benchmarks can be used to elaborate the more general NSES fundamental ability, in particular its statement on "critiquing explanations and procedures": Chapter 1 The Nature of Science Grades 3-5

Scientific investigations may take many different forms, including observing what things are like or what is happening somewhere, collecting specimens for analysis, and doing experiments. Investigations can focus on physical, biological, and social questions.

Chapter 1 The Nature of Science Grades 3-5

Recognize when comparisons might not be fair because some conditions are not kept the same.