12 HABITS OF MIND
12E) CRITICAL-RESPONSE SKILLS
Control of variables. Upper elementary-school students can reject a proposed experimental test where a factor whose effect is intuitively obvious is uncontrolled, at the level of "that's not fair" (Shayer & Adey, 1981). "Fairness" develops as an intuitive principle as early as 7 to 8 years of age and provides a sound basis for understanding experimental design. This intuition does not, however, develop spontaneously into a clear, generally applicable procedure for planning experiments (Wollman, 1977a, 1977b; Wollman & Lawson, 1977). Although young children have a sense of what it means to run a fair test, they frequently cannot identify all of the important variables, and they are more likely to control those variables that they believe will affect the result. Accordingly, student familiarity with the topic of the given experiment influences the likelihood that they will control variables (Linn & Swiney, 1981; Linn, et al., 1983). After specially designed instruction, students in 8th grade are able to call attention to inadequate data resulting from lack of controls (see for example Rowell & Dawson, 1984; Ross, 1988).
Theory and evidence. Middle-school students tend to invoke personal experiences as evidence to justify a particular hypothesis. They seem to think of evidence as selected from what is already known or from personal experience or second-hand sources, not as information produced by experiment (Roseberry et al., 1992). Most 6th-graders can judge whether evidence is related to a theory, although they do not always evaluate this evidence correctly (Kuhn et al., 1988). When asked to use evidence to judge a theory, students of all ages may make only theory-based responses with no reference made to the presented evidence. Sometimes this appears to be because the available evidence conflicts with the students' beliefs (Kuhn et al., 1988).
Interpretation of data. Students of all ages show a tendency to uncritically infer cause from correlation (Kuhn et al., 1988). Some students think even a single co-occurrence of antecedent and outcome is always sufficient to infer causality. Rarely do middle-school students realize the indeterminacy of single instances, although high-school students may readily realize it. Despite that, as covariant data accumulate, even high-school students will infer a causal relation based on correlations. Further, students of all ages will make a causal inference even when no variation occurs in one of the variables. For example, if students are told that light-colored balls are used successfully in a game, they seem willing to infer that the color of the balls will make some difference in the outcome even without any evidence about dark-colored balls (Kuhn et al., 1988).
Faced with no correlation of antecedent and outcome, 6th-graders only rarely conclude that the variable has no effect on the outcome. Ninth-graders draw such conclusions more often. A basic problem appears to be understanding the distinction between a variable making no difference and a variable that is correlated with the outcome in the opposite way than the students initially conceived (Kuhn et al., 1988).
Inadequacies in arguments. Most high-school students will accept arguments based on inadequate sample size, accept causality from contiguous events, and accept conclusions based on statistically insignificant differences (Jungwirth & Dreyfus, 1990, 1992; Jungwirth, 1987). More students can recognize these inadequacies in arguments after prompting (for example, after being told that the conclusions drawn from the data were invalid and asked to state why) (Jungwirth & Dreyfus, 1992; Jungwirth, 1987).