Footnotes for Criteria Used in Evaluating the Programs' Quality of Instructional Support

1. The purpose may be presented to students in the student book or in the teacher’s guide in the form of instructions to the teacher. If multiple purposes are presented to students, the reviewer should evaluate and rate each one separately and average the scores. If the presence of different purposes is likely to be confusing to students, then the reviewer may lower the average rating by one rating level.

2. If a material frames sections within a unit (rather than the whole unit), then the reviewer should evaluate and rate the purpose for each section separately and take the average. If a material attempts to frame both the unit overall and large sections within the unit, the reviewer should evaluate and rate each purpose separately. The overall rating of the criterion will be the average of the rating of the purpose for the unit overall and the average rating of the purposes provided for the individual sections.

3. For each set of key ideas, reviewers should use a list of the commonly held student ideas that have been published in the cognitive research literature.

4. Sufficiency should take into account key ideas, student misconceptions, and contexts in which ideas are probed.

5. Please note that high-quality activities that address student ideas need not have all of these characteristics.

6. Reviewers should note that the link between a phenomenon and an idea can be made either in the student book or in the teacher’s guide (e.g., through suggested discussions or through suggested answers to questions in the student book).

7. Phenomena that are briefly described in the material still count as evidence for this criterion, as long as their link to the idea is made explicit. The second criterion under category III, “Providing vivid experiences,” examines whether the descriptions are sufficient to provide students with a vicarious sense of the phenomena. In addition, to respond to this criterion, reviewers do not consider how much time is allotted for the discussion of the phenomena—this is examined in Category V: Promoting Students' Thinking about Phenomena, Experiences, and Knowledge.

8. Reviewers should examine end-of-chapter and/or end-of-unit tests. If developers indicate that review questions in the student book may be used for assessment purposes, then reviewers need to analyze both these tests and the end-of-chapter and/or end-of-unit tests included in the teacher guide, rate each type separately, and then average the ratings to obtain the final score.

9. To judge whether there is a sufficient number of assessment items, reviewers need to consider whether key ideas are adequately assessed. However, reviewers are not expected to evaluate and rate each idea separately and average the scores.