Center for Curriculum Materials in Science

National Research Agenda

Draft, June 2005

The Center for Curriculum Materials in Science (CCMS) has outlined a national research agenda for science curriculum materials in five categories: curriculum materials for all children, teacher learning and educative materials, the curriculum development process, assessment, and policy. Selection of these five categories was motivated by our belief that while curriculum materials can play a strong role in influencing student and teacher learning, the development, selection, and use of materials operates within a system that is poorly understood. We recognize that separating the world artificially into discrete topics is bound to partition questions into somewhat arbitrary categories; however we do this for ease of organizing the key questions in the research agenda, recognizing that there is much overlap between the categories. What defines the categories is not so much what is left in and left out of the definition but which aspects of the interactions in these complex systems are brought to the foreground in the questions. So, for example, examining what teachers do to help students learn when working with materials appears primarily in the Curriculum Materials section, but examining how teachers learn to do that from formal professional development or informal interactions with their peers arises in the Teacher Learning section. Our current goal is to help identify and encourage research on the critical issues facing design, selection, and use of science curriculum materials. Our next step will be to summarize the research literature in each category.

The research agenda contains varied types of questions and methodologies that need to be employed to conduct this research. The questions range from investigations of teaching and learning as it exists now (descriptive and explanatory research questions) to more design-focused questions. For example, a descriptive or explanatory research question looking at teacher knowledge might ask “What pedagogical knowledge do teachers need to successfully support student inquiry?” A more design-focused question might ask “How can we help teachers develop pedagogical approaches for supporting student inquiry?” Both styles of research are needed in the field, and both are represented in the agenda that follows and in the research currently underway at CCMS partner institutions.

Curriculum Materials for All Children

This section focuses on research on the nature of the curriculum materials themselves and how they can influence successful interactions between learners and teachers in classroom enactment. The research also investigates student learning and classroom practice that has direct implications for the curriculum design process so that the structure of designed curriculum, the activities it contains, and the instructional approaches it utilizes will promote the development of deep science understanding in all children. Much work has been done that informs the construction of this knowledge base, such as research into naïve conceptions, scaffolds, and learning technologies; however, even more remains to be done, especially in determining how these findings relate to curriculum development.

Support of Cognitive and Social Interactions in Materials

1. How can curriculum materials take account of students’ prior epistemologies and conceptions and build upon them successfully?
2. How can curriculum materials scaffold learners in complex new practices such as inquiry, scientific argumentation, etc.?
3. How can curriculum materials help students learn new types of social interactions inherent in more ambitious science learning, such as scientific argumentation and discourse, peer critique, and presentations of scientific arguments?
4. How can curriculum materials situate learning in meaningful contexts while also helping students develop general knowledge about scientific phenomena?

Support of Motivation and Affect in Materials

1. How can curriculum materials motivate students to both engage and learn?
2. How do differently structured curricula influence students’ conceptions of science and themselves in relation to science? What types or characteristics of curricula give the students a sense of “empowerment”?

Diversity

1. What obstacles do students face in learning science (e.g., cognitive, attitudinal)? How do students differ in how prepared they are to meet these barriers? How do cultural backgrounds affect this? What other factors affect this? How should curriculum materials consider these issues?
2. What are the attributes of curriculum materials that attend to the learning needs of all students regardless of their cultural backgrounds and prior experiences? How can curriculum materials build on the diverse cultural experiences of students?
3. How can we apply what is known and extend our understanding about how to create materials and experiences that motivate students from a variety of cultural and academic backgrounds to both engage and learn?

Representation of Learning Goals and Activities in Curriculum

1. What are the developmental trajectories for major strands of specific learning goals?
2. What are the cognitive performances that lead to and rely on deeper conceptual understanding? What are the characteristics of curriculum materials that support and assess the development of these cognitive performances?
3. What are the characteristics of effective instructional sequences? What makes these effective?
4. What are the most effective uses of learning technologies in supporting student learning? How can learning-technologies be most effectively integrated in a curriculum?

Classroom Enactment

1. What are the features of curriculum materials that foster classroom processes that support and lead to effective science learning?
2. What kinds of classroom practices can support students in being reflective learners?
3. What form of scientific inquiry is possible in classrooms (i.e., what are the cognitive strategies, social interactions, and activities that comprise student inquiry)? How may these be represented in and supported by curriculum materials?

Teacher Support

1. What are the characteristics of curriculum materials that allow them to be flexibly tailored and adapted by teachers around the needs of their classrooms?
2. What are the characteristics of instructional and support materials that are effective in supporting teachers in implementing effective teaching approaches?

Teacher Learning and Educative Materials

This portion of the agenda deals with teacher learning as it relates to science curriculum materials. Questions examine teachers' professional knowledge and four main inputs to teacher learning: curriculum materials themselves, professional development, professional learning communities, and pre-service teacher education.

Teacher Professional Knowledge

1. What knowledge, beliefs, and abilities do teachers need to teach effectively for student achievement of challenging learning goals?
2. How can teacher support materials, professional development, professional learning communities, and pre-service education effectively and efficiently help teachers construct this set of knowledge, beliefs, and abilities?

Educative Materials

1. What qualities of teacher support materials relate to effective use of high quality curriculum materials when "effective use" is intended to include both appropriate use and high levels of student learning outcomes?
2. How do teachers interpret and use curriculum materials and how do patterns of use relate to teacher and student learning outcomes?
3. How can support materials help teachers learn from examples of practice? How can learning technologies (e.g., Web sites, video cases) support this learning?
4. To what extent can patterns of use and teacher and student learning be accounted for by characteristics of the materials; teachers’ prior knowledge, conceptions, and beliefs; students’ characteristics and performance; and the nature of the professional development and other support the teachers received?
5. How can support materials take account of the varied needs of teachers with different levels of knowledge and experience?
6. How can support materials help teachers adapt to the needs of diverse classes of students and particular student populations?

Professional Development

1. What qualities of professional development support appropriate use of high-quality materials?
2. What qualities of professional development support teachers in understanding and appropriate adaptation of curricular materials?
3. How can professional development both be situated in curricular enactment, to support teachers’ enactment of new curricula, and help teachers learn more science content and pedagogical approaches useful beyond the specific curriculum?
4. What qualities of professional development help teachers adapt high-quality materials to the needs of diverse classes of students and particular student populations?

Professional Learning Communities

1. What qualities of professional communities are supportive of effective implementation and use of high-quality curriculum materials?
2. How can these qualities be fostered in professional communities?
3. What qualities of professional communities help teachers adapt high-quality materials to the needs of diverse classes of students and particular student populations?
4. What qualities of professional communities support appropriate adaptation and higher levels of student learning with existing materials?

Pre-service Teacher Education

1. What qualities of pre-service education prepare teachers to (1) successfully implement high-quality materials, (2) discriminate, select, and advocate for high- quality curriculum materials, (3) adapt high-quality materials to the needs of diverse classes of students and particular student populations, and (4) adapt existing materials based on principled refinement of those materials?

The Curriculum Development Process

Curriculum development has generally been treated as a craft that must either be handed down from master directly to apprentice or must be re-discovered by each practitioner. Since development is largely treated as a second-class citizen to research, it has been essentially overlooked by educational researchers. Therefore, the curriculum development process is an under-researched area. One exception to this is the field of instructional design, where design processes have been the focus of research. However, the field of instructional design has had difficulty making the transition from behaviorist approaches of pedagogy to the constructivist and social constructivist approaches that better reflect current research and theory on how people learn.

Orchestrating the Design Process

1. Participants and roles: What participants (kinds of expertise) should be engaged in the curriculum development process, in what roles, and at what stages? Participants that are often called for include: content area experts, classroom teachers, educational researchers, students, parents and community members, technologists, and evaluation experts.
2. Collaboration in teams with distributed expertise: How can design teams be organized to enable effective collaboration between members with expertise from different backgrounds? How can design teams support effective collaboration between practitioners (teachers, professional development specialists) and researchers?
3. Managing design: What types of work plans, deliverables, and specific processes can be employed to manage the work of a curriculum design team?
4. Role of analysis and evaluation: What is the appropriate timing for and role of curriculum analysis, formative evaluation, and summary evaluation?
5. Learning objectives: What are appropriate ways (pragmatically useful, theoretically grounded) to articulate learning objectives to guide curriculum development? What processes enable designers to effectively and efficiently translate learning objectives into curriculum designs?
6. Trade-offs: How can design processes take into account trade-offs among design considerations in a way that reflects theory and research?

Guiding the Design Process in Theory

1. How can theory of teaching and learning be used to guide design? (Note that other sections of this agenda, e.g., Curriculum Materials for All Children, describe basic theoretical issues about how teaching and learning can occur with materials.)
2. Learning processes: How can use of learning theory and research be built into the process to guide the design of curricula?
3. Motivation and engagement: What are effective and efficient ways to incorporate motivational considerations into the process of designing curricula?
4. Social context: What are effective and efficient ways to incorporate theory and research on culture and social context into the design process?

Nature of Theories of Design

1. Process specification: How can a design process best be articulated by its creators to support its use by others?
2. Documentation and design research: How can the design process be documented in a way that provides an appropriate rationale for designs and that supports research on the design process itself?

Case Study Research Opportunities

By and large, research on curriculum development processes must be case-driven. Edelson (2002) argues that by engaging in design for real-world settings, researchers can develop theories about effective design processes. These processes can then be tested by attempting to apply them to designs that differ systematically from each other. In CCMS, we have access to a number of different curriculum design projects being undertaken by different researchers at different institutions. These projects offer the opportunity to conduct comparative case studies that can provide insight into the questions listed above. Within the contemporary science education community and the development projects associated with CCMS, we have several different approaches to curriculum development in use. These fall into two categories: frameworks describing a development process (e.g., backwards design, Herb Thier/Bennett Daviss book [Developing Inquiry-Based Science Materials: A Guide for Educators], AAAS draft chapter on materials design) and frameworks describing the product, or aspects of the curriculum, that can be used as an analytical tool (e.g., project-based science, Learning-for-Use, AAAS curriculum materials analysis criteria). Within CCMS research, we have the opportunity to study the use of these frameworks to conduct research both on the content and form of these design frameworks.

Assessment

Research related to student assessment is central to the work of the Center. Assessment is a critical component of instructional materials themselves and a necessary tool for conducting research on those materials. Assessment is also important for measuring what teachers know about content, pedagogy, and their application to the design of particular curriculum materials.

Assessment as a Component of Curriculum Materials
As a component of curriculum materials, assessment tasks serve to focus student attention on important ideas, and they allow teachers to make informed judgments about what students are thinking and what they know and can do at particular points in time. This allows teachers to adjust their teaching to match what students know and think.

1. What types of assessment items, embedded in curriculum materials, are most effective as probes of student understanding?
2. What is the best way to integrate assessment items in curriculum materials?
3. How are embedded assessment items used by teachers?
4. What else is needed, besides the items themselves, to improve their use?

Assessment as a Research Tool
High quality assessment tasks and assessment instruments also allow researchers to determine what students know and can do at particular points in time, which enables them to determine the impact of instructional interventions on student understanding of important ideas and to modify the design of those materials accordingly. Assessment instruments can be used as research tools to measure the effectiveness of activities, explanations, examples, and representations that are built into curriculum materials; strategic sequences; time spent on various activities; and other features of curriculum materials. On a larger scale, high-quality assessment instruments are needed to compare materials that are based on differing design principles and pedagogical assumptions. This requires instruments that are considered a fair measure of student understanding of important learning goals. Research is needed on how to create instruments that are acceptable to widely varying segments of the educational community. Crucial to research on student assessment are questions regarding the validity of judgments made from test results, policy implications related to both testing and the interpretation of test results, and professional development requirements related to assessment.

1. How is alignment of assessment with content standards defined and how do alternative definitions of alignment affect the validity of claims that can be made about student learning?
2. What are the factors that come into play when students face an assessment task—previous experiences, language facility, cognitive abilities, related skills, and features of the assessment item itself—and how do these factors affect the interpretation of what students know and can do?
3. How do various factors (validity, reliability, alignment, etc.) affect the level of confidence that is placed in assessment results?
4. What is the most effective way of stating learning goals both to guide instruction and to allow for the development of assessment items that are aligned with those learning goals?
5. To what extent and through what mechanisms does assessment, combined with instruction, affect student learning?

Policy Implications of Assessment
Policy considerations include the relationship between large-scale accountability measures and classroom-based and research-focused assessment. High-stakes assessment has the potential to drive both what is taught and how it is taught.

1. What factors affect what the public values with respect to educational outcomes and on the basis of what evidence do they make their judgments?
2. How well do local, state, and national assessments currently align with instructional practices and curriculum materials and with state and national content standards?
3. To what extent does assessment drive what is taught and how it is taught?
4. How can the assessment of curriculum materials provide direction to content and pedagogy as well as to the nature of high-stakes assessment itself?

Professional Development Implications of Assessment
Professional development is needed to demonstrate the effective use of assessment and the proper interpretation of assessment results.

1. What reporting mechanisms and information will allow teachers and other educators to use assessment results to make decisions based on those results?
2. What knowledge of assessment theory and practice do teachers need to effectively use embedded assessment to guide student learning and to modify their own teaching?

Policy

Research on educational policy involves the examination of procedures and regulations developed in the context of given conditions, which are used to guide, direct, or manage a course of action. Policy is derived from knowledge, beliefs, and values and is implemented through social structures, personal leadership, and political will. In relation to curriculum materials, it involves the world of publishing, funding agencies, adoption committees, school administrators, teachers in classrooms, as well as parents and other community members. It is concerned with the production and dissemination of materials, the adoption of materials, and the enactment and sustainable use of materials.

Policy Questions Related to the Guiding Principles of the Center
With respect to the conceptual framework of the Center, policy issues arise in the context of many of its guiding principles, for example: (1) the centrality of clearly stated learning goals, (2) the importance of building pedagogical supports into instructional materials, (3) the usefulness of student investigations, (4) the value of incorporating learning technologies into the instructional materials, and (5) the need to serve diverse learners by designing instructional materials that are accessible to all students.

1. How are the guiding principles of the Center interpreted and received in the educational community and what is the range of arguments that are made both in support of and in opposition to each?
2. What are the bases for these arguments, including personal value systems, beliefs about the effectiveness of the practices that result, and judgments about cost-benefit ratios and educational efficiencies?
3. What is the context in which policy decisions related to curriculum materials are made and carried out, including issues of leadership (power, influence, and who can effect change), systems organization (including management structures, funding resources, publishers), and obstacles to and support for implementation (values, beliefs, and attitudes of participants, and how policy decisions are received)?

Policy Questions Related to the Development and Publication of Curriculum Materials
Important policy issues come into play in the development, publication, and dissemination of curriculum materials. Developers and publishers are driven by what they believe to be the marketability of their product, the educational value of their materials, and current beliefs about what high-quality materials should look like.

1. How do both academic and commercial concerns shape science materials?
2. How can researchers work with developers and publishers to produce and disseminate more effective science materials?
3. What impact do state and national policies—content standards, accountability systems, certification requirements, standards for in-service, No Child Left Behind legislation, research funding, etc.—have on efforts to improve the curriculum materials that are developed to advance the state and national science education agenda?

Policy Questions Related to the Adoption of Curriculum Materials
The adoption of curriculum materials is based on beliefs about what is most effective educationally and what is acceptable in the local community. In adopting materials, individual teachers, districtwide selection groups, and state adoption committees use their knowledge of what makes curriculum materials effective and apply both formal and informal analysis criteria. Adoption of materials also depends on the effective communication of ideas about curriculum materials to others in the educational system.

1. How is leadership for materials selection distributed in schools? Who are the key players for adoption of curriculum materials? How are decisions made? How widespread is their impact?
2. What are the obstacles to and strategies for selecting high quality curriculum materials?
3. How do current policies support or hamper mechanisms for adoption of high quality curricular materials?

Policy Questions Related to the Enactment of Curriculum Materials
Many factors contribute to how effectively curriculum materials are enacted in the classroom. These include the knowledge that teachers have of the underlying rationale for the materials, formal structures such as district policies, and informal structures such as the support teachers receive from their colleagues and the community at large. Teacher knowledge and skill is addressed in the Teacher Learning section; here we focus on contextual and organizational issues that affect teachers’ enactment of curriculum.

1. What organizational and contextual issues (e.g., teacher community, support from parents, administrators, and other teachers) affect teachers’ attitudes toward and ability to work with innovative materials?
2. How can we create organizational contexts that support the ability of teachers to work with innovative materials?
3. How much do curriculum materials matter? How can the differential effects of materials, teachers, administrators, and organizational structures on outcomes be determined and categorized?
4. What factors influence our current conceptions of science curriculum materials? How are these conceptions related to broader educational policy issues such as our ideas about science, the role of schools, etc.?
5. What impact do state and national policies have on efforts of districts and schools to support teachers in their enactment of curriculum materials?