Project 2061

An electronic newsletter for the science education community

May/June 2005

Goals-Based Mathematics Study Produces Unique Data Set
Videotaped lessons aid analysis of changes in teacher practice

At the end of three years of data collection for Project 2061’s study Improving Mathematics Teacher Practice and Student Learning through Professional Development, researchers have collected hundreds of hours of carefully selected videotaped lessons taught in middle school mathematics classrooms. Project 2061 researchers have recorded the lessons to explore how the interactions of curriculum materials, teaching practices, and professional development can lead to improved student learning in mathematics. The five-year study—being conducted in partnership with the University of Delaware and Texas A&M University—is funded by the Interagency Education Research Initiative (IERI), a joint program of the National Science Foundation, the Department of Education, and the National Institute of Child Health and Human Development. (Read an overview of the study.) Preliminary analysis of the classroom videotapes provides insight into possible answers to one of the study’s questions: How does professional development and ongoing support that is focused on specific mathematics learning goals build teacher knowledge and lead to more effective teaching practices?

To detect any changes in teacher behavior as a result of the study’s professional development interventions, researchers videotaped teachers over the course of two or three years teaching the same set of 3–5 lessons. The lessons were selected because they address important concepts and skills related to Number, Algebra, and Data. These same mathematics learning goals were also the focus of the study’s professional development and its student assessments. While there are other collections of videotaped mathematics lessons, this one is unique in documenting how teachers’ approach to the same set of lessons, which targets specific learning goals, changes over time and in response to specific kinds of feedback and support. The videotaped lessons are also being correlated with test data showing the extent to which each teacher’s students have achieved the targeted learning goals. When complete, the study results will provide an unmatched resource for mathematics educators and researchers.

About the Professional Development
Prior to the videotaping, teachers participated in a two-day workshop that included an introduction to and clarification of the Number, Algebra, and Data learning goals being targeted by the study. Videotaped lessons from the first two years were then used in the design and delivery of ongoing professional development. Videotapes from Year 1 revealed trends in teacher practice across teachers and curriculum materials. Based on those first year trends, researchers chose to concentrate professional development for the subsequent years on three areas where research has shown that teachers’ practices can make a significant difference in students’ mathematical understanding: (1) teachers’ use of mathematical representations, (2) teachers’ ability to probe their students’ understanding, and (3) teachers’ ability to guide their students’ interpretation and reasoning.

Professional development following the second year of videotaping built on the three areas studied the previous year, and was expanded to help teachers examine evidence of student understanding. Teachers viewed the videotaped lessons to analyze their own and others’ practices, focusing particularly on the extent to which students were encouraged to express their ideas in the classroom and to demonstrate their reasoning. They also looked at student responses to specially-designed assessments to see how well students were learning the mathematics in the target learning goals.

As we wrap up three years of data collection, we are in the early stages of data analysis. We now have the video data from two or three years in a classroom that we can examine for change in a given teacher’s practice. Researchers have begun to take a qualitative look at some of the teachers’ videos, while a quantitative analysis will be undertaken with the assistance of a Web-based computer program designed by Project 2061 for this study. The program allows analysts to document what is happening in lesson segments, to rate the teaching on very specific indicators related to criteria for evaluating teacher practice, and to produce numeric ratings and detailed summaries and justifications. The preliminary results from one case study illustrate the kinds of changes in teacher practice we hope to find in the videos, which we hope will correlate with the student assessment results.

Mr. Smith* Makes a Leap: A Case Study
Mr. Smith is a veteran mathematics teacher in a rural-suburban school in the middle Atlantic region of the U.S. He is experienced using a mathematics curriculum material that was highly rated in Project 2061’s middle grades mathematics textbook analysis. Mr. Smith has participated in professional development for three years and provided researchers with a three-year video record of him teaching the same lessons in his sixth grade mathematics classroom. The lessons focus on the learning goal stating that students should be able to “use...numbers in several equivalent forms such as integers, fractions, decimals, and percents” (AAAS, 1993, p. 291; see benchmark 12B6-8#2). While the design of the lessons intends that most of the class period be used by students to explore the mathematics through activities in the curriculum material, researchers found a very different climate in Mr. Smith’s classroom in the first two years of videotaping.

For the purpose of illustration, we will focus on one lesson. The lesson is designed to have students investigate the concepts of comparison and equivalence of fractions. Our first analysis of the lesson was qualitative in nature. Researchers simply watched the videos from the three years and characterized the nature of the teaching each year.

The videotapes in Years 1 and 2 show the teaching to be strictly teacher-directed: Mr. Smith does most of the talking and does not listen to the students. In some instances when he asks for student input, he re-voices student responses to articulate his understanding or his agenda. In both years, he adds in the same “extra activity,” one that is not in the material. While this activity is not incorrect, it leads to confusion on the part of students and detracts from their understanding of the mathematics. The “Launch” or introduction to the lesson and the mathematics is lengthy, while the “Explore” part of the lesson is brief, even though the curriculum intends it to be a significant part of the class period in which students have the opportunity to work with and reason through the mathematics. As a consequence, students have limited opportunities during the lesson to contribute and express their understanding. The few interactions that the teacher does allow with students are brief, on average less than 1 minute in length.

In Year 3, a metamorphosis appears to take place. Students take the stage with a voice and a more active role in their learning. Mr. Smith is listening to them, as a whole class and in small groups, and is using evidence from what students say to move the lesson forward. The introduction to the mathematics is brief and clear, and students’ exploration of the mathematics expands, as designed in the curriculum. There are many opportunities for students to express their understanding and the teacher’s interactions with students expand to an average of 3 minutes each. Mr. Smith exhibits confidence in his knowledge of the mathematics and the lesson.

After gaining this qualitative perspective, researchers moved to a quantitative analysis—a necessary step in order to examine a teacher’s practice, during a single class and from one year to the next, with an objective lens. Researchers wanted to find out (1) if the quantitative analysis would capture the character of the teaching that was suggested in the preliminary examination of the videos; (2) if the quality of the teaching from year to year could be differentiated by the ratings of the criteria; and, in the case of the criterion “Encouraging students to explain their ideas,” (3) if the indicators of meeting the criterion would capture a change in the quality of student-teacher interactions. Researchers took this opportunity to determine if the Web-based program being used for quantitative analysis would corroborate the characterization of the teaching.

“Encouraging students to explain their ideas” is one of a subset of five criteria researchers selected for the quantitative analysis. This criterion requires that the teaching routinely encourage each student to express, clarify, justify, interpret, and represent his or her knowledge/understanding about the mathematics and provide accurate and helpful feedback to students. When Mr. Smith’s lessons over three years were measured against this criterion using the quantitative analysis, researchers found that preliminary results do indeed bear out the initial characterization, as illustrated in the following graphic.

Each bar in this chart represents one class period, for the same lesson, for each of the three years. The dark dotted lines denote the beginning and end of the main parts of the lesson (i.e., the teacher’s introduction of the lesson and the student exploration of the mathematics) and the light dotted line denotes the point at which the teacher moves from the introduction to the exploration. The colors coordinate with the ratings to illustrate the quality of student-teacher interactions when students are explaining their ideas and understanding about the mathematics.

In Years 1 and 2, there is a lot of time during which there is no student interaction with the teacher, as shown by the white segments. The teacher is talking; students are not contributing to the conversation. The shaded segments show that students had a few opportunities to share their ideas and understanding, but for the most part the quality of those comments is fairly low, as represented by the light shading. These instances may have been times when students were asked to respond to a question but were not asked to give further explanation or clarification; when only one or two students were called on; or when the teacher did not provide helpful feedback.

In Year 3, the frequency and quality of student talk is dramatically increased. The lesson introduction is brief, students move quickly into a lengthy exploration of the mathematics, and the quality of the student talk is high throughout most of the lesson, as denoted by the dark shading.

While this is only preliminary and partial data of one case study, researchers are encouraged by the results, both in the change in the teacher’s practice and in the way that the change has been captured using our analysis tool. As third year assessment results are scored and analyzed, a clearer picture should emerge about how changes in teacher practice influence student understanding of the mathematics.

* Teacher’s name has been changed.

# # #

For more information about this study, please contact:

Senior Program Associate: Dr. Kathleen Morris
Principal Investigator: Dr. Jo Ellen Roseman, (202) 326-6666

---

---

[Table of Contents]