Proceedings of the Second AAAS Technology Education Research Conference

Goals and Technology Education: The Example of Design Challenges

 

Marc Schwartz and Philip M. Sadler

Harvard-Smithsonian Center for Astrophysics

 

For the last six years, we at the Harvard-Smithsonian Center for Astrophysics have been studying activities for students involving design challenges. We and many of our colleagues have found such activities to be very popular with students in grades six to nine; however, we have been curious about what makes these activities different from other hands-on activities. What do students learn from participating in design-based curricula, and how do curricula that contain these activities fare when compared to more traditional curricula that attempt to deliver the same content? In this paper we address both questions in an attempt to understand the nature of these activities and to provide a justification and strategy for further development of technology based curricula.

The Deceptive Nature of Goals

 

The success in design-based activities share some of the same underlying principles used in video games that have also succeeded in capturing the attention of many. Both activities provide clear goals, they suggest obvious ways of taking action, and offer feedback that the student/player can translate into adjustments they need to make in order to reach the goal. As an example, consider Tetris, a game children and adults spend hours, days, and months playing. The goal of Tetris is to align each tetris (patterns made of four attached squares) as it moves down the screen so that by the time they reach the bottom they are aligned in an unbroken chain of squares. The player rotates or horizontally translates each tetris with one of two toggle switches. If the player successfully completes a row of squares they disappear, if not, the partially completed row stays and the play space becomes a little smaller. When the number of partially completed rows fills the screen the game ends and your tabulated score remains. Figure 1 shows one sequence where the player successfully maneuvers the tetris labeled "A" into a partially complete row, and at "time 4" the row disappears.

Figure 1 : Tetris

 

We have observed an unbroken chain of people in airports, hotel lobbies, and classrooms playing Tetris. Their heads are fixed in strained concentration. The only sign of life is their index finger moving one of the two toggle switches. They appear oblivious to outside noises or the passage of time. They stay involved for extended periods of time. Csikszentmihalyi and Rathunde described such involvement as being in "flow:"

 

In flow, a person is fully concentrated on the task at hand. There is a feeling that action and awareness merge in a single beam of focused consciousness. In flow, it is very clear what needs to be done from one moment to the next; goals are clearly ordered and sequenced. One also knows immediately how well one is doing: feedback is unambiguous. The tennis player knows whether the ball was hit well, the violinist hears whether the note just played was right or wrong. (1998, p. 646)

 

What is it about this simple game that captures a player's attention so successfully that it maintains flow?

 

Our understanding of this phenomenon has been framed by the work of Powers (1973, 1998) who introduced Perceptual Control Theory (PCT) as a means of understanding human behavior. From the perspective of PCT, all forms of life are viewed as control systems. Control systems choose their own goals. They also choose actions to achieve those goals, and they adjust those actions on the basis of their analysis of the success of the actions they choose to reach their goals. Goal, action, feedback are the essential elements of this model. Although this sequence sounds deceptively simple, we as educators and curriculum designers have commonly failed to appreciate its importance in designing lessons. A simple illustration from the aviation world may help understand why.

 

Consider an important pilot task-landing the plane. From the perspective of PCT, her goal is not really to land the airplane. Landing sounds like a goal, and from the perspective of the passengers this final step sounds like a most desirable goal. However to dramatize this important point, would this "goal" be useful to a novice that we put into the pilot's seat? "Landing" fails as a goal because it does not help the novice decide what to do as he approaches the runway. At best, landing becomes his most passionate wish as he struggles with the controls of the plane. So, if landing is not the goal for the novice or even the pilot, what is the goal?

 

For an effective pilot, the goal is to match a reference picture of what she sees outside her cockpit with what she wants to see. The goal allows her to act in a manner that brings the real world into alignment with her mental picture of the world she wants to see as she approaches the runway. Any time the internal picture and real world picture are out of alignment the pilot adjusts the aircraft. Thus, the wish to land shifts to goal status when the reference picture the pilot wants becomes specific enough that she can choose the appropriate actions to bring the airplane into a position that allows her to see what she wants to see. Thus landing is not the goal; not even landing in the middle of the runway or in the middle of the first three thousand feet of the runway. The goal is to put the plane on the large numbers that are painted at beginning of runway. This final reference picture is specific enough that the pilot can judge how well she is managing to keep what she sees in accordance to what she wants to see. Powers (1973, 1998) calls this behavior "controlling for your perception." Being able to do coordinate these actions is an empowering event, and one that maintains flow as long as the pilot has the skills to maintain the picture she seeks.

 

To summarize, an effective goal provides a specific picture of the world that allows the individual to choose strategies he believes will close the gap between what he sees and wants to see. An effective goal is one that allows the individual to judge the success of the strategies selected. The goal must "call us forth into action" (Gregory, 1998). Any goal that does not stand up to this standard does not have enough detail to function as a goal. Furthermore if the goal fails to help the individuals see ways to take action, or allow them to judge the effectiveness of any action they choose, then the goal degenerates to a "wish." Wishes, like goals, suggest that action is necessary, but not necessarily the type of action needed (Schwartz, 1998). (See Figure 2.)

 

Figure 2 : Goal vs. Wish

	Goal	Action	Feedback
The Wish*	Suggests that action is needed, but not a specific action	Action needed is clear, but individual does not have the necessary skills	Feedback is not obvious or not clear
Perceptual Control Theory (Powers)   Flow (Csikszentmihalyi)	Action needed is obvious	Action needed can be executed by the individual	Feedback is interpretable and allows individual to recognize necessary correction to action to achieve goal

* Goals degenerate into wishes when any of the three requirements for a successful goal are not met

 

Goals and Classrooms

Goals show up so often in the educational lexicon that for most people they signify nothing more than the beginning of some operation or a wish. Educators talk about them and textbooks begin with them; however, students and teachers rarely know how to respond or interact with the stated education goals. In conversation with educators about their goals, we are often met with a look that suggests that the goals are obvious. That may be so, but the problem is often that their students do not know what the goal is, what action is necessary, or how to interpret the outcome of their actions. Unfortunately their teachers operate on the premise that the goal is just as clear and meaningful to their students as to them. Teachers are able to choose the appropriate strategies to meet their goals and can effectively judge the effectiveness of their actions, but how about their students?

 

During the last five years we have had the opportunity to ask students in many of the classrooms what they thought the goal of the activity was. Almost invariably students told us that the goal was to follow the instructions. What our students are learning to do is follow instructions. As Holt (1964) observed nearly forty years ago, the goal from the student's point of view is to figure out ways of pleasing their teachers. Students use a variety of strategies that they learn from life, and to help them judge whether they are successful they need only evaluate their teacher's reaction or their grade. For students who decide that they do not want to please their teachers there are always plenty of alternative, non-educational goals that they can adopt to pass the time (often resulting in disciplinary issues).

What Have Students Learned in Design-based Activities?

What design-based activities offer students are opportunities to remove the teacher from the feedback loop and recognize a different kind of classroom goal. This kind of activity may, for the first time in a student's school experience, offer a goal that is so clear that there is little question about what he or she needs to do. Properly managed, the design-based activity invites students to use strategies they think useful, and if teachers exploit the iterative possibilities of such activities, then students can respond to the feedback that is generated from the strategies they choose. In essence, design-based activities allow students to capitalize on the three elements identified in Perceptual Control Theory. For many students, this type of activity is their first experience of flow in schools.

 

Emotionally the experience is considered a success from the student's and teacher's perspective. Students discover that they can be successful in school. Participating in such activities feels empowering. As a result of this kind of success they want to take part in more activities of this kind. However, as rewarding as this experience is for students, more vexing problems remain. What did students learn? Was the time well spent?

 

How do Design-Based Curricula Compare to More Traditional Curricula?

During the last two years we have been comparing what students learn in different types of curricula that ostensibly cover the same material. For this study, we chose a chapter in electromagnetism from a middle-school science textbook to test three different curricula: one that involves design-based activities, a more traditional curriculum with laboratory experiments, and a discovery-based curriculum. We view the major difference in each set of activities as the amount of responsibility students have in choosing their own goals and the strategies for reaching those goals. Each curriculum can also be characterized as fitting on a "responsibility" continuum (Figure 3).

 

Figure 3 : The Goals and Responsibility Continuum

At one extreme, the discovery curriculum allowed students to choose both their own goals and strategies for attaining those goals. Students were given materials and resources to work on electromagnets, make discoveries at their own pace, exploring phenomena that they found interesting. At the other extreme (in the traditional curriculum) students were not allowed to choose goals or strategies to reach those goals. The teacher or the curriculum designers made those choices. Readings, worksheets, questions, and procedural laboratory experiments were taken from the text or supporting materials. Somewhere closer to the middle of the continuum is the design-based curriculum where teachers chose the goal, but students chose the strategy for reaching those goals. Our goal is to create a design challenge that embodies a goal that students can understand. In this case, students are invited to examine a prototype electromagnet and then make decisions about how they might improve it (Figure 4). A further step in this curriculum is to build follow-on activities that are in themselves challenges that invite students to build on what they discover during earlier iterations of design work.

Figure 4: The Prototype Electromagnet

 

 

Our Study

All three curricula investigated the same content in the same number of days. One hundred twenty-five students in two middle schools in two different states participated. The two teachers who participated in the study taught three different classes and used a different curriculum with each group. There were four principal activities in each curriculum and at the end of each activity (that lasted one to two days) students were given a concept assessment questionnaire to measure student progress (Figure 5).

Figure 5: General Path of Instruction and Evaluation for each Curriculum

 

 

 

Each curriculum (Discovery, Traditional, and DESIGNS) was designed twelve weeks prior to its use in September 1999 by both teachers who took part in the study, an additional middle school teacher from New Hampshire who had been with the project for two years, and myself. Our goal was to make sure that the pedagogy used to create each activity fit the philosophy of the curriculum proposed. After collecting inputs from all three teachers, we prepared each curriculum for review six weeks later. Further fine-tuning of activities was accomplished at this time. The first activity for all three groups was an orientation to permanent and temporary magnets. Since the activity did not involve electromagnets it was referred to as a "pre-activity" to the electromagnet curriculum. The three follow-on activities for all groups focused on electromagnetism (Figure 6).

 

Figure 6: Content of Each Curriculum

TRADITIONAL		DESIGNS		DISCOVERY
PRE-ACTIVITY Lecture: The history and nature of permanent and temporary magnets		PRE-ACTIVITY Challenge: Improve both permanent and temporary magnets.		PRE-ACTIVITY Explore permanent and temporary magnets.
↓		↓		↓
ACTIVITY 1 Lecture: The variables that affect electromagnets		ACTIVITY 1 Challenge: Improve the prototype electromagnet		ACTIVITY 1 Explore or improve the prototype electromagnet
↓		↓		↓
ACTIVITY 2 Discussion: Students answer questions bsed on lecture, and the discuss answers.		ACTIVITY 2 Discussion: Students explore questions generated by their experiences.		ACTIVITY 2 Continue to explore or improve the prototype electromagnet
↓		↓		↓
ACTIVITY 3 Lab: Explore the nature of one variable in electromagnets		ACTIVITY 3 New challenge: Use controls to find the most important variable		ACTIVITY 3 Presentation of findings

Results

 

The concept questionnaire was given four times during the study. Students in each treatment group improved after the first activity, but only the DESIGNS and Discovery groups showed significant gains after the first activity [DESIGNS: t = 2.19 (1, 36) p<.04; Discovery: t = 2.24 (1, 42) p<.03]. However, more importantly, only the DESIGNS group continued to improve after each activity. Although the improvement in mean scores as measured by Tests III and IV was not significant, the overall gain in DESIGNS scores after all three activities was significant [t = 3.21 (1, 36) p<.003]. These results were unique to the DESIGNS group. Only the DESIGNS students demonstrated a significant gain in understanding by participating in the study. The three different groups showed different patterns of gains that can be summarized as follows:

• The Discovery group showed a large initial gain in conceptual understanding followed by a slow reversion to their prior ideas. They ended with no significant gain in understanding.
• The Traditional group made no significant progress at all. Scores for the Traditional group never varied by more than one or two percent. The activities had limited impact on how students in this curriculum thought about electromagnetism. The results raise serious doubts about whether the overall curriculum helped students understand electromagnets.
• The DESIGN group showed gains at each step, no reversion, and an overall significant improvement in understanding by the end of the curriculum.

Future Implications

 

So what should be the goal of technology education? Is the goal for students to demonstrate the effective use of specific skills in specific situations or for students to be able to transfer a smaller set of skills to a number of different contexts?

 

In the past, technical educators defined skills as specific abilities such as adjusting a carburetor, repairing a toaster, or building a stool. This view of skills did not pose a problem in a world where there were fewer technical innovations than today. If we believe that the goal of a technical education is to transfer skills as they were defined 25 years ago, we would leave teachers and students exasperated. Fixing a toaster does not lead to marketable skills in a world where students use software to define or solve problems they encounter today.

 

Today, not only has the number of possible technical skills multiplied, but the term "skill" itself poses a challenge to our understanding of what it means to understand. Another theoretical framework, skill theory (Fischer, 1980), was used to analyze the developmental demand of each activity. At issue is defining skills in a manner that allows us to quantify what students have accomplished as they try to understand their world. We propose looking at skills from a developmental point of view to allow educators to understand what is possible from students at different points in their developmental trajectory, as well as allow educators to identify the foundational skills available and appropriate for transfer. Further work will be needed to match the developmental needs of students to activities that have specific developmental demands. Ultimately we want a design-based curriculum that allows students to build upon what they discover as they improve their designs.

Conclusions

 

Design-based activities are well understood from the perspective of two cognitive models: PCT and Flow Theory. Both models help us understand why design activities are so popular. However, design-based activities alone do not constitute a curriculum. Future work in design-based curricula will focus on skill levels. A skill framework is helping us develop strategies for analyzing what students understand, what steps are necessary to enrich student understanding, and what constitutes scaffolding to support this growth.

 

References

 

Csikszentmihalyi, M., & Rathunde, K. (1998). The development of the person: An experiential perspective on the ontogenesis of psychological complexity. In W. Damon and. R. Lerner (Eds.). Handbook of child psychology: Theoretical models of human development (Vol. 1). New York: Wiley.

 

Fischer, K. W. (1980). A theory of cognitive development: The control and construction of hierarchies of skills. Psychological Review, 87, 477-531.

 

Gregory, B. (1997). Tetris: a way of understanding the role of goals in perceptual control theory. Personal communication.

 

Holt, J. (1964). How children fail. New York: Dell Publishing Co.

 

Powers, W. (1973). Perceptual control theory. Hawthorne , NY: Aldine DeGruyter.

 

Powers, W. (1998). Making sense of behavior: The meaning of control. New Canaan, CT: Benchmark Publications Inc.

 

Schwartz, M. (1998). The role of standard designs in goal setting in a science activity. Unpublished qualifying paper, Harvard University, Cambridge, MA .

 
Acknowledgements

 

This work was supported by the National Science Foundation (ESI-9452767 and ESI-9730469). The ideas expressed in this paper are those of the authors and not those of the National Science Foundation. Special thanks to Dr. Gerhard Salinger for his feedback and constructive ideas. Thanks to our colleague, Bruce Gregory of the Center for Astrophysics, for this insight into Tetris. Special thanks to Marcus Leiberman and Annette Trenga, who conducted evaluation activities and DESIGNS team members Harold Coyle, Kerry Rasmussen, Francine Rogers, Cynthia Crockett, and Anila Asghar. We thank the DESIGNS teachers: Stephen Adams, Marilyn Benim, Anne Brown, Nancy Cianchetta, Carolyn Fretz, Mary Ann Guerin, Anton Gulovsen, Kimberly Hoffman, Marti Lyons, Teresa Jimarez, Paul D. Jones, David Jurewicz, James Kaiser, Milton Kop, Laura Kretschmar, Barbara Lee, James MacNeil, Linda Maston McMurry, Daniel Monahan, Sarah Napier, Doug Prime, Diana Stiefbold, and Mary Trabulsi for their innovative ideas and creative teaching.