| Reprinted here with the permission of Science
News. No further republication or redistribution is permitted
without the written permission of the editor.
Source:
Science News - March 17, 2001 - Volume 159 - Number 11 |
 |
Errant Texts
Why some schools may not want to go by the book
By Janet Raloff
First of a two-part series on middle-school science curricula.
Last December, the National Center for Education Statistics unveiled a series
of disquieting findings from an international study of students' math and
science achievement. The data showed that among 38 surveyed nations, eighth
graders in the United States ranked no better than the middle of the pack.
Thirteen-year-olds in the United States outperformed peers in Italy, Lithuania,
Jordan, Turkey, Morocco, and South Africa, but they trailed their counterparts
in at least 17 countries, including Singapore, Japan, Australia, and the Slovak
Republic.
So, what's new? To many U.S. educators, there was something even more troubling
in this latest survey, a repeat of the Third International Mathematics and
Science Study (TIMSS). The new study found that the ranking for U.S. eighth
graders was lower than that of U.S. fourth graders 4 years earlier (SN: 10/19/96,
p. 244).
What this means, says Gary W. Phillips, acting commissioner of the National
Center for Education Statistics in Washington, D.C., is "that although
U.S. students learned a lot of math and science from the fourth to eighth
grade, the other nations learned more."
The big question is, Why? Many school administrators and especially scientists
are coming to the conclusion that one major problem resides in the textbooks
U.S. middle schoolers use. The science texts in most of their classrooms are
nothing short of an embarrassment, according to several recent studies.
The latest review of such texts, which focused on physical-science books for
sixth to eighth graders, grabbed headlines in January. "We were asked
to evaluate whether the information in the books was valid," explains
study leader John L. Hubisz of North Carolina State University in Raleigh.
"And the answer is, No!"
Carefully acknowledging that the 12 books he and his colleagues reviewed contain
many useful elements and passages, Hubisz says that each text was riddled
with errors, misconceptions, and confusing presentations.
The best-trained teachers may be able to guide students around such problems,
but they shouldn't have to, argues George Nelson, director of Project 2061,
a childhood-education program of the American Association for the Advancement
of Science in Washington, D.C. Even among top teachers, the vast majority
lack the time to develop an alternative curriculum, he says.
In fact, the latest TIMSS follow-up suggests that many U.S. math and science
teachers might not be up to the job. The study found that teachers in the
United States are less likely than their international counterparts to have
made science or math a "main area of study" in college.
Teachers who didn't major in science tend to "use textbooks—lean
on them—more than better-qualified teachers do," notes Arthur Eisenkraft,
president of the National Science Teachers Association and the science coordinator
for public schools in Bedford, N.Y.
At the moment, that kind of reliance on textbooks can pose particular risks.
Nelson, whose AAAS program has also reviewed texts, observes that "we
didn't find any good middle-school science books or good high-school biology
books." Indeed, he told Science News, "we will claim that
these books offer very little potential for having students learn, even if
they're used as they were intended."
One reason for this pessimism, he says, is that the books' deficiencies go
deeper than just errors of fact. These texts probably contribute not only
to the low performance by U.S. eighth graders in the TIMSS follow-up, Nelson
contends, but also to the waning interest in science that afflicts a growing
number of children beginning in middle-school years.
What most disturbs Hubisz and many others contacted for this story, is that
schools aren't more irate—and publishers more embarrassed—about
the quality of so many children's textbooks.
Hubisz, a physicist, has been informally reviewing precollege science texts
since the 1960s. His new study on middle-school books was sponsored by the
David and Lucile Packard Foundation of Los Altos, Calif., which also funded
a separate review of high-school physics texts.
After identifying some 30 books in common use in middle schools, Hubisz and
a team of reviewers focused their analysis on the dozen that had dominated
classrooms over the previous 5 years. They represented all major science-text
publishers. Each reviewer plowed through two books, line by line, with a simple
mission: Record every error, large or small. Their daunting compilation made
a list 500 pages long.
Diagrams often did not display what the text or caption indicated. Sometimes,
a book asked questions that were impossible to answer—either because
it offered too little information (such as giving values for two dimensions
when the student needed to compute a three-dimensional volume) or because
explanations necessary to solve a problem wouldn't appear for another couple
pages or even chapters. The texts depicted or defined a scientific principle
incorrectly with a disturbingly high frequency.
Though Hubisz pared the list of errors to 100 pages for his report, by this
summer he hopes to have the entire error-catalog loaded onto a new Web site.
Accompanying each error will be an explanation of how it could mislead or
discourage a student. The researchers' hope, Hubisz says, "is that this
will encourage everyone to read [all classroom materials] more critically."
Hubisz' team isn't alone in its negative critique of science texts. Roughly
18 months ago, Nelson's Project 2061 released its review of 10 middle-school
science books. In addition to some of the most widely used texts from major
publishers, it included a few new volumes from projects funded by the National
Science Foundation. The latter have the potential to gain widespread use in
classrooms.
Unlike the Packard study, Project 2061 "took a deep look at the philosophy
inside each book, asking how does it attempt to teach ideas," Nelson
says. These analyses explored how each book approached content and instruction
for six selected topics. For example, the study probed how a physical science
text uses the concept of invisible atoms and molecules to explain changes
of state and energy transfer.
The reviewers concluded, according to Nelson, that "even if the science
had been 100 percent accurate, students still wouldn't learn from these books,
because the instruction was inadequate."
Studies by education researchers indicate that to learn, students must confront
their preconceptions of how the universe works, compare these with what they
glean from books and hands-on experiments, and then discuss discrepancies
among themselves and with teachers. Middle-school texts don't foster such
reflective contemplation, Project 2061 found.
Rather, they present statements, questions, and experiments with little to
bind them into a whole. Indeed, many publishers crammed in legions of facts,
Nelson says, not seeming to realize that "encyclopedias aren't good textbooks."
None of these observations surprises William J. Bennetta. As president of the
Textbook League, a public-interest group in Sausalito, Calif., he's spent
the past 9 years recruiting experts to review textbooks used in his state's
middle and high schools.
Through its newsletter and Web site, http://www.textbookleague.org,
the league has issued some 200 reviews, primarily of science and math books.
Detailed and amusing, most find little positive to say about middle-school
books.
"Educators should avoid this book like the plague," screamed the
headline for the league's review of the 1995 edition of one Prentice-Hall
middle-school text, Exploring Physical Science. The book had recycled
"dreadfully wrong material" from earlier books by the same publisher,
said the review.
Hubisz' team also reviewed the same edition of Exploring Physical Science,
as well as updates of the book from 1997 and 1999. In even the latest edition,
the reviewers found hundreds of errors.
Books for middle-school students seem to be particularly problematic. High-school
texts are far better, says Clifford Swartz, a high-energy physicist at the
State University of New York, Stony Brook, who headed the Packard Foundation-financed
analysis of seven high-school physics texts. His team of 14 reviewers reported
its findings in the May 1999 The Physics Teacher, a magazine that Swartz
edited for 29 years.
"Compared with books at the middle school level, these got a relatively
clean bill of health," he says, "with only one having really major
mistakes in physics." He attributes the generally sound—if, at times,
lackluster—science in these books to the input of physicists during the
texts' writing and production.
Hubisz says that such input from scientists appears to be absent in the middle-school
texts that his group surveyed, even though the books often listed an impressive
lineup of scientists as contributors. In fact, he notes, many of these apparent
authors "did not even know that their names had been listed." A
few argued that their only contribution had been to review the text—which,
ironically, they had "panned," says Hubisz.
He and others focusing on middle school science books find that they're typically
authored by committees of contract writers that may wield little control over
the final product.
Jennie Dusheck, a California-based biology writer, knows what it's like to
work on a middle-school book. "As the author of a college textbook, I
ultimately have authority over—and responsibility for—every word.
If I don't like something [that editors have inserted], they will take it
out," she says.
Not so for middle-school books. A publisher's relationship with these writers
"is so profoundly different," Dusheck finds, that "I would
hesitate to call [the writers] authors." In these projects, a publisher
typically hires an author to write a segment of a book, then someone else
edits it to conform to segments produced by other contract writers. She recalls
reviewing one of her contributions after it had been through this process
and asking, "Did I write this?"
Here, Dusheck argues, "the editor is really the author." The editor
sets the book's tone and commissions elements to adorn the text: photos, sidebars,
puzzles, and graphs. In Dusheck's experience, these peripheral elements are
where errors most often crop up.
Yet even when errorfree, Hubisz found, these attractive additions can diminish
young students' understanding of science. Middle schoolers aren't yet adept
at integrating information from a collection of elements, he says. He speculates
that too often, these young students focus on the tangential items and miss
entirely the main point of an adjacent paragraph. In one book, for example,
Hubisz found a sidebar on careers in jewelry-making in a chapter on the properties
of metals.
With disparate elements from a host of authors and artists being cobbled into
these books, it's not hard to imagine how mistakes occur. So, it's especially
important that scientists review the material throughout a book's many stages
of production, argues Kenneth Ford, retired director of the American Institute
of Physics in College Park, Md. Ford has taught extensively, from the university
level down through ninth grade.
Ironically, he observes, publishers of elementary and middle-school science
books are less likely than high school and college publishers to enlist professional
scientists to vet a book before its printing. Trying to fix mistakes after
publication—for future editions—can prove next to impossible. He
relates his own experience with an Addison-Wesley book he had criticized.
The company offered to pay him to highlight the book's problems and possible
solutions for a revised edition. However, after working on some 100 pages,
Ford gave up. "The book couldn't be fixed," he maintains, "because
it was too deeply flawed—from beginning to end."
He notes that the book, which became a big seller, was later reviewed by Hubisz'
team. Its assessment: "No student will increase his or her understanding
of science by using this text."
Stephen Driesler, executive director of the Association of American Publishers'
school division, insists that text publishers try hard to ensure accuracy.
They "go through extensive efforts of proof checking, fact verification,"
says Driesler. "I have seen where literally every page of the text where
there's a fact will have to be accompanied by at least two independent sources
verifying that fact . . . . Notwithstanding, any system devised by humans
and implemented by humans is subject to things slipping through the cracks."
If errors occur and reviews such as the ones by Project 2061 and Hubisz' group's
are documenting them, why do schools continue to buy error-riddled books?
The answer, says Driesler, is that these books cover more of the topics state
curriculum committees have asked for than the more-accurate books do. "It
is the states who are the [textbook publishers'] customers," he observes.
Increasingly, states are assessing the performance of students and schools
with standardized tests. To make sure students encounter exam topics before
they actually take the tests, curriculum committees issue detailed lists of
what they want in textbooks. Twenty-one states now adopt lists of approved
texts. Public schools in these states can get state funding to buy those books.
Schools must dig into their own pocket for books not on the list. Those states,
which include the "big-volume buyers" [California, Texas, and Florida]
"really drive the publishers," Driesler says.
There is no reason why a book that includes all the topics requested by a state
can't also be accurate and simple to understand. However, Driesler argues,
"I would have a hard time—and I think [Project 2061] would have
a hard time—pointing you to a textbook that does both."
In an effort to change that, Project 2061 has begun working with states and
publishers on better criteria for middle-school textbooks. Last month, Nelson's
group hosted its first conference at which educators, publishers, and state-curriculum
officials convened to explore ways to overcome their sometimes conflicting
objectives.
As for factual errors of science, Driesler notes, "the publishers' goal
is zero." One remedial step that publishers are beginning to take is
to post textbook errors and corrections on their Web sites, Driesler says.
Last fall, his organization also launched Accuracy E-line, a Web site where
anyone can log reports of apparent errors in any textbook.
Although better and more accurate textbooks should be a goal, Nelson cautions
that good books alone are not likely to propel science-achievement scores
of U.S. students to the top of the international charts.
To make that happen, teachers will need more and better training, he argues.
His own organization holds science workshops for classroom teachers. Traveling
to schools around the country, science-education experts offer tips and strategies
for making the best use of currently available materials. This spring, Project
2061 will also host regional, 3-day workshops in about a dozen cities.
Others, such as Paul Hickman of the Center for the Enhancement of Science and
Mathematics Education at Northeastern University in Boston, are considering
more-radical changes in science and math education. The ultimate answer to
avoiding erroneous information in classrooms may not be to fix textbooks,
says Hickman, but to do away with them entirely. Increasingly, small publishers
are starting to offer curriculum packages that include experiment kits, explanatory
booklets, and other nontraditional materials.
Mistakes from a science-text survey
By surveying a dozen physical science textbooks, in some cases multiple editions
of the same book, John Hubisz of North Carolina State University in Raleigh
and his colleagues logged 500 pages of errors. Some were silly, others substantive.
All, however, run the risk of making science appear confusing or even nonsensical
to students, warns Hubisz. Below are some examples of errors identified in
the texts.
- One book said: "Unlike fission, fusion doesn't happen spontaneously."
Yet fusion reactions power the sun. The sentence should have been amended
with: "...at temperatures usually found on Earth."
- This book also stated that the acceleration of gravity on the moon is
one-sixth that on Earth because the moon's mass is one-sixth of Earth's.
In fact, the moon's mass is roughly one-eightieth of Earth's and the initial
statement ignored the fact that the acceleration due to gravity is also
related to the radius of a body.
- One book defines some elephant vocal sounds at around 400 hertz—below
a frequency perceptible to human ears. However, 400 Hz is about the frequency
produced by keys on the middle of a piano.
- One text shows a photo of rubber—with a density of 1.19 grams per
cubic centimeter—sinking to the bottom of glycerin, which has a density
of 1.26 g/cm3. As the reviewer points out, "It
cannot happen!"
- One passage notes that "sound travels faster through warm air than
through cold air." Twelve pages later, the same book notes "...but
sound travels faster in cold air." You can't have it both ways.
For references and sources for this article please visit
Science News Online.
Raloff, J. 2001. Errant Texts: Why some schools may not want to
go by the book. Science News, 159 (11).
